PM design question: Scopes
Guy Steele
guy.steele at oracle.com
Mon Nov 20 18:33:15 UTC 2017
I like this. One question: what does this new theory have to say about the situation
switch (x) {
case Foo(int x):
int y = x;
// fall through
case Bar(int x, int y):
…
}
? Perhaps it is forbidden because the “int y” in the pattern would shadow the “int y” in the earlier declaration? Or can the two be merged?
—Guy
> On Nov 20, 2017, at 1:17 PM, Brian Goetz <brian.goetz at oracle.com> wrote:
>
>
> We had a long meeting regarding scoping and shadowing of pattern variables. We ended up in a good place, and we were all a bit surprised at where it seems to be pointing.
>
> We started with two use cases that we thought were important:
>
> Re-use of binding variables:
>
> switch (x) {
> case Foo(var a): ... break;
> case Bar(var a): ...
> }
>
> Short-circuiting tests:
>
> if (!(x matches Foo(var a))
> throw new NotFooException();
> // use a here
>
> We had a few nice-to-haves:
> - that binding variables should be ordinary variables, not something new;
> - that binding, when assigned, be final
>
> Where we expected to land was something like:
> - binding variables are treated as blank finals
> - binding variables are hoisted into a synthetic block, which starts right before the statement containing the expression defining the binding
> - it is permitted for locals to shadow other locals that are DU at the point of shadowing. (This, as a bonus, would rescue the existing unfortunate scoping of local variables defined in switch blocks.)
>
> We thought this was a sensible place to land because it built on the existing notion of scoping and local variables. The remaining question, it seemed, was: "where does this synthetic scope end."
>
> First, a note about where the scope starts. Consider:
>
> if (e1 && x matches Foo(var a)) {
> ...
> }
>
> Logically, we'd like to start the scope for `a` right where it is first declared; this is how locals work. But, if we want to maintain the existing concept of local variable scope, it has to start earlier. The latest candidate is right before the if starts; we act as if there is an invisible { ... } containing the entirety of the if statement, and declare `a` there.
>
> This means, though, that the scope of `a` includes `e1`, even though `a` is declared later. This is confusing, but maybe we can ignore this, and provide a clear diagnostic if the user stumbles across it.
>
> So, where does the scope end? The obvious candidate is right after the if statement. This means `a` is in scope for the entire if-else, but, because it is DU in the else-blocks, can be reused if we adopt the "shadowing OK if DU" rule.
>
> FWIW, the "shadowing ok if DU" rule is clever, and gives us the behavior we want for switch / if-else chains with patterns, but has some collateral damage. For example, the following would become valid code:
>
> int x; // declared but never used
> float x = 1.0f; // acceptable shadowing of int x
>
> Again, maybe we can ignore this. But where things really blew up was attempting to handle the short-circuiting if case:
>
> if (!(x matches Foo(var a))
> throw new NotFooException();
> // use a here
>
> For this to work, we'd have to extend the scope to the end of the block containing the if statement. Now, given our "shadowing is OK if DU rule", this is fine, right? Not so fast. In this simpler case:
>
> if (x matches Foo(var b)) { }
> // try to reuse b here, I dare you
>
> we find that
> - B is neither DU nor DA after the if, so we can't shadow it;
> - B is final and not DU, so we can't write to it;
> - B is not DA, so we can't use it.
>
> In other words, B is a permanent toxic waste zone, we can neither use, nor redeclare, nor assign it. Urk.
>
> Note too that our scoping rule is not really about unbalanced ifs; it's about abrupt completion. This is reasonable too:
>
> if (x matches Foo(var a)) {
> println("Matched!");
> }
> else
> throw new NotFooException();
> // reasonable to use a here too!
>
> Taking stock: our goal here was to try and use normal scopes and blank final semantics to describe binding variables, out of a desire to not introduce new concepts. But it's a bad fit; the scope may be unnaturally large on the beginning side, and wherever we set the end of the scope, we end up in a choice of bad situations (either something we want in scope is not, or something we don't want in scope is.) So traditional scopes are just a bad approximation, and what we gain in "reusing familiar concepts", we lose in the mismatch.
>
>
> STEPPING BACK
>
> What we realized at this point is that the essence of binding variables is their _conditionality_. There is not a single logical old-style scope that describes the right set of places for a binding to be in scope, but there is a well-defined control-flow analysis that tells us exactly where we can use the binding, and where we can't. This is the flow-scoping construct we initially worried was too "new and different." But, after some further thought, and a few tweaks, this seems exactly what we want, and I think can be made understandable.
>
> The basic idea behind flow-scoping is: a binding variable is in scope where it is well-defined, and not in scope when it is not. We'll provide a complete calculus, but the key thing to understand is that the rules of flow scoping are just plain old DA/DU; if a binding is DA, then it is well-defined.
>
> In particular, flow-scoping can handle abrupt termination naturally; for a statement:
>
> if (x matches Foo(var a)) { A }
> else { B }
> C
>
> the scope of `a` includes A, and also includes C iff B completes abruptly. We can easily explain this as:
> - if x matches Foo(var a), we execute the A block, and in this case `a` is clearly well-defined (as we'd not execute A if the match failed);
> - The only way to reach C, if B completes abruptly, is if the match succeeds, so `a` is well defined during C in this case too.
>
> Because the scope of a binding variable is precisely the cases in which it is well defined, there is no need to tinker with shadowing.
>
> Conditional variables can now always be final, because they will never be in scope and not DA.
>
> Similarly, folding reachability into scoping for conditional variables also means that fallthrough has a well-defined meaning. If we have:
>
> case Foo(int x): ... break;
> case Bar(int x): ....
>
> then the Bar case is not reachable from where x would be initialized, so the first x is not in scope when the second x is declared, and everything is great. On the other hand:
>
> case Foo(int x): ... no break ...
> case Bar(int x): ... A ...
>
> now x is well-defined in A, no matter how we got there. (The merging of the two xs is the same merging we have to do anyway for "if (x matches Foo(int a) || x matches Bar(int a)".)
>
>
> People had originally expressed concern that flow-scoping leaves a scope "with holes", and allows puzzlers with shadowing of fields. (This is the "swiss cheese" problem.) For example:
>
> // Field
> String s
>
> if (!(x matches String s)) {
> a(s);
> }
> else {
> b(s);
> }
>
> This would be confusing because the `s` passed to a() is the field, but the `s` passed to b() is the binding. But, there's a really simple way to prevent this: do not allow conditional variables to shadow fields or locals. Now, there is no chance of this confusion, and this is not a big constraint, because the names of conditional variables are strictly local. (Further, we can disallow shadowing of in-scope conditional variables by locals (or other conditional variables.))
>
>
> Scorecard:
> - Relatively straightforward to spec, as we have a clean calculus for flow-scoped conditional variables;
> - Relatively straightforward to implement (our prototype already does this);
> - One new concept: conditional variables;
> - Conditional vars are scope where they make sense, and not in scope where they do not, cannot be assigned to (always DA and final when in scope), and are never in scope when not DA;
> - No changes to shadowing;
> - Meets all the target use cases.
>
>
>
>
> On 11/3/2017 6:44 AM, Gavin Bierman wrote:
>> Scopes
>>
>> Java has five constructs that introduce fresh variables into scope: the local variable declaration statement, the for statement, the try-with-resources statement, the catch block, and lambda expressions. The first, local variable declaration statements, introduce variables that are in scope for the rest of the block that it is declared in. The others introduce variables that are limited in their scope.
>>
>> The addition of pattern matching brings a new expression, matches, and extends the switch statement. Both these constructs can now introduce fresh (and, if the pattern match succeeds, definitely assigned (DA)) variables. But the question is what is the scope of these ‘pattern’ variables?
>>
>> Let us consider the pattern matching constructs in turn. First the switch statement:
>>
>> switch (o) {
>> case int i: ...
>> case ..
>> }
>> What is the scope of the pattern variable i? There are a range of options.
>>
>> The scope of the pattern variable is from the start of the switch statement until the end of the enclosing block.
>>
>> In this case the pattern variable is in scope but would be definitely unassigned (DU) immediately after the switch statement.
>>
>> switch (o) {
>> case int i : ... // DA
>> ... // DA
>> case T t : // i is in scope
>> }
>> ... // i in still in scope and DU
>> +ve Simple
>> -ve Can’t simply reuse a pattern variable in the same switch statement (without some form of shadowing)
>> -ve Pattern variable poisons the rest of the block
>> The scope of the pattern variable extends only to the end of the switch block.
>>
>> In this case the pattern variable would be considered DA only for the statements between the current case label and the subsequent case labeled statement. For example:
>>
>> switch (o) {
>> case int i : ... // DA
>> ... // DA
>> case T t : // i is in scope but not DA
>> }
>> ... // i not in scope
>> +ve Simple
>> +ve Pattern variables not poisoned in subsequent statements in the rest of the block
>> +ve Similar technique to for identifiers (not a new idea)
>> -ve Can’t simply reuse a pattern variable in the same switch statement (without some form of shadowing)
>> The scope of the pattern variable extends only to the next case label.
>>
>> switch (o) {
>> case int i : ... // in scope and DA
>> ... // in scope and DA
>> case T i : // int i not in scope, so can re-use
>> }
>> ... // i not in scope
>> +ve Simple syntactic rule
>> +ve Allows reuse of pattern variable in the same switch statement.
>> -ve Doesn’t make sense for fallthrough
>> NOTE This final point is important - supporting fallthrough impacts on what solution we might choose for scoping of pattern variables. (We could not support fallthrough and instead support OR patterns - a further design dimension.)
>>
>> ASIDE Should we support a switch expression; it seems clear that scoping should be treated in the same way as it is for lambda expressions.
>>
>> The matches expression is unusual in that it is an expression that introduces a fresh variable. What is the scope of this variable? We want it to be more than the expression itself, as we want the following example code to be correct:
>>
>> if (e matches String s) {
>> System.out.println("It's a string - " + s);
>> }
>> In other words, the variable introduced by the pattern needs to be in scope for an enclosing IfThen statement.
>>
>> However, a match expression could be nested within another expression. It seems reasonable that the patterns variables are in scope for at least the rest of the expression. For example:
>>
>> (e matches String s || s.length() > 0)
>> Here the s should be in scope for the subexpression s.length (although it is not DA). In contrast:
>>
>> (e matches String s && s.length() > 0)
>> Here the s is both in scope and DA for the subexpression s.length.
>>
>> However, what about the following:
>>
>> if (s.length() > 0 && e matches String s) {
>> System.out.println(s);
>> }
>> Given the idea that a pattern variable flows from the inside-out to the enclosing statement, it would appear that s is in scope for the subexpression s.length; although it is not DA. Unless we want scopes to be non-contiguous, we will have to accept this rather odd situation (consider where s shadows a field). [This appears to be what happens in the current C# compiler.]
>>
>> Now let’s consider how far a pattern variable flows wrt its enclosing statement. We have a range of options:
>>
>> The scope is both the statement that the match expression occurs in and the rest of the block. In this scenario,
>>
>> if (o matches T t) {
>> ...
>> } else {
>> ...
>> }
>> is treated as equivalent to the following pseudo-code (where match-and-bind is a fictional pattern matching construct that pattern-matches and binds to a variable that has already been declared)
>>
>> T t;
>> if (o match-and-bind t) {
>> // t in scope and DA
>> } else {
>> // t in scope and DU
>> }
>> // t in scope and DU
>> This is how the current C# compiler works (although the spec describes the next option; so perhaps this is a bug).
>>
>> The scope is just the statement that the match expression occurs in. In this scenario,
>>
>> if (o matches T t) {
>> ...
>> } else {
>>
>> }
>> ...
>> is treated as equivalent to the pseudo-code
>>
>> { T t;
>> if (o match-and-bind t) {
>> // t in scope and DA
>> } else {
>> // t in scope and DU
>> // thus declaration int t = 42; is not allowed.
>> }
>> }
>> // t not in scope
>> ...
>> This restricted scope allows reuse of pattern variables, e.g.
>>
>> if (o matches T x) { ... }
>> if (o matches S x) { ... }
>> The scope of the pattern variable is determined by a flow analysis of the enclosing statement. (It could be thought of as a refinement of option b.) This is currently implemented in the prototype compiler. For example:
>>
>> if (!!(o matches T t)) {
>> // t in scope
>> } else {
>> // t not in scope
>> }
>> +ve Code will work in the presence of most refactorings
>> +ve We have this code working already :-)
>> -ve This is a break to the existant notion of scope as a contiguous program fragment. A scope can now have holes in it. Will users ever understand this? (Although they are very similar to the flow-based rules for DA/DU.)
>> ASIDE Regardless of whether we opt for (b) or (c) we may consider a further extension where we allow the scope to extend beyond the current statement for the case of an unbalanced if statement. For example
>>
>> ```
>> if (!(o matches T t)) {
>> return;
>> }
>> // t in scope
>> ...
>> return;
>> ```
>> +ve Supports a common idiom where else blocks are not needed
>> -ve Yet further complication of notion of scope.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/amber-spec-experts/attachments/20171120/626e6a42/attachment-0001.html>
More information about the amber-spec-experts
mailing list