From gavin.bierman at oracle.com Tue May 11 14:56:43 2021 From: gavin.bierman at oracle.com (Gavin Bierman) Date: Tue, 11 May 2021 14:56:43 +0000 Subject: [sealed-types] Draft Spec for JEP 409 Sealed Types (Final) In-Reply-To: <228BFF00-9BEE-4403-894E-9D41DA9AF8B4@oracle.com> References: <228BFF00-9BEE-4403-894E-9D41DA9AF8B4@oracle.com> Message-ID: Sorry, meant to thank you for pointing this out. You are correct in that (i) when writing these I wrote them side-by-side, and (ii) when we fixed up the text surrounding top-level/member/local I forgot to put them side-by-side again to check there wasn?t any stray old text. Ug. (It?s also super annoying that the section numbers don?t match to make this even better!) Alex spotting a small issue that will break up this nice symmetry some more: the restriction on the canonical name excludes the neat trick that we play compiling out enums with constants that have bodies into a sealed hierarchy (as we need to have permitted subclasses that are anonymous classes). It?s fixable but we have to special case enum classes :-( Thanks again, Gavin > On 23 Apr 2021, at 21:19, John Rose wrote: > > On Apr 20, 2021, at 2:00 AM, Gavin Bierman wrote: >> >> >> http://cr.openjdk.java.net/~gbierman/jep409/latest/ > > Out of curiosity I compared the two large new sections > on permits, 8.1.4 and 9.1.4, perhaps to find unintentional > divergences. Here are the differences I found FYI, > aside from than the required differences arising > from section numbering and the parallel terminology > of interfaces and lists. > >> The optional permits clause in a [C/I] declaration {specifies, lists} the [C/I]s > >> It is a compile-time error if {the same class, a class or interface} is {specified, named} more than once in a {, single} permits clause > >> The permitted direct sub[C/I]s of a sealed [C/I] are the [C/I]s listed by its permits clause, or, if [C/I] lacks a permits clause, each {, top level or member} [C/I] declared in the same compilation unit as [C/I] (7.3) which has a canonical name (6.7) > > This last one is especially odd, since ?top level or member? > seems redundant when ?has a canonical name? is specified. > The latter condition is in both places, and is more pertinent. > > Also, this comment was in the class section, but there was no > parallel comment in the interface section: > >> This means that if a sealed class C lacks a permits clause one will be inferred that contains all the top-level and member classes in the same compilation unit that list C as their direct superclass. The requirement of having a canonical name means that no local classes or anonymous classes will be considered. > > Perhaps you added the ?canonical name? and did not replace > all of the predecessor language about ?top level or member?? > > (As expected, I found no substantive differences.) > > ? John From gavin.bierman at oracle.com Mon May 17 16:48:04 2021 From: gavin.bierman at oracle.com (Gavin Bierman) Date: Mon, 17 May 2021 16:48:04 +0000 Subject: [patterns-switch] Draft JLS spec for JEP 406 (Pattern Matching for Switch) In-Reply-To: <498B5850-1209-4776-A287-E7704087B7A1@oracle.com> References: <498B5850-1209-4776-A287-E7704087B7A1@oracle.com> Message-ID: Dear experts, I have amended the spec for JEP 406 (Pattern Matching for Switch) to reflect the consensus following the recent lengthy discussions on this list about two design changes, specifically: 1. treating total type patterns as match-everything switch labels (i.e. matching null) 2. requiring non-legacy switches to be total like switch expressions The updated spec is available at: http://cr.openjdk.java.net/~gbierman/jep406/latest/ Please let us know what you think! Thanks, Gavin PS: The details: To capture the notion of a total type pattern matching everything, I have not followed the idea of extending the notion of pattern matching to be a three-place relation, taking a value, a compile-time type, and a pattern. This passing of a compile-time type around at *runtime* is not at all how the JLS works (which maintains a pretty strict phase distinction), and it doesn?t accurately reflect what we do in the compiler either. So, here?s how it has been specified: 1. I introduced a new pattern, called an *any pattern*. Right now, it?s a pattern that is expressible but not denotable, i.e. it is an error if it appears anywhere in the source program. It arises from a process of ?resolving? a pattern... 2. There is a new notion of "resolving" a pattern with a type. It works like this: A pattern P resolves with a type U as follows: 1. if P is T t where U <: T, P resolves to an any pattern `var t`, where the type of t is T; otherwise P resolves to P. 2. if P is var x, P resolves to an any pattern `var x` where the type of x is U; 3. if P is (Q), then P resolves to (Q'), where Q' is the result of resolving Q with U 4. if P is Q && e, then (i) If Q && e is total for U then P resolves to Q' && e where Q' is the result of resolving Q with U; (ii.) If Q && e is not total for U then P resolves to P. So the basic idea of resolving a pattern is that it replaces any total patterns with an any pattern. Now, the runtime behaviour of pattern matching is specified to always be with respect to a resolved pattern. In other words, with e instanceof P, we resolve P with the type of e to get a resolved pattern Q, and then we pattern match the value of e with the resolved pattern Q. With switches we resolve any pattern labels with the type of the selector expression before finding a switch label that matches. Pattern matching can now be more fully specified as follows: - A value v (including null) matches `var x` and x is initialized to v. - The null reference does not match T t. - A non-null value v matches T t if v can be cast to T without raising a ClassCastException, and t is initialized to v; and does not match otherwise. - A value v matches p && e if (i) v matches p, and (ii) e evaluates to true; and does not match otherwise. - A value v matches (p) if v matches p; and does not match otherwise. This means that total type patterns appearing as switch labels will now match all values *including null*, as they will be resolved to any patterns. I have also changed the specification of the runtime semantics of switches by breaking it up into two different cases depending on whether the selector expression evaluates to `null` or not. If it does evaluate to null then we look for either a `case null` or a total pattern case label. If we find none then we NPE. Otherwise, we look at all the switch labels as before (including the legacy ones involving constants). This simplifies matters with the legacy labels so they don't need to explicitly exclude null. I have added a new condition on switch statements that if it is a non-legacy switch then the switch block must be complete (the same condition as for switch expressions). The execution of a switch statement now has a new test: if no switch label matches in the body, then either (i) if it is a non-legacy switch then ICCE is thrown; or (ii) if it is a legacy switch then execution completes normally (as before). Some other minor tweaks were also made. On 23 Apr 2021, at 16:58, Gavin Bierman > wrote: Dear experts, Apologies for the delay, but here is the first draft of the spec for JEP 406 (Pattern Matching for Switch): http://cr.openjdk.java.net/~gbierman/jep406/latest/ As you will see, the design has evolved a little since we wrote the JEP (which I will update shortly). The chief changes are: 1. Instead of enumerating the special case labels, e.g. case null, default, case null, T t and so on; the notion of switch label has been made more general, with a set of restrictions to cut out the things we don?t want. In particular, this means that we get (a) symmetric versions of the compound patterns, e.g. case null, default and case default, null: and (b) we get a perfect alignment of the old : style with the new , style, e.g. we can write both `case null, default: ?` and `case null: default: ?` with identical treatment. 2. I figured out a way to remove the restriction from the JEP that switches must be all pattern-matching, or all non-pattern-matching. Apart from simplifying matters, this allows for some nice new features, e.g. enum E { F, G, H } switch(...) { // A switch block with an enum constant and pattern label! case F -> ? case E e -> ? // Acts like a default but e is in scope! } A couple of odd extras come for the ride, just for your information: switch(o) { /* empty */ } is now well-typed for all types of o, as is: switch(o) { default: ? } [Basically any type restrictions on the selector expression come from the switch labels. We still maintain that if you use a constant case label, the type of the selector must be char, byte, short, int, Character, Byte, Short, Integer, or String as in Java 16.] As always, your opinions are welcomed! Thanks, Gavin From brian.goetz at oracle.com Mon May 17 21:36:30 2021 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 17 May 2021 17:36:30 -0400 Subject: Rehabilitating switch -- a scorecard Message-ID: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> This is a good time to look at the progress we've made with switch.? When we started looking at extending switch to support pattern matching (four years ago!) we identified a lot of challenges deriving from switch's C legacy, some of which is summarized here: http://cr.openjdk.java.net/~briangoetz/amber/switch-rehab.html We had two primary driving goals for improving switch: switches as expressions, and switches with patterns as labels.? In turn, these pushed on a number of other uncomfortable aspects of switch: fall through, totality, scoping, and null handling. Initially, we were unsure we would be able to rehabilitate switch to support these new requirements without being forever bogged down by the mistakes of the past.? Bit by bit, we have chipped away at the negative aspects of switch, while respecting the existing code that depends on those aspects.? I think where we've landed is, in many ways, better than we could have initially hoped for. Throughout this exercise, there were periodic calls for "just toss it and invent something new" (which we sometimes called "snitch", shorthand for "new switch"*), and no shortage of people's attempts to design their ideal switch construct.? We resisted this line of attack, because we believed having two similar-but-different constructs living side by side would be more annoying (and confusing) to users than a rehabilitated, albeit more complex, construct. The first round of improvements came with expression switches. This was the easy batch, because it didn't materially change the set of questions we could ask with switch, just the form in which we asked the question.? This brought the following improvements: ?- Switches as expressions.? Many existing switch statements are in reality modeling expressions, in a more roundabout and less safe way.? Expressing it directly is simpler and less error-prone. ?- Checked totality.? The compiler enforces that a switch expression is exhaustive (because, expressions must be total). In the case of enum switches, a switch that covers all the cases needs no default clause, and the compiler inserts an extra case to catch novel values and throw (ICCE) on them.? (Eventually the same will be true for switches on sealed classes as well.) ?- A fallthrough-free option.? Switches now give us a choice between two styles of _switch blocks_, the old willy-nilly style, and the new single-consequent (arrow) style.? Switches that choose arrow-style need not reason about fallthrough. Unfortunately, it also brought a new asymmetry; switch expressions must be total (and you get enhanced type checking for this), but switch statements cannot be.? This is a shame, since the improved type checking for totality is one of the best things about the improvements in switch, as a switch that is total by virtue of actually covering all the cases acts as a tripwire against new enum constants / permitted subtypes being added later, rather than papering it over with a catch-all.? We explored several ways to explicitly add back totality checking, but this always felt like a hack, and requires the programmer to remember to ask for this checking. Our resolution here offers a path to true healing with minimal user impact, by (temporarily) carving out the semantic space of old statement switches.? A "legacy switch" is a statement switch on a numeric primitive or its box, enum, or string, and which contains no pattern labels (i.e., a statement switch that is valid today.)? Like expression switches, we will require non-legacy statement switches to be exhaustive, and warn on non-exhaustive legacy switches.? (To make the warning go away, just insert a "default: " or "default: break" at the bottom of the switch; not painful.)? After some time, we should be able to make this warning an error, which again is easy to mitigate with a single line.? In the end, all switch constructs will be total and type-checked for exhaustiveness, and once done, the notion of "legacy switch" can be garbage-collected. Looking ahead to patterns in switch, we have several legacy considerations to navigate: ?- Fallthrough and bindings.? While fallthrough is not inherently problematic (though the choice of fallthrough-by-default was unfortunate), if a case label introduces a pattern variable, then fallthrough to another case (at least one that doesn't introduce the same pattern variable with the same type) makes little sense, and such fallthrough has been outlawed. ?- Scoping.? The block of a switch is one big scope, rather than each case label group being its own scope.? (Again, one might call this a historical error, since there's little good that comes from this.)? With case labels introducing variable declarations, this could have been a big problem, if one case polluted later cases (forcing users to pick unique names for each binding in a switch statement), but flow scopoing solves that one. ?- Nulls.? In Java 1.0, switching over reference types was not permitted, so we didn't have to worry about this.? In Java 5, autoboxing and enums meant we could switch over some reference types, but for all of these, null was a "silly" value so we didn't care about NPEing on null.? In Java 7, when we added string switch, we could have conceivably allowed `case null`, but instead chose to follow the precedent set by Java 5.? But once we introduce switches over any type, with richer patterns, eagerly NPEing on null becomes much more problematic.? We've navigated this by say that switches can NPE on null if they have no nullable cases; nullable cases are those that explicitly say "null", and total patterns (which always come last since they dominate all others.)? The old rule of "switches throw on null" becomes "switches throw on null, except when they say 'case null' or the bottom case is total."? Default continues to mean what it always did -- "anything not already matched, except null." The new treatment of null actually would have fallen out of the decisions on totality, had we not gotten there already via another path.? Our notion of totality accounts for "remainder", which includes things like novel subclasses of sealed types that did not exist at compile time, which it would not be reasonable to ask users to write code to deal with, and null fits into this treatment as well.? We type check that a switch is sufficiently total, and then insert extra code to catch "silly" values that are not otherwise handled, including null, and throw.? (This also enables DA analysis to truly trust switch totality.) Where we land is a single unified switch construct that can be either a statement or an expression; that can use either old-style flow (colon) or the more constrained flow style (arrow); whose case labels can be constant, patterns (including guarded patterns), or a mix of the two; which can accept the legacy null-hostility behavior, or can override it by explicitly using nullable case labels; and which are almost always type checked for totality (with some temporary, legacy exceptions.) Fallthough is basically unchanged; you can get fallthrough when using the old-style flow, but becomes less important as fallthrough is (mostly) nonsensical in the presence of pattern cases with bindings, and the compiler prevents this misuse.? The distinction between "legacy" switches and pattern switches is temporary, with a path to getting to "all switches are total" over time. I think we've done a remarkable job at rehabilitating this monster. *Someone actually suggested using the syntax "new switch", on the basis that new was already a keyword.? Would not have aged well. From forax at univ-mlv.fr Mon May 17 22:08:09 2021 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 18 May 2021 00:08:09 +0200 (CEST) Subject: Rehabilitating switch -- a scorecard In-Reply-To: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> Message-ID: <1217367787.983746.1621289289267.JavaMail.zimbra@u-pem.fr> > De: "Brian Goetz" > ?: "amber-spec-experts" > Envoy?: Lundi 17 Mai 2021 23:36:30 > Objet: Rehabilitating switch -- a scorecard > This is a good time to look at the progress we've made with switch. When we > started looking at extending switch to support pattern matching (four years > ago!) we identified a lot of challenges deriving from switch's C legacy, some > of which is summarized here: > [ http://cr.openjdk.java.net/~briangoetz/amber/switch-rehab.html | > http://cr.openjdk.java.net/~briangoetz/amber/switch-rehab.html ] > We had two primary driving goals for improving switch: switches as expressions, > and switches with patterns as labels. In turn, these pushed on a number of > other uncomfortable aspects of switch: fall through, totality, scoping, and > null handling. > Initially, we were unsure we would be able to rehabilitate switch to support > these new requirements without being forever bogged down by the mistakes of the > past. Bit by bit, we have chipped away at the negative aspects of switch, while > respecting the existing code that depends on those aspects. I think where we've > landed is, in many ways, better than we could have initially hoped for. > Throughout this exercise, there were periodic calls for "just toss it and invent > something new" (which we sometimes called "snitch", shorthand for "new > switch"*), and no shortage of people's attempts to design their ideal switch > construct. We resisted this line of attack, because we believed having two > similar-but-different constructs living side by side would be more annoying > (and confusing) to users than a rehabilitated, albeit more complex, construct. > The first round of improvements came with expression switches. This was the easy > batch, because it didn't materially change the set of questions we could ask > with switch, just the form in which we asked the question. This brought the > following improvements: > - Switches as expressions. Many existing switch statements are in reality > modeling expressions, in a more roundabout and less safe way. Expressing it > directly is simpler and less error-prone. > - Checked totality. The compiler enforces that a switch expression is exhaustive > (because, expressions must be total). In the case of enum switches, a switch > that covers all the cases needs no default clause, and the compiler inserts an > extra case to catch novel values and throw (ICCE) on them. (Eventually the same > will be true for switches on sealed classes as well.) > - A fallthrough-free option. Switches now give us a choice between two styles of > _switch blocks_, the old willy-nilly style, and the new single-consequent > (arrow) style. Switches that choose arrow-style need not reason about > fallthrough. > Unfortunately, it also brought a new asymmetry; switch expressions must be total > (and you get enhanced type checking for this), but switch statements cannot be. > This is a shame, since the improved type checking for totality is one of the > best things about the improvements in switch, as a switch that is total by > virtue of actually covering all the cases acts as a tripwire against new enum > constants / permitted subtypes being added later, rather than papering it over > with a catch-all. We explored several ways to explicitly add back totality > checking, but this always felt like a hack, and requires the programmer to > remember to ask for this checking. > Our resolution here offers a path to true healing with minimal user impact, by > (temporarily) carving out the semantic space of old statement switches. A > "legacy switch" is a statement switch on a numeric primitive or its box, enum, > or string, and which contains no pattern labels (i.e., a statement switch that > is valid today.) Like expression switches, we will require non-legacy statement > switches to be exhaustive, and warn on non-exhaustive legacy switches. (To make > the warning go away, just insert a "default: " or "default: break" at the > bottom of the switch; not painful.) After some time, we should be able to make > this warning an error, which again is easy to mitigate with a single line. In > the end, all switch constructs will be total and type-checked for > exhaustiveness, and once done, the notion of "legacy switch" can be > garbage-collected. > Looking ahead to patterns in switch, we have several legacy considerations to > navigate: > - Fallthrough and bindings. While fallthrough is not inherently problematic > (though the choice of fallthrough-by-default was unfortunate), if a case label > introduces a pattern variable, then fallthrough to another case (at least one > that doesn't introduce the same pattern variable with the same type) makes > little sense, and such fallthrough has been outlawed. > - Scoping. The block of a switch is one big scope, rather than each case label > group being its own scope. (Again, one might call this a historical error, > since there's little good that comes from this.) With case labels introducing > variable declarations, this could have been a big problem, if one case polluted > later cases (forcing users to pick unique names for each binding in a switch > statement), but flow scopoing solves that one. > - Nulls. In Java 1.0, switching over reference types was not permitted, so we > didn't have to worry about this. In Java 5, autoboxing and enums meant we could > switch over some reference types, but for all of these, null was a "silly" > value so we didn't care about NPEing on null. In Java 7, when we added string > switch, we could have conceivably allowed `case null`, but instead chose to > follow the precedent set by Java 5. But once we introduce switches over any > type, with richer patterns, eagerly NPEing on null becomes much more > problematic. We've navigated this by say that switches can NPE on null if they > have no nullable cases; nullable cases are those that explicitly say "null", > and total patterns (which always come last since they dominate all others.) The > old rule of "switches throw on null" becomes "switches throw on null, except > when they say 'case null' or the bottom case is total." Default continues to > mean what it always did -- "anything not already matched, except null." > The new treatment of null actually would have fallen out of the decisions on > totality, had we not gotten there already via another path. Our notion of > totality accounts for "remainder", which includes things like novel subclasses > of sealed types that did not exist at compile time, which it would not be > reasonable to ask users to write code to deal with, and null fits into this > treatment as well. We type check that a switch is sufficiently total, and then > insert extra code to catch "silly" values that are not otherwise handled, > including null, and throw. (This also enables DA analysis to truly trust switch > totality.) > Where we land is a single unified switch construct that can be either a > statement or an expression; that can use either old-style flow (colon) or the > more constrained flow style (arrow); whose case labels can be constant, > patterns (including guarded patterns), or a mix of the two; which can accept > the legacy null-hostility behavior, or can override it by explicitly using > nullable case labels; and which are almost always type checked for totality > (with some temporary, legacy exceptions.) Fallthough is basically unchanged; > you can get fallthrough when using the old-style flow, but becomes less > important as fallthrough is (mostly) nonsensical in the presence of pattern > cases with bindings, and the compiler prevents this misuse. The distinction > between "legacy" switches and pattern switches is temporary, with a path to > getting to "all switches are total" over time. > I think we've done a remarkable job at rehabilitating this monster. I believe the only pending issue on that matter is the position of default inside the switch, With the legacy switch, default can be in the middle, with a switch on types that default has to be the last case. I think we should try to emit a warning if "default" is not at last position, both Eclipse and IntelliJ already have that warning. R?mi > *Someone actually suggested using the syntax "new switch", on the basis that new > was already a keyword. Would not have aged well. if we add a prefix "new" to switch for each LTS release, e.g. new new switch for 6 years after 2018, it would help the future historians because radiocarbon dating does not work well on the source code. From brian.goetz at oracle.com Mon May 17 22:23:10 2021 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 17 May 2021 18:23:10 -0400 Subject: [External] : Re: Rehabilitating switch -- a scorecard In-Reply-To: <1217367787.983746.1621289289267.JavaMail.zimbra@u-pem.fr> References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> <1217367787.983746.1621289289267.JavaMail.zimbra@u-pem.fr> Message-ID: <057223f5-e275-e626-573e-88d33349c371@oracle.com> > I think we've done a remarkable job at rehabilitating this monster. > > > I believe the only pending issue on that matter is the position of > default inside the switch, > With the legacy switch, default can be in the middle, with a switch on > types that default has to be the last case. > > I think we should try to emit a warning if "default" is not at last > position, both Eclipse and IntelliJ already have that warning. > > In an earlier round, we pended this issue because there were corner cases such as default falling into another case, such as: ??? switch (n) { ??????? case 1: println("one"); break; ??????? case 2: println("two"); break; ??????? default: print("more than "); ??????? case 3: println("three"); break; ??? } which prints "one", "two", "three", or "more than three". This is legal, if silly.? We pended this issue until the rest of the story shook out; now that it has, I think we are still kind of in the same place here.? A warning is OK here but it feels more like a style hint than? a real warning. From john.r.rose at oracle.com Mon May 17 23:57:10 2021 From: john.r.rose at oracle.com (John Rose) Date: Mon, 17 May 2021 23:57:10 +0000 Subject: Rehabilitating switch -- a scorecard In-Reply-To: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> Message-ID: In May 17, 2021, at 4:36 PM, Brian Goetz wrote: > > I think we've done a remarkable job at rehabilitating this monster. +100, especially after re-reading your 2017 document. > *Someone actually suggested using the syntax "new switch", on the basis that new was already a keyword. Would not have aged well. Yeah, and we can have as many switches as we want if we use hyphenated keywords: switch-jdk16, switch-jdk17, ? It looks to me like we anticipated the design space pretty accurately in 2017. We have apparently returned to the ?&&? guard syntax, after some discussion, which makes me happy. And we have avoided the need for ?||? pattern connectors. There are a few roads not taken: ?switch ()? with boolean case expressions has not showed itself worthy yet. The ?break x;? syntax, meaning ?return value x from the switch,? is obviated by doubling down on lambda-inspired ?return x;?. I?d also like to point out ?switch (a, b)? as a possible area of future work for switch, where the thing after ?switch? is a more generalized argument expression, and the machinery of method binding could be applied to the list of values to be matched (as if they were actual arguments) and the case labels would (somehow) correspond to method formal argument lists. Having said that, I don?t think it is urgent, at all. That said, I will take the opportunity to mention a few small items of ?fit and finish?, mostly anticipated in the 2017 document, which are still (IMO) useful proposals, and are currently on the back burner. 1. Allow certain simple statements after ?->? without requiring ?{ ? }? wrappers. This is not just luxury but improves readability. I have reached for the sequence ?->break;? in my IDE and been disappointed after going back to the fine print of the language. I had to write ?->{break;}?, which is IMO noisy enough to be harder to read than ?->break;?. As both writer and reader of code, I would rather have the use of ?{}? be reserved for stuff that is truly a block or other composite (while-loop, etc.). This issue wasn?t raised with lambdas because lambdas cannot complete with control flow, but switch cases often do that. For consistency, I suggest that any type of statement that does not have sub-statements be allowed immediately after the ?->?. (Of switch, and lambda if applicable.) In addition to the statement expressions already allowed, these are return, break, continue. They are also the empty statement (?;?), and assert statements. Not allowed would be declarations, labeled statements, conditionals, loops, and other nested things like try and synchronized. 2. Allow a way to add a given case to the ?remainder? set, to be treated similarly to ?built-in? remainder processing. Proposed syntax: ?throw;? as in ?case null->throw;?. (Raises the question of whether to support such a notation outside of switch, but we could just say ?no?. Or spell it ?throw default;? or ?throw assert?.) The idea of a remainder (nulls, maybe, or new enums or sealed class subtypes) is probably useful enough that users should be given a way to opt into remainder processing explicitly. (I don?t see this in the 2017 document; we hadn?t yet generalized the idea of a ?remainder?.) 3. Allow a way for imperative case logic to branch to the next applicable case. Proposed syntax: ?continue switch;?, as in ?case Foo f->{ if (f.bad) continue switch; }?. The purpose for this would be to allow refactoring of guards from the compact form (?case P && G:?) to a more general imperative form (?case P: if(!G) continue switch;?). Such refactoring is sometimes important in practice, if a guard G needs a temporary variable. Today?s workaround is building a helper method nearby, which hurts readability. (The ?continue switch? syntax suggests other kinds of tags for break and continue, and the symmetry between switch and if/else chains suggests ?continue if? as a way to branch after the next ?else?. None of these require follow-up to gain value for switch.) From brian.goetz at oracle.com Tue May 18 02:37:55 2021 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 17 May 2021 22:37:55 -0400 Subject: Rehabilitating switch -- a scorecard In-Reply-To: References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> Message-ID: <1263779e-b96f-1a8d-4084-59257130010a@oracle.com> > There are a few roads not taken: ?switch ()? with boolean > case expressions has not showed itself worthy yet. Yep, this one can sit on the shelf. > I?d also like to point out ?switch (a, b)? as a possible area > of future work for switch, where the thing after ?switch? is a > more generalized argument expression ML supports this, because it just treats the operand as a tuple, and automatically destructures tuples.?? The killer use case is, of course, FizzBuzz: ??? switch (n % 3, n % 5) { ??????? case (0, 0) -> "FizzBuzz"; ??????? case (0, _) -> "Fizz"; ??????? case (_, 0) -> "Buzz"; ??????? default -> n.toString(); ??? } > 1. Allow certain simple statements after ?->? without > requiring ?{ ? }? wrappers. We already do this with `throw`, but could extend.?? I'm waiting for Valhalla to make `void` a type, and then see how painful it is to merge statement switch with void expression switch.? (Now that we've totalized, it's easier.) > 2. Allow a way to add a given case to the ?remainder? > set, to be treated similarly to ?built-in? remainder > processing. Proposed syntax: ?throw;? as in > ?case null->throw;?. (Raises the question of whether > to support such a notation outside of switch, > but we could just say ?no?. Or spell it ?throw default;? > or ?throw assert?.) There's two aspects of this: matching on the remainder (case else?), and denoting the default throw. From forax at univ-mlv.fr Tue May 18 15:40:21 2021 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 18 May 2021 17:40:21 +0200 (CEST) Subject: Rehabilitating switch -- a scorecard In-Reply-To: <1263779e-b96f-1a8d-4084-59257130010a@oracle.com> References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> <1263779e-b96f-1a8d-4084-59257130010a@oracle.com> Message-ID: <1476743750.1668373.1621352421750.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Brian Goetz" > ?: "John Rose" > Cc: "amber-spec-experts" > Envoy?: Mardi 18 Mai 2021 04:37:55 > Objet: Re: Rehabilitating switch -- a scorecard >> There are a few roads not taken: ?switch ()? with boolean >> case expressions has not showed itself worthy yet. > > Yep, this one can sit on the shelf. > >> I?d also like to point out ?switch (a, b)? as a possible area >> of future work for switch, where the thing after ?switch? is a >> more generalized argument expression > > ML supports this, because it just treats the operand as a tuple, and > automatically destructures tuples.?? The killer use case is, of course, > FizzBuzz: > > ??? switch (n % 3, n % 5) { > ??????? case (0, 0) -> "FizzBuzz"; > ??????? case (0, _) -> "Fizz"; > ??????? case (_, 0) -> "Buzz"; > ??????? default -> n.toString(); > ??? } with the destructuring pattern, our current version is record Result(int d3, int d5) {} switch(new Result(n % 3, n % 5)) { case Result(var d3, var d5) && d3 == 0 && d5 == 0 -> "FizzBuzz"; case Result(var d3, var __) && d3 == 0 -> "Fizz"; case Result(var __, var d5) && d5 == 0 -> "Buzz"; default -> "" + n; } to fill the gap, we need - _ as pattern equivalent to var _ + _ not be entered in the scope - constant as pattern, Foo(constant) being equivalent to Foo(var x) && x == constant - inference of Type in a destructuring pattern, case Foo(var x, var y) becomes case (var x, var y) if the type of the switched value is Foo This kind of inference is also useful in pattern assignment with Foo(var x, var y) = foo; being written (var x, var y) = foo; - tuple as first class citizen (expr, expr2) is equivalent to either creating a record + new SyntheticRecord(expr, expr2) or new Tuple(expr, expr2) This feature is also useful to model methods that returns several values/deconstructor, but it means that either methods descriptor will contains synthetic record name which is ugly (TypeRestriction may help here), or we are able to cook a primitive parametrized/specialized class Tuple, the creation can be an indy converted as (default + a sequence of withfields), but we have to wait Valhalla. R?mi From brian.goetz at oracle.com Tue May 18 17:07:41 2021 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 18 May 2021 13:07:41 -0400 Subject: [External] : Re: Rehabilitating switch -- a scorecard In-Reply-To: <1476743750.1668373.1621352421750.JavaMail.zimbra@u-pem.fr> References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> <1263779e-b96f-1a8d-4084-59257130010a@oracle.com> <1476743750.1668373.1621352421750.JavaMail.zimbra@u-pem.fr> Message-ID: > to fill the gap, we need > - _ as pattern equivalent to var _ + _ not be entered in the scope On the list, will come at some point. > - constant as pattern, Foo(constant) being equivalent to Foo(var x) && x == constant Maybe, not sure it carries its weight. > - inference of Type in a destructuring pattern, case Foo(var x, var y) becomes case (var x, var y) if the type of the switched value is Foo > This kind of inference is also useful in pattern assignment > with Foo(var x, var y) = foo; being written (var x, var y) = foo; Coming. > > - tuple as first class citizen Sorry, no.? I know you really want this, but this is similar to the function-type-vs-functional-interface issue.? We made our choice when we did records -- records are our tuples. From forax at univ-mlv.fr Tue May 18 18:12:27 2021 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Tue, 18 May 2021 20:12:27 +0200 (CEST) Subject: [External] : Re: Rehabilitating switch -- a scorecard In-Reply-To: References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> <1263779e-b96f-1a8d-4084-59257130010a@oracle.com> <1476743750.1668373.1621352421750.JavaMail.zimbra@u-pem.fr> Message-ID: <815481630.1760758.1621361547450.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Brian Goetz" > ?: "Remi Forax" > Cc: "John Rose" , "amber-spec-experts" > Envoy?: Mardi 18 Mai 2021 19:07:41 > Objet: Re: [External] : Re: Rehabilitating switch -- a scorecard [...] >> >> - tuple as first class citizen > > Sorry, no.? I know you really want this, but this is similar to the > function-type-vs-functional-interface issue.? We made our choice when we > did records -- records are our tuples. I'm hunting around deconstructor and methods that returns several values. I agree that records are our tuples but we had only anonymous classes and nevertheless we add lambdas, similarly we have records and we may still want to add anonymous primitive records (aka tuples). Not as new structural types but as an anonymous value carriers. R?mi From brian.goetz at oracle.com Tue May 18 18:19:45 2021 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 18 May 2021 14:19:45 -0400 Subject: [External] : Re: Rehabilitating switch -- a scorecard In-Reply-To: <815481630.1760758.1621361547450.JavaMail.zimbra@u-pem.fr> References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> <1263779e-b96f-1a8d-4084-59257130010a@oracle.com> <1476743750.1668373.1621352421750.JavaMail.zimbra@u-pem.fr> <815481630.1760758.1621361547450.JavaMail.zimbra@u-pem.fr> Message-ID: <209e43a4-e4c8-e3be-1c2b-c66431396721@oracle.com> >>> - tuple as first class citizen >> Sorry, no.? I know you really want this, but this is similar to the >> function-type-vs-functional-interface issue.? We made our choice when we >> did records -- records are our tuples. > I'm hunting around deconstructor and methods that returns several values. I know that, and I'm telling you these are not the droids you are looking for. From guy.steele at oracle.com Tue May 18 19:33:45 2021 From: guy.steele at oracle.com (Guy Steele) Date: Tue, 18 May 2021 15:33:45 -0400 Subject: [External] : Re: Rehabilitating switch -- a scorecard In-Reply-To: <209e43a4-e4c8-e3be-1c2b-c66431396721@oracle.com> References: <209e43a4-e4c8-e3be-1c2b-c66431396721@oracle.com> Message-ID: > On May 18, 2021, at 2:19 PM, Brian Goetz wrote: > > ? >>>> - tuple as first class citizen >>> Sorry, no. I know you really want this, but this is similar to the >>> function-type-vs-functional-interface issue. We made our choice when we >>> did records -- records are our tuples. >> I'm hunting around deconstructor and methods that returns several values. > > I know that, and I'm telling you these are not the droids you are looking for. If only we had tail calls. Then instead of writing (int q, int r) = quotientAndRemainder(m, n); whatever we could write (using my preferred syntax for a mandatory tail call in Java) goto quotientAndRemainder(m, n, (int q, int r) -> whatever); (the method quotientAndRemainder would of course tail-call its third argument), with no need of tuples or even records to get multiple values out of a method, and then everyone would be happy, right? RIGHT? :-) :-/ :-P ?Guy From forax at univ-mlv.fr Wed May 19 09:57:36 2021 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 19 May 2021 11:57:36 +0200 (CEST) Subject: [External] : Re: Rehabilitating switch -- a scorecard In-Reply-To: References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> <1263779e-b96f-1a8d-4084-59257130010a@oracle.com> <1476743750.1668373.1621352421750.JavaMail.zimbra@u-pem.fr> Message-ID: <1434239608.2089227.1621418256925.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Brian Goetz" > ?: "Remi Forax" > Cc: "John Rose" , "amber-spec-experts" > Envoy?: Mardi 18 Mai 2021 19:07:41 > Objet: Re: [External] : Re: Rehabilitating switch -- a scorecard >> to fill the gap, we need >> - _ as pattern equivalent to var _ + _ not be entered in the scope > > On the list, will come at some point. > >> - constant as pattern, Foo(constant) being equivalent to Foo(var x) && x == >> constant > > Maybe, not sure it carries its weight. The problem of using guards for constants instead of having a constant pattern introduces a weird asymmetry when you have all the constants are total. By example enum Color { RED, BLUE } record Car(Color color) {} This switch is total switch(color) { case RED: ... case BLUE: ... } This one is not total switch(car) { case Car(color) && color == Color.RED: ... case Car(color) && color == Color.BLUE: ... } so it needs a default or at least Car(var color). Introducing a new enum value will not be detected by the compiler. If we add a constant pattern then this switch is total switch(car) { case Car(Color.RED): ... case Car(Color.BLUE): ... } And if we add target typing so when the compiler sees Car(UNKONWN) it tries to see if Color is not an Enum and Enum.UNKNOWN exists (to have the same behavior as a switch on an enum) We can write switch(car) { case Car(RED): ... case Car(BLUE): ... } [...] R?mi From forax at univ-mlv.fr Wed May 19 10:43:20 2021 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 19 May 2021 12:43:20 +0200 (CEST) Subject: [External] : Re: Rehabilitating switch -- a scorecard In-Reply-To: References: <209e43a4-e4c8-e3be-1c2b-c66431396721@oracle.com> Message-ID: <933304581.2115158.1621421000777.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Guy Steele" > ?: "Brian Goetz" > Cc: "Remi Forax" , "John Rose" , "amber-spec-experts" > > Envoy?: Mardi 18 Mai 2021 21:33:45 > Objet: Re: [External] : Re: Rehabilitating switch -- a scorecard >> On May 18, 2021, at 2:19 PM, Brian Goetz wrote: >> >> ? >>>>> - tuple as first class citizen >>>> Sorry, no. I know you really want this, but this is similar to the >>>> function-type-vs-functional-interface issue. We made our choice when we >>>> did records -- records are our tuples. >>> I'm hunting around deconstructor and methods that returns several values. >> >> I know that, and I'm telling you these are not the droids you are looking for. > > If only we had tail calls. Then instead of writing > > (int q, int r) = quotientAndRemainder(m, n); > whatever > > we could write (using my preferred syntax for a mandatory tail call in Java) > > goto quotientAndRemainder(m, n, (int q, int r) -> whatever); > > (the method quotientAndRemainder would of course tail-call its third argument), > with no need of tuples or even records to get multiple values out of a method, > and then everyone would be happy, right? RIGHT? :-) :-/ :-P > > ?Guy This remember me something, tail call optimization is not the only optimization that avoid the stack to grow. If you have calls like g(a, () -> h(a, () -> f(a))) with 'a' being the same arguments, you can transform them to r1 = f(a) r2 = h(a, () -> r1) r3 = g(a, () -> r2) Here the calls can be at any location in the method, but the argument should not depend on the values computed by the function. The interceptors of Spring or CDI exhibit calls like this, they are using the same arguments so the result do not depend on the order of which the functions/interceptors are executed, but as far as i know, there is not implementation that do that transformation so when an exception occurs we see long stack traces. Asking for a friend, is this transformation have already a name ? R?mi From brian.goetz at oracle.com Wed May 19 11:12:43 2021 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 19 May 2021 07:12:43 -0400 Subject: Rehabilitating switch -- a scorecard In-Reply-To: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> Message-ID: <3f3c56d7-1e7f-ad8c-64cb-1e906e0c7b4d@oracle.com> So, here's another aspect of switches rehabilitation, this time in terms of syntactic rewrites.? By way of analogy with lambdas, there's a sequence of ??? x -> e???????????????? // parens elided in unary lambda is-shorthand-for ??? (x) -> e?????????????? // types elided is-shorthand-for ??? (var x) -> e // explicit request for inference is-shorthand-for ??? ( x) -> e // explicit types That is, there is a canonical (lowest) form, and the various shorthands form a chain of embeddings.? The chain shape reduces cognitive load on the user, because instead of thinking "there are seven forms of lambda", they can instead think there is single canonical form, with progressive options for leaving things out / mushing things together. We get more of a funnel with the syntax of switch: ??? case L, J, K -> X; is-shorthand-for ??? case L, J, K: yield X; // expression switch, X is an expression ??? case L, J, K: X; ??????? // expression switch, X is a block ??? case L, J, K: X; break;? // statement switch and ??? case L, J, K: X; is-shorthand-for ???? case L: ???? case J: ???? case K: ???????? X; On 5/17/2021 5:36 PM, Brian Goetz wrote: > This is a good time to look at the progress we've made with switch.? > When we started looking at extending switch to support pattern > matching (four years ago!) we identified a lot of challenges deriving > from switch's C legacy, some of which is summarized here: > > http://cr.openjdk.java.net/~briangoetz/amber/switch-rehab.html > > We had two primary driving goals for improving switch: switches as > expressions, and switches with patterns as labels.? In turn, these > pushed on a number of other uncomfortable aspects of switch: fall > through, totality, scoping, and null handling. > > Initially, we were unsure we would be able to rehabilitate switch to > support these new requirements without being forever bogged down by > the mistakes of the past.? Bit by bit, we have chipped away at the > negative aspects of switch, while respecting the existing code that > depends on those aspects.? I think where we've landed is, in many > ways, better than we could have initially hoped for. > > Throughout this exercise, there were periodic calls for "just toss it > and invent something new" (which we sometimes called "snitch", > shorthand for "new switch"*), and no shortage of people's attempts to > design their ideal switch construct.? We resisted this line of attack, > because we believed having two similar-but-different constructs living > side by side would be more annoying (and confusing) to users than a > rehabilitated, albeit more complex, construct. > > The first round of improvements came with expression switches.? This > was the easy batch, because it didn't materially change the set of > questions we could ask with switch, just the form in which we asked > the question.? This brought the following improvements: > > ?- Switches as expressions.? Many existing switch statements are in > reality modeling expressions, in a more roundabout and less safe way.? > Expressing it directly is simpler and less error-prone. > ?- Checked totality.? The compiler enforces that a switch expression > is exhaustive (because, expressions must be total).? In the case of > enum switches, a switch that covers all the cases needs no default > clause, and the compiler inserts an extra case to catch novel values > and throw (ICCE) on them.? (Eventually the same will be true for > switches on sealed classes as well.) > ?- A fallthrough-free option.? Switches now give us a choice between > two styles of _switch blocks_, the old willy-nilly style, and the new > single-consequent (arrow) style.? Switches that choose arrow-style > need not reason about fallthrough. > > Unfortunately, it also brought a new asymmetry; switch expressions > must be total (and you get enhanced type checking for this), but > switch statements cannot be.? This is a shame, since the improved type > checking for totality is one of the best things about the improvements > in switch, as a switch that is total by virtue of actually covering > all the cases acts as a tripwire against new enum constants / > permitted subtypes being added later, rather than papering it over > with a catch-all.? We explored several ways to explicitly add back > totality checking, but this always felt like a hack, and requires the > programmer to remember to ask for this checking. > > Our resolution here offers a path to true healing with minimal user > impact, by (temporarily) carving out the semantic space of old > statement switches.? A "legacy switch" is a statement switch on a > numeric primitive or its box, enum, or string, and which contains no > pattern labels (i.e., a statement switch that is valid today.)? Like > expression switches, we will require non-legacy statement switches to > be exhaustive, and warn on non-exhaustive legacy switches.? (To make > the warning go away, just insert a "default: " or "default: break" at > the bottom of the switch; not painful.)? After some time, we should be > able to make this warning an error, which again is easy to mitigate > with a single line.? In the end, all switch constructs will be total > and type-checked for exhaustiveness, and once done, the notion of > "legacy switch" can be garbage-collected. > > Looking ahead to patterns in switch, we have several legacy > considerations to navigate: > > ?- Fallthrough and bindings.? While fallthrough is not inherently > problematic (though the choice of fallthrough-by-default was > unfortunate), if a case label introduces a pattern variable, then > fallthrough to another case (at least one that doesn't introduce the > same pattern variable with the same type) makes little sense, and such > fallthrough has been outlawed. > ?- Scoping.? The block of a switch is one big scope, rather than each > case label group being its own scope.? (Again, one might call this a > historical error, since there's little good that comes from this.)? > With case labels introducing variable declarations, this could have > been a big problem, if one case polluted later cases (forcing users to > pick unique names for each binding in a switch statement), but flow > scopoing solves that one. > ?- Nulls.? In Java 1.0, switching over reference types was not > permitted, so we didn't have to worry about this.? In Java 5, > autoboxing and enums meant we could switch over some reference types, > but for all of these, null was a "silly" value so we didn't care about > NPEing on null.? In Java 7, when we added string switch, we could have > conceivably allowed `case null`, but instead chose to follow the > precedent set by Java 5.? But once we introduce switches over any > type, with richer patterns, eagerly NPEing on null becomes much more > problematic.? We've navigated this by say that switches can NPE on > null if they have no nullable cases; nullable cases are those that > explicitly say "null", and total patterns (which always come last > since they dominate all others.)? The old rule of "switches throw on > null" becomes "switches throw on null, except when they say 'case > null' or the bottom case is total."? Default continues to mean what it > always did -- "anything not already matched, except null." > > The new treatment of null actually would have fallen out of the > decisions on totality, had we not gotten there already via another > path.? Our notion of totality accounts for "remainder", which includes > things like novel subclasses of sealed types that did not exist at > compile time, which it would not be reasonable to ask users to write > code to deal with, and null fits into this treatment as well.? We type > check that a switch is sufficiently total, and then insert extra code > to catch "silly" values that are not otherwise handled, including > null, and throw.? (This also enables DA analysis to truly trust switch > totality.) > > Where we land is a single unified switch construct that can be either > a statement or an expression; that can use either old-style flow > (colon) or the more constrained flow style (arrow); whose case labels > can be constant, patterns (including guarded patterns), or a mix of > the two; which can accept the legacy null-hostility behavior, or can > override it by explicitly using nullable case labels; and which are > almost always type checked for totality (with some temporary, legacy > exceptions.)? Fallthough is basically unchanged; you can get > fallthrough when using the old-style flow, but becomes less important > as fallthrough is (mostly) nonsensical in the presence of pattern > cases with bindings, and the compiler prevents this misuse.? The > distinction between "legacy" switches and pattern switches is > temporary, with a path to getting to "all switches are total" over time. > > I think we've done a remarkable job at rehabilitating this monster. > > > *Someone actually suggested using the syntax "new switch", on the > basis that new was already a keyword.? Would not have aged well. From forax at univ-mlv.fr Wed May 19 12:21:55 2021 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 19 May 2021 14:21:55 +0200 (CEST) Subject: Rehabilitating switch -- a scorecard In-Reply-To: <3f3c56d7-1e7f-ad8c-64cb-1e906e0c7b4d@oracle.com> References: <23f3db2e-3106-98ea-ba8a-27074102fc3c@oracle.com> <3f3c56d7-1e7f-ad8c-64cb-1e906e0c7b4d@oracle.com> Message-ID: <192181521.2165146.1621426915035.JavaMail.zimbra@u-pem.fr> > De: "Brian Goetz" > ?: "amber-spec-experts" > Envoy?: Mercredi 19 Mai 2021 13:12:43 > Objet: Re: Rehabilitating switch -- a scorecard > So, here's another aspect of switches rehabilitation, this time in terms of > syntactic rewrites. By way of analogy with lambdas, there's a sequence of > x -> e // parens elided in unary lambda > is-shorthand-for > (x) -> e // types elided > is-shorthand-for > (var x) -> e // explicit request for inference > is-shorthand-for > ( x) -> e // explicit types > That is, there is a canonical (lowest) form, and the various shorthands form a > chain of embeddings. The chain shape reduces cognitive load on the user, > because instead of thinking "there are seven forms of lambda", they can instead > think there is single canonical form, with progressive options for leaving > things out / mushing things together. > We get more of a funnel with the syntax of switch: > case L, J, K -> X; > is-shorthand-for > case L, J, K: yield X; // expression switch, X is an expression > case L, J, K: X; // expression switch, X is a block > case L, J, K: X; break; // statement switch > and > case L, J, K: X; > is-shorthand-for > case L: > case J: > case K: > X; We also have the inverse problem, rehabilitating lambda syntax to be aligned with the switch syntax. The only discempancy i'm aware of is case Foo -> throw ... being allowed while (Foo foo) -> throw .... is not. BTW, this year i've presented the switch expression before the lambda, so a student ask me why Java does not allow colon in lambda, like this (a, b) : X break; instead of (a, b) -> { X } I answered explaining that a lambda was a function not a block of instructions, but I still feel a diffuse guilt about the reuse of -> inside the switch. R?mi > On 5/17/2021 5:36 PM, Brian Goetz wrote: >> This is a good time to look at the progress we've made with switch. When we >> started looking at extending switch to support pattern matching (four years >> ago!) we identified a lot of challenges deriving from switch's C legacy, some >> of which is summarized here: >> [ http://cr.openjdk.java.net/~briangoetz/amber/switch-rehab.html | >> http://cr.openjdk.java.net/~briangoetz/amber/switch-rehab.html ] >> We had two primary driving goals for improving switch: switches as expressions, >> and switches with patterns as labels. In turn, these pushed on a number of >> other uncomfortable aspects of switch: fall through, totality, scoping, and >> null handling. >> Initially, we were unsure we would be able to rehabilitate switch to support >> these new requirements without being forever bogged down by the mistakes of the >> past. Bit by bit, we have chipped away at the negative aspects of switch, while >> respecting the existing code that depends on those aspects. I think where we've >> landed is, in many ways, better than we could have initially hoped for. >> Throughout this exercise, there were periodic calls for "just toss it and invent >> something new" (which we sometimes called "snitch", shorthand for "new >> switch"*), and no shortage of people's attempts to design their ideal switch >> construct. We resisted this line of attack, because we believed having two >> similar-but-different constructs living side by side would be more annoying >> (and confusing) to users than a rehabilitated, albeit more complex, construct. >> The first round of improvements came with expression switches. This was the easy >> batch, because it didn't materially change the set of questions we could ask >> with switch, just the form in which we asked the question. This brought the >> following improvements: >> - Switches as expressions. Many existing switch statements are in reality >> modeling expressions, in a more roundabout and less safe way. Expressing it >> directly is simpler and less error-prone. >> - Checked totality. The compiler enforces that a switch expression is exhaustive >> (because, expressions must be total). In the case of enum switches, a switch >> that covers all the cases needs no default clause, and the compiler inserts an >> extra case to catch novel values and throw (ICCE) on them. (Eventually the same >> will be true for switches on sealed classes as well.) >> - A fallthrough-free option. Switches now give us a choice between two styles of >> _switch blocks_, the old willy-nilly style, and the new single-consequent >> (arrow) style. Switches that choose arrow-style need not reason about >> fallthrough. >> Unfortunately, it also brought a new asymmetry; switch expressions must be total >> (and you get enhanced type checking for this), but switch statements cannot be. >> This is a shame, since the improved type checking for totality is one of the >> best things about the improvements in switch, as a switch that is total by >> virtue of actually covering all the cases acts as a tripwire against new enum >> constants / permitted subtypes being added later, rather than papering it over >> with a catch-all. We explored several ways to explicitly add back totality >> checking, but this always felt like a hack, and requires the programmer to >> remember to ask for this checking. >> Our resolution here offers a path to true healing with minimal user impact, by >> (temporarily) carving out the semantic space of old statement switches. A >> "legacy switch" is a statement switch on a numeric primitive or its box, enum, >> or string, and which contains no pattern labels (i.e., a statement switch that >> is valid today.) Like expression switches, we will require non-legacy statement >> switches to be exhaustive, and warn on non-exhaustive legacy switches. (To make >> the warning go away, just insert a "default: " or "default: break" at the >> bottom of the switch; not painful.) After some time, we should be able to make >> this warning an error, which again is easy to mitigate with a single line. In >> the end, all switch constructs will be total and type-checked for >> exhaustiveness, and once done, the notion of "legacy switch" can be >> garbage-collected. >> Looking ahead to patterns in switch, we have several legacy considerations to >> navigate: >> - Fallthrough and bindings. While fallthrough is not inherently problematic >> (though the choice of fallthrough-by-default was unfortunate), if a case label >> introduces a pattern variable, then fallthrough to another case (at least one >> that doesn't introduce the same pattern variable with the same type) makes >> little sense, and such fallthrough has been outlawed. >> - Scoping. The block of a switch is one big scope, rather than each case label >> group being its own scope. (Again, one might call this a historical error, >> since there's little good that comes from this.) With case labels introducing >> variable declarations, this could have been a big problem, if one case polluted >> later cases (forcing users to pick unique names for each binding in a switch >> statement), but flow scopoing solves that one. >> - Nulls. In Java 1.0, switching over reference types was not permitted, so we >> didn't have to worry about this. In Java 5, autoboxing and enums meant we could >> switch over some reference types, but for all of these, null was a "silly" >> value so we didn't care about NPEing on null. In Java 7, when we added string >> switch, we could have conceivably allowed `case null`, but instead chose to >> follow the precedent set by Java 5. But once we introduce switches over any >> type, with richer patterns, eagerly NPEing on null becomes much more >> problematic. We've navigated this by say that switches can NPE on null if they >> have no nullable cases; nullable cases are those that explicitly say "null", >> and total patterns (which always come last since they dominate all others.) The >> old rule of "switches throw on null" becomes "switches throw on null, except >> when they say 'case null' or the bottom case is total." Default continues to >> mean what it always did -- "anything not already matched, except null." >> The new treatment of null actually would have fallen out of the decisions on >> totality, had we not gotten there already via another path. Our notion of >> totality accounts for "remainder", which includes things like novel subclasses >> of sealed types that did not exist at compile time, which it would not be >> reasonable to ask users to write code to deal with, and null fits into this >> treatment as well. We type check that a switch is sufficiently total, and then >> insert extra code to catch "silly" values that are not otherwise handled, >> including null, and throw. (This also enables DA analysis to truly trust switch >> totality.) >> Where we land is a single unified switch construct that can be either a >> statement or an expression; that can use either old-style flow (colon) or the >> more constrained flow style (arrow); whose case labels can be constant, >> patterns (including guarded patterns), or a mix of the two; which can accept >> the legacy null-hostility behavior, or can override it by explicitly using >> nullable case labels; and which are almost always type checked for totality >> (with some temporary, legacy exceptions.) Fallthough is basically unchanged; >> you can get fallthrough when using the old-style flow, but becomes less >> important as fallthrough is (mostly) nonsensical in the presence of pattern >> cases with bindings, and the compiler prevents this misuse. The distinction >> between "legacy" switches and pattern switches is temporary, with a path to >> getting to "all switches are total" over time. >> I think we've done a remarkable job at rehabilitating this monster. >> *Someone actually suggested using the syntax "new switch", on the basis that new >> was already a keyword. Would not have aged well. From guy.steele at oracle.com Wed May 19 15:11:43 2021 From: guy.steele at oracle.com (Guy Steele) Date: Wed, 19 May 2021 11:11:43 -0400 Subject: [External] : Re: Rehabilitating switch -- a scorecard In-Reply-To: <933304581.2115158.1621421000777.JavaMail.zimbra@u-pem.fr> References: <209e43a4-e4c8-e3be-1c2b-c66431396721@oracle.com> <933304581.2115158.1621421000777.JavaMail.zimbra@u-pem.fr> Message-ID: > On May 19, 2021, at 6:43 AM, forax at univ-mlv.fr wrote: > > . ..\ . > This remember me something, tail call optimization is not the only optimization that avoid the stack to grow. > > If you have calls like > g(a, () -> h(a, () -> f(a))) > > with 'a' being the same arguments, you can transform them to > r1 = f(a) > r2 = h(a, () -> r1) > r3 = g(a, () -> r2) > > Here the calls can be at any location in the method, but the argument should not depend on the values computed by the function. > > The interceptors of Spring or CDI exhibit calls like this, they are using the same arguments so the result do not depend on the order of which the functions/interceptors are executed, > but as far as i know, there is not implementation that do that transformation so when an exception occurs we see long stack traces. > > Asking for a friend, is this transformation have already a name ? Interesting! No, I don?t recognize this transformation, and don?t have a name for it. It reminds me a little bit of ?code hoisting?, that is, moving code earlier (typically so as to lift, or ?hoist?, it out of a loop), because a prerequisite for this transformation is that side effects in f, g, and h may be reordered. In effect, f(a) is being computed earlier than normal, and similarly for h. In fact, if g and h invoke their second arguments only conditionally, then f(a) may be called even in situations where it might not have been called in the original code. ?Guy From john.r.rose at oracle.com Fri May 21 23:49:20 2021 From: john.r.rose at oracle.com (John Rose) Date: Fri, 21 May 2021 23:49:20 +0000 Subject: [External] : Re: Rehabilitating switch -- a scorecard In-Reply-To: References: <209e43a4-e4c8-e3be-1c2b-c66431396721@oracle.com> Message-ID: <36926CF6-BC2E-4193-A00A-0F4803659DF8@oracle.com> On May 18, 2021, at 12:33 PM, Guy Steele > wrote: If only we had tail calls. Then instead of writing (int q, int r) = quotientAndRemainder(m, n); whatever we could write (using my preferred syntax for a mandatory tail call in Java) goto quotientAndRemainder(m, n, (int q, int r) -> whatever); (the method quotientAndRemainder would of course tail-call its third argument), with no need of tuples or even records to get multiple values out of a method, and then everyone would be happy, right? RIGHT? :-) :-/ :-P As it happens, this week I have been enjoying ?Compiling without Continuations? by Maurer at al [1]. Their work is in Haskell which already has tail calls, but needs (for some key optimizations) a primitive which is a non-local jump with arguments. (They mention Rabbit but not Lambda the Ultimate Goto. :-) It basically throws, not an exception but an argument bundle, to a statically (not dynamically) determined catch point; it compiles to a goto, assuming it the special catch and jump points are only introduced after inlining. Using their jump primitive, the Q&R call looks something like this: QR: join(int q, int r) { quotientAndRemainder(m, n, QR); } and AlwaysThrows quotientAndRemainder(int m, int n, BiConsumer QR) { int q = m/n, r = m-n*q; throw QR.apply(q, r); } If it could compile to JVM-level LET-JUMP-POINT (?join?) and GOTO-JUMP-POINT (?jump?) IR commands, then you could get the right kind of ?ultimate goto? by the route of throws instead of tail calls. Threading the types through exceptions is very icky though. Also, tail calls, proper, allow you to jump to any dynamically specified next-callee, not just a selection of catch points already on the stack. Still, in this example, a statically present catch point is enough. ? John [1]: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/11/compiling-without-continuations.pdf From fw at deneb.enyo.de Sat May 22 12:51:30 2021 From: fw at deneb.enyo.de (Florian Weimer) Date: Sat, 22 May 2021 14:51:30 +0200 Subject: Rehabilitating switch -- a scorecard In-Reply-To: <933304581.2115158.1621421000777.JavaMail.zimbra@u-pem.fr> (forax's message of "Wed, 19 May 2021 12:43:20 +0200 (CEST)") References: <209e43a4-e4c8-e3be-1c2b-c66431396721@oracle.com> <933304581.2115158.1621421000777.JavaMail.zimbra@u-pem.fr> Message-ID: <87pmxjorsd.fsf_-_@mid.deneb.enyo.de> * forax: > This remember me something, tail call optimization is not the only > optimization that avoid the stack to grow. > > If you have calls like > g(a, () -> h(a, () -> f(a))) > > with 'a' being the same arguments, you can transform them to > r1 = f(a) > r2 = h(a, () -> r1) > r3 = g(a, () -> r2) > > Here the calls can be at any location in the method, but the > argument should not depend on the values computed by the function. > Asking for a friend, is this transformation have already a name ? It looks a bit like strictness analysis. However it would specialize h, g to the non-thunk version, and the end result would look like this: r1 = f(a) r2 = h(a, r1) r3 = g(a, r2)