Totality at switch statements

[Hunor Szegi]

Thank you for your detailed answer.

I agree that the developers will have to change their way of thinking using
the new Java features. I am primarily a Java developer, but I use Scala as
well. I remember how challenging it was to use it "properly". There was no
strict force, at least not always, but we adapted to the new programming
paradigm. (For example we always use immutable variables, while the
language allows the mutability as well.) Of course we use pattern matching
and sealed classes as well, so I fully get that the Java switch enhancement
isn't only about the switch. It will have a much bigger effect, but
possibly not everything should be forced by the language, the community and
the usefulness can have a similar effect.

Surely with pattern matching the intended totality will be much more
frequent compared to the use cases with the old switch. And I agree, the
totality check is useful. I have only reservation against forcing to make
the switch statement total. For my taste this is a good practice, but not a
compulsory requirement.

Looking at your example, when an additional sealed subclass will be added.
Yes, it would be very useful to warn the programmers. But imagine a
situation when the totality isn't intended, so the programmer wants a side
effect only in a few cases. She will be forced to add an empty default
clause. And later, when a new sealed subclass will be added, there won't be
any alert, as the switch is already total, forever. Surely, it helped a
little bit to increase the safety initially (the programmer was forced to
think about the missing cases), but with a lot of empty default clauses
this safety will be partial.

You are right, a partial benefit could be better than nothing. But a false
belief of safety can be dangerous as well.

If you want to minimize the usages of the empty default clauses, a strict
force creating those looks counterproductive. Raising a warning is a lesser
force. Checking the totality only when the programmer asked it (with a
@total annotation for example) seems the optimal solution. (Or the other
way round, marking it with a @partial annotation.) I like safety as a
requirement, like static typing against dynamic, maybe I'm just too liberal
here. :-)

I agree, the cognitive load on the new learners must also be taken into
account. But again, I think that it is more difficult to understand why the
enums are treated differently to sealed classes. (Historical, backward
compatibility reasons.) While that is very easy to understand why totality
is necessary at switch expressions, and why optional at switch statements.
It is logical.

You are right, with my preferred way, theoretically it is possible to
lose some safety by refactoring a switch expression to a statement. But the
original expression was forced to be total, possibly had a default clause,
so it isn't something that can be forgotten. And the original returned a
value, the statement will contain only the side effects. If there was no
side effect in some cases, why should the developer make a total statement?
Is it really a loss in safety?

BTW, Scala allows switch expressions without totality, only a warning is
raised. This seems a safety risk to me, no idea why it was implemented this
way.

>>* Additionally, there are issues with the "empty" default clause
**>> ...*

> This is a fair observation; this should probably be cleaned up.

Thank you, I hope it will be fixed by a non-preview release. (Hopefully by
the next LTS release.) If I have to use an empty default, my preferred way
would be the "default ->" syntax, which isn't possible now.

I have read your messages to Nathan as well. I agree with you, the default
clause would be better floated to the bottom with a fixed manner. In spite
of that fall though example, which is a possible code, but hard to read.
Placing an empty default somewhere in the middle shows even less clearly we
don't want totality. But even without the totality requirements, or with a
body, it seems breaking the logic, the more specific case should be placed
before a more universal one at pattern matching.

Regarding the automatically added throwing default if there was no default
case. Yes, this would be an additional difference between the statement and
expression, if the statement could lack totality.

I agree that it would be good if the programmer could show her intent about
the totality. And yes, adding an empty default clause shows that intent,
the totality isn't necessary. You are right about it. I'm just worried that
an empty clause isn't as explicit as something else could be.

Finally, I have an important point against me. :-) Forcing the totality can
be removed later without backward compatibility issues, while it isn't
possible to make the rules stricter later.

I can imagine you are right, and in the future, with more experience using
the new features I will be glad this decision was made. So it's only an
attempt to raise my thoughts before it will be too late for a change. I am
glad you shared your motivations.

Thank you very much,

Regards,
Hunor




On Sun, 19 Jun 2022 at 14:31, Brian Goetz <brian.goetz at oracle.com> wrote:

>
> Surely, the totality is necessary at switch expressions, but forcing it at
> a statement is questionable.
>
>
> I understand why you might feel this way; indeed, when we made this
> decision, I knew that we would get mails like this one.  There were a
> number of forces that pushed us in this direction; I will try to explain
> some of them, but I don't expect the explanation to be compelling to all
> readers.
>
> One of the things that makes upgrading `switch` difficult is that
> developers have an existing mental model of what switch "is" or "is for",
> and attempted upgrades often challenge these models.  But the goal here is
> not simply to make switch "better" in a number of ways (better
> exhaustiveness checking, better null handling, patterns, smoother syntax).
> Of course we like these improvements, but the goal is bigger.  But it's
> easier to be aware of the role switch plays today, than the role we expect
> it to play tomorrow, so these improvements might feel more like "forced
> improvements for the sake of some abstract ivory tower purity."
>
> It can be helpful for the programmers to check the totality in those cases
> when the intent is that. But it is quite common to create a switch
> statement that doesn't handle all of the possibilities.
>
>
> This raises two questions:
>
>  - Why is it so common?
>  - Is it good that it is common?
>
> One of the reasons it is common is that the switch statement we had is so
> weak!  The set of types you can switch over is limited.  The set of things
> you can put in case labels is limited.  The set of things you can do is
> limited (until recently, you were forced to do everything with
> side-effects.)  The switch statement we have in Java was copied almost
> literally from C, which was designed for writing things like lexers that
> look at characters and accumulate them conditionally into buffers.
> Partiality is common in these weak use cases, but in the use cases we want
> to support, partiality is more often a bug than a feature.  So saying "it
> is common" is really saying "this is what we've managed to do with the
> broken, limited switch statement we have today."  Great that we've been
> able to do something with it, but we shouldn't limit our imagination to
> what we've been able to achieve with such a limited tool.
>
> To my other question, try this thought experiment: if switch was total
> from day 1, requiring a `default: ;` case to terminate the switch (which
> really isn't that burdensome), when you were learning it back then, would
> you even have thought to complain?  Would you even have *noticed*?  If
> there was a budget for complaining about switch in that hypothetical world,
> I would think 99% of it would have been spent on fallthrough, rather than
> "forced default".
>
> Why would it be different using patterns? Why is it beneficial to force
> totality?
>
>
> Because patterns don't exist in a vacuum.  There's a reason we did
> records, sealed classes, and patterns together; because they work
> together.  Records let us easily model aggregates, and sealed types let us
> easily model exhaustive choices (records + sealed classes = algebraic data
> types); record patterns make it easy to recover the aggregated state, and
> exhaustive switches make it easy to recover the original choice.  We expect
> that the things people are going to switch over with patterns, will have
> often been modeled with sealed classes.
>
> Java has succeeded despite having gotten many of the defaults wrong.
> We've all had to learn "make things private unless they need to be
> public."  "Make fields final unless they need to be mutable."  It would
> have been nice if the language gave us more of a nudge, but we had to learn
> the hard way.  Switch partiality is indeed another of those wrong defaults;
> you don't notice it until it is pointed out to you, but then when you think
> about it for enough time, you realize what a mistake it was.
>
> In the current world, partiality is the default, and even if a switch is
> total, it may not be obvious (unless you explicitly say "default");
> flipping this around, a switch with default is a sign that says "hey, I'm
> partial."  Partiality is an error-prone condition that is worth calling
> attention to, so flipping this default is valuable -- and we have an
> opportunity to do so without breaking compatibility.
>
> The value of totality checking is under-appreciated, in part because until
> recently there were so few sources of exhaustiveness information in the
> language (basically, enums).  But many non-exhaustive switches are an error
> waiting to happen; the user thinks they have covered all the cases (either
> in fact, or in practicality).  But something may happen elsewhere that
> undermines this assumption (e.g., a new enum constant or subtype was
> added.)  With totality, we are made aware of this immediately, rather than
> having to debug the runtime case where a surprising value showed up.
>
> Worse, now the language has switch expressions, which *must* be total.
> Having one kind of switch be total and another not is cognitive load that
> users (and students) have to carry.  (Yes, there are still asymmetries in
> switch that have this effect; that's not a reason to load up with more.)
> But it gets even worse, because refactoring from a switch expression to a
> switch statement means you lost some safety that you were depending on when
> you wrote the original code, and may not be aware of this.
>
> If you've not programmed with sealed types or equivalent, it is easy to
> underestimate how powerful this is.  I'd like us to be able to get to a
> world where we almost never use "default" in switch, unless we are
> deliberately opting into partiality -- in which case the "default" is a
> reminder to future maintainers that this is a deliberately partial switch.
>
> This check can be an IDE feature.
>
>
> Yes, that was one of the choices.  And we considered that.  And it is
> reasonable that you wish we'd made another choice.  But, be aware you are
> really arguing "make the language less safe and more error-prone, please"
> -- and ask yourself why you think its a good idea to make the language less
> uniform and less safe?  I think you'll find that the reason is mostly
> "someone moved my cheese."  (
> https://urldefense.com/v3/__https://en.wikipedia.org/wiki/Who_Moved_My_Cheese*3F__;JQ!!ACWV5N9M2RV99hQ!OGvj_ez3M-Rb0G93AvynF_Hcdr-5f8cE-sTr7OxEXJEnGxKl-jHXYwLubpyZnfpkvDgjAf8ZARG71rKaN0DcpX0$ ).
>
> Honestly I feel that the rule, when the totality is forced, is dictated
> simply by the necessity of backward compatibility. What will happen if a
> new type (for example double) will be allowed for the selector expression?
> The backward compatibility wouldn't be an issue, but it would be weird
> behaving differently with an int compared to a double, so I guess the
> totality won't be forced. What would happen if the finality requirement was
> removed, and the equality could be checked for all types? What about the
> totality check in this imagined future?
>
>
> I don't really understand what you're getting at in this paragraph, but
> I'll just point out that it really underscores the value of a uniform
> switch construct.  You're saying "but I don't see how you'll get there from
> legacy int switches, so therefore its inconsistent" (and implicitly, one
> unit of inconsistency is as bad as a million.)  But you don't seem to be
> equally bothered by the much more impactful inconsistency we'd have if
> expression switches were total and statement switches were not.
>
> You are correct that there are legacy considerations that will make it
> harder to get to a fully uniform construct (but, there's still things we
> can do there.)  But that's not an excuse to not design towards the language
> we want to have, when we can do so at such minor inconvenience.
>
> But the main thing I want you to consider is: right now, the switch we
> have is very, very limited, and so we've convinced ourselves it is "for"
> the few things we've been able to do with it.  By making it more powerful
> (and combining it with complementary features such as pattern matching, and
> sealing), these few cases -- which right now feel like the whole world of
> switch -- will eventually recede into being the quirky odd cases.
>
> Additionally, there are issues with the "empty" default clause. In the JEP
> the "default: break;" was recommended, but interestingly it doesn't work
> with the arrow syntax. ("default -> break;" is a compile time error, only
> the "default: {break;}" is possible.) We can use both the "default: {}" and
> "default -> {}", which is fine. But while the "default:" is
> possible (without body), the "default ->" is an error. I don't know what is
> the reason behind it. Allowing an empty body with the arrow syntax would
> make the actual solution a little bit cleaner.
>
>
> This is a fair observation; this should probably be cleaned up.
>
> It would be possible to allow the programmer to mark the intended
> totality. Maybe a new keyword would be too much for this purpose.
>
>
> Yes, we considered this, and came to the conclusion that the problem is
> the wrong default.  Adding a new keyword for "total switch" is bad in three
> ways: it is, as you say, "too much"; it doesn't fix the underlying problem;
> and the cases in which it most needs to be used, people will probably
> forget to use it.
>
>
On Sun, 19 Jun 2022 at 14:31, Brian Goetz <brian.goetz at oracle.com> wrote:

>
> Surely, the totality is necessary at switch expressions, but forcing it at
> a statement is questionable.
>
>
> I understand why you might feel this way; indeed, when we made this
> decision, I knew that we would get mails like this one.  There were a
> number of forces that pushed us in this direction; I will try to explain
> some of them, but I don't expect the explanation to be compelling to all
> readers.
>
> One of the things that makes upgrading `switch` difficult is that
> developers have an existing mental model of what switch "is" or "is for",
> and attempted upgrades often challenge these models.  But the goal here is
> not simply to make switch "better" in a number of ways (better
> exhaustiveness checking, better null handling, patterns, smoother syntax).
> Of course we like these improvements, but the goal is bigger.  But it's
> easier to be aware of the role switch plays today, than the role we expect
> it to play tomorrow, so these improvements might feel more like "forced
> improvements for the sake of some abstract ivory tower purity."
>
> It can be helpful for the programmers to check the totality in those cases
> when the intent is that. But it is quite common to create a switch
> statement that doesn't handle all of the possibilities.
>
>
> This raises two questions:
>
>  - Why is it so common?
>  - Is it good that it is common?
>
> One of the reasons it is common is that the switch statement we had is so
> weak!  The set of types you can switch over is limited.  The set of things
> you can put in case labels is limited.  The set of things you can do is
> limited (until recently, you were forced to do everything with
> side-effects.)  The switch statement we have in Java was copied almost
> literally from C, which was designed for writing things like lexers that
> look at characters and accumulate them conditionally into buffers.
> Partiality is common in these weak use cases, but in the use cases we want
> to support, partiality is more often a bug than a feature.  So saying "it
> is common" is really saying "this is what we've managed to do with the
> broken, limited switch statement we have today."  Great that we've been
> able to do something with it, but we shouldn't limit our imagination to
> what we've been able to achieve with such a limited tool.
>
> To my other question, try this thought experiment: if switch was total
> from day 1, requiring a `default: ;` case to terminate the switch (which
> really isn't that burdensome), when you were learning it back then, would
> you even have thought to complain?  Would you even have *noticed*?  If
> there was a budget for complaining about switch in that hypothetical world,
> I would think 99% of it would have been spent on fallthrough, rather than
> "forced default".
>
> Why would it be different using patterns? Why is it beneficial to force
> totality?
>
>
> Because patterns don't exist in a vacuum.  There's a reason we did
> records, sealed classes, and patterns together; because they work
> together.  Records let us easily model aggregates, and sealed types let us
> easily model exhaustive choices (records + sealed classes = algebraic data
> types); record patterns make it easy to recover the aggregated state, and
> exhaustive switches make it easy to recover the original choice.  We expect
> that the things people are going to switch over with patterns, will have
> often been modeled with sealed classes.
>
> Java has succeeded despite having gotten many of the defaults wrong.
> We've all had to learn "make things private unless they need to be
> public."  "Make fields final unless they need to be mutable."  It would
> have been nice if the language gave us more of a nudge, but we had to learn
> the hard way.  Switch partiality is indeed another of those wrong defaults;
> you don't notice it until it is pointed out to you, but then when you think
> about it for enough time, you realize what a mistake it was.
>
> In the current world, partiality is the default, and even if a switch is
> total, it may not be obvious (unless you explicitly say "default");
> flipping this around, a switch with default is a sign that says "hey, I'm
> partial."  Partiality is an error-prone condition that is worth calling
> attention to, so flipping this default is valuable -- and we have an
> opportunity to do so without breaking compatibility.
>
> The value of totality checking is under-appreciated, in part because until
> recently there were so few sources of exhaustiveness information in the
> language (basically, enums).  But many non-exhaustive switches are an error
> waiting to happen; the user thinks they have covered all the cases (either
> in fact, or in practicality).  But something may happen elsewhere that
> undermines this assumption (e.g., a new enum constant or subtype was
> added.)  With totality, we are made aware of this immediately, rather than
> having to debug the runtime case where a surprising value showed up.
>
> Worse, now the language has switch expressions, which *must* be total.
> Having one kind of switch be total and another not is cognitive load that
> users (and students) have to carry.  (Yes, there are still asymmetries in
> switch that have this effect; that's not a reason to load up with more.)
> But it gets even worse, because refactoring from a switch expression to a
> switch statement means you lost some safety that you were depending on when
> you wrote the original code, and may not be aware of this.
>
> If you've not programmed with sealed types or equivalent, it is easy to
> underestimate how powerful this is.  I'd like us to be able to get to a
> world where we almost never use "default" in switch, unless we are
> deliberately opting into partiality -- in which case the "default" is a
> reminder to future maintainers that this is a deliberately partial switch.
>
> This check can be an IDE feature.
>
>
> Yes, that was one of the choices.  And we considered that.  And it is
> reasonable that you wish we'd made another choice.  But, be aware you are
> really arguing "make the language less safe and more error-prone, please"
> -- and ask yourself why you think its a good idea to make the language less
> uniform and less safe?  I think you'll find that the reason is mostly
> "someone moved my cheese."  (
> https://urldefense.com/v3/__https://en.wikipedia.org/wiki/Who_Moved_My_Cheese*3F__;JQ!!ACWV5N9M2RV99hQ!OGvj_ez3M-Rb0G93AvynF_Hcdr-5f8cE-sTr7OxEXJEnGxKl-jHXYwLubpyZnfpkvDgjAf8ZARG71rKaN0DcpX0$ ).
>
> Honestly I feel that the rule, when the totality is forced, is dictated
> simply by the necessity of backward compatibility. What will happen if a
> new type (for example double) will be allowed for the selector expression?
> The backward compatibility wouldn't be an issue, but it would be weird
> behaving differently with an int compared to a double, so I guess the
> totality won't be forced. What would happen if the finality requirement was
> removed, and the equality could be checked for all types? What about the
> totality check in this imagined future?
>
>
> I don't really understand what you're getting at in this paragraph, but
> I'll just point out that it really underscores the value of a uniform
> switch construct.  You're saying "but I don't see how you'll get there from
> legacy int switches, so therefore its inconsistent" (and implicitly, one
> unit of inconsistency is as bad as a million.)  But you don't seem to be
> equally bothered by the much more impactful inconsistency we'd have if
> expression switches were total and statement switches were not.
>
> You are correct that there are legacy considerations that will make it
> harder to get to a fully uniform construct (but, there's still things we
> can do there.)  But that's not an excuse to not design towards the language
> we want to have, when we can do so at such minor inconvenience.
>
> But the main thing I want you to consider is: right now, the switch we
> have is very, very limited, and so we've convinced ourselves it is "for"
> the few things we've been able to do with it.  By making it more powerful
> (and combining it with complementary features such as pattern matching, and
> sealing), these few cases -- which right now feel like the whole world of
> switch -- will eventually recede into being the quirky odd cases.
>
> Additionally, there are issues with the "empty" default clause. In the JEP
> the "default: break;" was recommended, but interestingly it doesn't work
> with the arrow syntax. ("default -> break;" is a compile time error, only
> the "default: {break;}" is possible.) We can use both the "default: {}" and
> "default -> {}", which is fine. But while the "default:" is
> possible (without body), the "default ->" is an error. I don't know what is
> the reason behind it. Allowing an empty body with the arrow syntax would
> make the actual solution a little bit cleaner.
>
>
> This is a fair observation; this should probably be cleaned up.
>
> It would be possible to allow the programmer to mark the intended
> totality. Maybe a new keyword would be too much for this purpose.
>
>
> Yes, we considered this, and came to the conclusion that the problem is
> the wrong default.  Adding a new keyword for "total switch" is bad in three
> ways: it is, as you say, "too much"; it doesn't fix the underlying problem;
> and the cases in which it most needs to be used, people will probably
> forget to use it.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-dev/attachments/20220620/0a021bcb/attachment-0001.htm>