PROPOSAL: Null safe type system

[Alessandro Autiero]

My proposal to leverage Optional was in fact to make this feature feel
comfortable with everyone, especially considering that this wrapper has
been part of the language for quite a while now and has seen considerable
use, also in JDK libraries(see the new HTTP API for example). It's clear
though from the points you raised and others that were raised in a previous
conversation that this approach is far from ideal. Using synthetic null
checks is probably a better approach, but doing so requires having a way
for the compiler to tell if a type is nullable, non-null or legacy from
bytecode, for example when a library uses the old type system. The latter
case is easy enough, but what about the other two? For example, Kotlin's
compiler uses annotations to understand this. Here is an example:
(Kotlin)

fun hello(a: String){}
fun anotherHello(a: String?){}


(Decompiled Java code from bytecode)

import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;

public final void hello(@NotNull String a) {}

public final void anotherHello(@Nullable String a) {}

I think that a very good approach would be to also leverage annotations in
Java as we could use them to have a foundation for type migration. For
example, if a library uses the legacy type system, but has type annotations
the java compiler can infer some types more effectively instead of
defaulting to platform types always. I propose though for these annotations
to be reserved for the compiler in order to not create any
misunderstandings about how they are supposed to be used. What do you think
though about my proposal to use the module system to make this type system
opt-in and the other operators that I suggested?


On Mon, May 16, 2022, 15:29 Brian Goetz <brian.goetz at oracle.com> wrote:

> There's a few things here.  First, you make an argument for "its time to
> bring nullability into the type system."  No arguments with your argument,
> but let's be aware that this is a significant project -- not only does it
> fundamentally affect the type system, but we have 25 years of libraries
> that have been written for "unknown nullity".  Just as generics was huge
> *and then* generifying the existing libraries was even bigger, the same
> relationship is in play here.
>
> While I think it is a good time to raise awareness of the issue, I think
> its premature to try and pick solutions here.  Because, Valhalla is going
> to generate some of the answers, and some new constraints, with regard to
> nullity.
>
> The effort would probably be considerable though in the case of the
> compiler.
>
>
> In general, for any serious language feature, the compiler work is never
> the long pole.  The long pole is designing the feature so it looks like
> it's always been there, both from a specification-facing and user-facing
> perspective.
>
> The high-order bit of your proposal is to use the Optional class as the
> representation for nullable values; translate `Foo?` as `Optional<Foo>`.  I
> think this is a false economy, though; there is so much work required to
> make this lift, that the benefit of reusing Optional is very small, and it
> carries with it some constraints you might not want.  For example:
>
>     void m(Foo? blah) { ... }  // method descriptor (LOptional;)V
>     void m(Bar? blah) { ... }  // method descriptor (LOptional;)V
>
> error: Foo? and Bar? both have the same erasure
>
> Ouch!  You've trade one big problem for another, more subtle one: you
> can't overload two nullable types.  This is not only painful, it violates
> the principle of least astonishment.  I'd rather erase `Foo?` to Foo, and
> insert synthetic null checks at the boundary between `T?` and `T`, just as
> we insert synthetic casts at the boundary between generic code and its
> non-generic clients.
>
> In other words: this proposal names a problem worth solving, but picks a
> solution that seems (a) a false economy and (b) creates a host of new
> problems.
>
> Eclipse null checking framework allows the programmer to put
> @NonNullByDefault in front of the package declaration in
> package-info.java.  Why not create a JDK annotation instead of a flag?
>
>
> You should take a look at the `jspecify` effort (https://jspecify.dev/)
> which is a community effort trying to unify the various nullity
> annotations.
>
>
>
> On 5/15/2022 1:59 PM, Alessandro Autiero wrote:
>
> Null safe type system
>
> Hello, this is my first time writing a proposal for OpenJDK, but I hope
> that it will meet the quality standards.
> I've looked through the discussions in the OpenJDK mailing lists, but I
> couldn't find any regarding this issue, so I thought I'd propose something
> on my own.
> I discussed this proposal previously on the java subreddit here<https://www.reddit.com/r/java/comments/tiw76i/backwards_compatible_null_safe_nonnull_by_default/> <https://www.reddit.com/r/java/comments/tiw76i/backwards_compatible_null_safe_nonnull_by_default/>to
> collect ideas to make this proposal as comprehensive as possible.
> I apologize in advance for any mistakes.
>
> **OVERVIEW**
>
> The Java Type System is made up of two different types: primitives and
> objects.
> While the first cannot be assigned to the special type null, the latter can.
> If an expression, for example, a method invocation, takes as an argument a
> null reference a NullPointerException is thrown.
> This design choice introduces the need to check for null values to make
> sure that this doesn't happen.
> The most common values that are checked are method parameters, especially
> if the method is exposed publicly, and ones returned by method invocations.
> To make the latter easier, in Java 8 the java.util.Optional wrapper class
> was introduced.
> Being a class, Optional is not suitable as an equivalent to, for example,
> Swift's Optional which is used often as a method parameter.
> Despite this obvious design choice, some very popular libraries in the Java
> Ecosystem, most notably Selenium in my experience, have chosen to use
> Optional in the context just described.
> This indicates firmly that there is a need for better handling of null in
> Java's type system.
> As a matter of fact, annotations have become a sort of fix for this
> issue(see the Checker Framework for example), though, in my opinion, this
> issue should be resolved in the type system itself.
> What I'd like to propose is to improve Java's type system by leveraging the
> Optional wrapper, the module system introduced in Java 9(Jigsaw) and the
> recent developments over at Project Valhalla to better handle the special
> type null.
> Considering that our neighbour in the JVM universe, Kotlin, has been
> designed with a type system that takes into account exactly this issue, I
> think that it's appropriate to discuss JetBrain's approach.
> Kotlin's type system, differently from Java's, is made up only of objects.
> While non-null scalar types, such as Int, use JVM primitive types as an
> implementation detail, they are not part of the language.
> Object types can be nullable(the name of the type is followed by a question
> mark), non-null(the name of the type) or platform types(the name of the
> type is followed by an exclamation mark) to provide a compatibility layer
> with Java's type system.
> In the latter case, the programmer should explicitly declare the type as
> nullable or non-null(it's not technically needed as the compiler will issue
> a warning and not an error, but it's advised), while in the others can be
> inferred.
> In practice, this means that, for example, the type Int and Int? are two
> completely different types for the compiler.
> While this approach can be appropriate when designing a new language, like
> in Kotlin's case, it certainly isn't feasible in Java's case considering
> its twenty years of legacy.
> Now let's look at Swift's approach instead. Swift's type system allows for
> nullable and non-null object values and follows the same naming conventions
> as Kotlin, except platform types aren't a thing here.
> Furthermore, nullable types aren't compiler magic in this case: they are an
> alias for the Optional enum.
> For example, the type Int? and Optional<Int> are the same type.
> This also means that declaring, for example, a variable with type Int?? is
> legal, as this simply translates to Optional<Optional<Int>>.
> This approach is better suited for Java as we already have a very similar
> class that we can leverage: java.util.Optional.
> Implementing the same concept in Java wouldn't have been previously
> possible because the Optional wrapper, being a class, is too slow.
> Thanks to Project Valhalla, though, the Optional wrapper can be transformed
> in a primitive class.
> Non-null values assigned to Optional types should be autoboxed by using
> Optional<T>#of(T), nullable values should be autoboxed by using
> Optional<T>#ofNullable(T) while the null literal should be an alias of
> Optional<T>#empty().
> This can either be done with some compiler magic, like we currently do with
> primitive wrappers, or by introducing a full-fledged feature to support
> boxing and unboxing for classes, but this conversation is not relevant here
> I think.
> This means that the performance hit should no longer be a concern.
> NOTE: An instance of a primitive class doesn't allow for null initializers,
> though, considering the previously described boxing and unboxing mechanism
> this should be syntactically legal. If this approach introduced ambiguity,
> though, a better approach would be to use a value class instead.
>
> **CHALLENGES**
>
> As previously mentioned, Java's legacy makes implementing such a feature
> quite hard when considering backwards compatibility.
> The first step is obviously to make Optional into a value class and this is
> possible without breaking anything, as far as I know.
> Furthermore, it's also necessary to make sure that a developer using a Java
> version with this new hypothetical type system can, for example, use a
> library built using a version of Java that doesn't.
> This can be technically done by treating older types as equivalent to
> platform types and allowing javac to place null checks(NULLCHK operator)
> when needed.
> While platform types as a concept aren't strictly necessary, I think that
> having older types be nullable by default is not the right approach as if
> the programmer wants to declare the variable as non-null requires implicit
> conversion(from ExampleType? to ExampleType), which goes against the
> purpose of this type system, or a manual cast which can be perceived as
> very verbose and redundant.
> This challenge was already faced by Google when they transitioned to Dart
> 2.12 which introduced the concept of nullable and non-null types.
> Finally, the only challenge would be to make migrating a project from an
> older Java version as painless as possible.
> For this new type system to work effectively, types should be non-null by
> default.
> Though, Java's current type system has object types nullable by default.
> This means that if a developer were to transition to this new hypothetical
> version, everything in the codebase would break.
> To face this issue, I think that introducing a flag, for example,
> sound(sound null safety), in the module-info, just like open, is the best
> approach.
> The default flag would be, for example, unsound, so that backwards
> compatibility is preserved even if a module-info is already present.
> Some may argue though that migrating a whole module to sound null safety is
> unlikely in major, especially enterprise, applications all at once.
> This is the reason why we might consider having this flag also on a package
> level by using the package info.
> An idea would be to have the flag in front of the package declaration,
> though there are currently no flags available for packages in package info,
> having different type systems in a single module is not optimal for a
> developer working on a codebase and there should also be a hierarchical
> resolution of flags(for example a package is unsound but its parent module
> is sound, what do we do?).
> Apart from these implementation details, this approach ensures backwards
> compatibility and an easy transition for developers to newer versions.
>
> **COMPLEMENTARY OPERATORS**
>
> To support such a type system, at least the null-safe de-referencing
> operator(?.) would need to be introduced.
> This should not break any existing application because the question mark is
> not a legal character for variables' names, so no ambiguities can occur.
> Other useful operators would be the Elvis operator(?:) which is a binary
> operator that returns the left operand if it's not null otherwise the right
> one and the double bang operator which is a unary operator that throws a
> NullPointerException if its operand is null.
> These aren't strictly needed, but they certainly improve the developer
> experience.
>
> **COMPLEMENTARY KEYWORDS**
>
> Introducing an equivalent to the var keyword, introduced in Java 10, for
> nullable types could be appropriate.
> Let's say that the initializer of a variable is a literal, but the
> developer later wants to assign a new value to this variable: how should
> the compiler know if the type to infer is ExampleType or ExampleType?.
> The var? keyword would only allow for nullable types, while the var keyword
> for both nullable and non-nullable.
> This, though, should probably be discussed.
>
>
> **EXAMPLES**
>
> Module Flags:
>
> module com.example.project { // The module has unsound null safety
>
> }
>
> sound module com.example.project { // The module has sound null safety
>
> }
>
> unsound module com.example.project { // Unsound is redundant, the module
> has unsound null safety
>
> }
>
> Example interaction between legacy code and modern code:
>
> public void someMethod(){
>    String? someValue = OldAPI.getSomeValue(); // The return type of
> getSomeValue is specified by the programmer to be nullable, no checks needed
>    String someOtherValue = OldAPI.getSomeOtherValue(); // The return type
> of getSomeValue is specified by the programmer to be non-null, the compiler
> should include a NULLCHK
>    var inferredValue = OldAPI.getSomeValue(); // The return type of
> getSomeValue is inferred to be nullable by var considering it's a platform
> type
> }
>
> **CONCLUSION**
>
> Discussing the technical changes needed for javac and for the JLS is
> necessary, though this proposal will probably need a lot of discussion and
> opinions before we get to this stage in my opinion.
> The effort would probably be considerable though in the case of the
> compiler.
>
>
>