PROPOSAL: Null safe type system

[Alessandro Autiero]

Sorry for anyone reading this, but I didn't forward my response to the
mailing list. My bad, sorry, it's my first time using one and I didn't
check. Here is an extract of the last exchange:

Me:
The first assignment should compile, in fact it's equivalent to:
Optional<String> value = Optional.empty();
This is also the case in Swift for example.

I would say that instanceof should work in this way:
String? value = "value";
value instanceof String; // false
value instanceof Optional; // true

The reason why value is not an instance of String is that the following
assignment is illegal:
String? value = "value";
String implicit = value; // String? is not applicable to String
String cast = (String) value; // ClassCastException

Furthermore checking if a value is an instance of a nullable type should
not be allowed considering type erasure:
String? value = "value";
value instanceof String?; // String? is equivalent to Optional<String>
which is not something we can check against
I don't think that this is an important limitation considering that this is
already the case for collections for example.
I think that it would be pretty obvious for developers to understand that
String? is not an instance of String, so there should be no ambiguous there.

The null check should check the value inside the optional type, so:
String? value = null;
value == null; // true
String? anotherValue = "value";
anotherValue == null; // False

This would imply that Optional.empty() == null, which makes sense in my
opinion in this context. A problem would be that currently Optional.empty()
!= null which would break backwards compatibility. If otherwise we decide
that Optional.empty() is not equal to null than there would be the
ambiguity that you described. I think that this is something that
definitely needs to be discussed.

Rectification isn't an issue in my opinion in this case, there is really no
reason why we would want to know the type of T in this case.

Still me:
I gave the null operator some more thought and I concluded that it makes no
sense for Optional.empty() to be null. Let's say that a method returns
potentially Optional.empty(), than if Optional.empty() == null wouldn't
calling, for example, Optional#orElse on that instance throw a Null pointer
exception logically? But this isn't what actually happens, so this is not
something that we can do. So by logic Optional.empty() cannot be == null,
but if it's not than a variable whose type is an instanceof optional and is
initialized with a null value is not actually equal to null. Practically:
String? value = null;
value == null; // false
value.isEmpty(); // true
This is appropriate, but not that intuitive considering the initializer.
Perhaps a better idea, keeping in mind this considerations, is to create a
new construct called Option instead of building upon Optional? Perhaps
someone has a better idea to solve this inconsistency

Mariell Hoversholm:
I don't find the concept of building around the `Optional` and hiding the
type to be a good idea. Ideally, the language would differentiate between
`T!` (non-null) and `T?` (nullable) explicitly in the type-system, just
like Kotlin does. I'm not quite sure what the best solution to this problem
would be, as I am absolutely in favour of having nullity as part of the
type system (Kotlin did it well), but it is clear that hiding types is not
the best way to go.

Me:
On second thought, maybe a distinction would be a better approach. The
module system idea is still applicable effectively and we don't need to
think about these complications with optional.



On Mon, May 16, 2022, 09:05 Mariell Hoversholm <mariell.hoversholm at paf.com>
wrote:

> How will assigning types work? Will this compile?:
>
>     String? value = Optional.empty();
>
>
> How would `instanceof` work with this? For example, is this legal?:
>
>     String? value = "value";
>     assert value instanceof String;
>     assert !(value instanceof Optional);
>
> If it is, it begs the question of how `instanceof` will truly work, given
> it is supposed to just check the raw type.
> If it is not, I wonder the implications of having the type be truly
> different from what the code reads.
>
>
> How does `null` equality tie into all this? Is this legal?:
>
>     String? value = null;
>     assert value == null;
>
> If it is, it would also imply `value != Optional.empty()`, which is not
> true from what I infer.
> If it is not, it would imply `value != null`, despite the clear
> initialiser setting it to `null`.
>
>
> If `T?` compiles down to `Optional<T>`, how will the `T` be known? We
> don't have reified generics, so it's not actually part of the type.
>
> On Sun, 15 May 2022 at 20:01, Alessandro Autiero <alautiero at gmail.com>
> wrote:
>
>> Null safe type system
>>
>> Hello, this is my first time writing a proposal for OpenJDK, but I hope
>> that it will meet the quality standards.
>> I've looked through the discussions in the OpenJDK mailing lists, but I
>> couldn't find any regarding this issue, so I thought I'd propose something
>> on my own.
>> I discussed this proposal previously on the java subreddit here
>> <
>> https://www.reddit.com/r/java/comments/tiw76i/backwards_compatible_null_safe_nonnull_by_default/
>> >to
>> collect ideas to make this proposal as comprehensive as possible.
>> I apologize in advance for any mistakes.
>>
>> **OVERVIEW**
>>
>> The Java Type System is made up of two different types: primitives and
>> objects.
>> While the first cannot be assigned to the special type null, the latter
>> can.
>> If an expression, for example, a method invocation, takes as an argument a
>> null reference a NullPointerException is thrown.
>> This design choice introduces the need to check for null values to make
>> sure that this doesn't happen.
>> The most common values that are checked are method parameters, especially
>> if the method is exposed publicly, and ones returned by method
>> invocations.
>> To make the latter easier, in Java 8 the java.util.Optional wrapper class
>> was introduced.
>> Being a class, Optional is not suitable as an equivalent to, for example,
>> Swift's Optional which is used often as a method parameter.
>> Despite this obvious design choice, some very popular libraries in the
>> Java
>> Ecosystem, most notably Selenium in my experience, have chosen to use
>> Optional in the context just described.
>> This indicates firmly that there is a need for better handling of null in
>> Java's type system.
>> As a matter of fact, annotations have become a sort of fix for this
>> issue(see the Checker Framework for example), though, in my opinion, this
>> issue should be resolved in the type system itself.
>> What I'd like to propose is to improve Java's type system by leveraging
>> the
>> Optional wrapper, the module system introduced in Java 9(Jigsaw) and the
>> recent developments over at Project Valhalla to better handle the special
>> type null.
>> Considering that our neighbour in the JVM universe, Kotlin, has been
>> designed with a type system that takes into account exactly this issue, I
>> think that it's appropriate to discuss JetBrain's approach.
>> Kotlin's type system, differently from Java's, is made up only of objects.
>> While non-null scalar types, such as Int, use JVM primitive types as an
>> implementation detail, they are not part of the language.
>> Object types can be nullable(the name of the type is followed by a
>> question
>> mark), non-null(the name of the type) or platform types(the name of the
>> type is followed by an exclamation mark) to provide a compatibility layer
>> with Java's type system.
>> In the latter case, the programmer should explicitly declare the type as
>> nullable or non-null(it's not technically needed as the compiler will
>> issue
>> a warning and not an error, but it's advised), while in the others can be
>> inferred.
>> In practice, this means that, for example, the type Int and Int? are two
>> completely different types for the compiler.
>> While this approach can be appropriate when designing a new language, like
>> in Kotlin's case, it certainly isn't feasible in Java's case considering
>> its twenty years of legacy.
>> Now let's look at Swift's approach instead. Swift's type system allows for
>> nullable and non-null object values and follows the same naming
>> conventions
>> as Kotlin, except platform types aren't a thing here.
>> Furthermore, nullable types aren't compiler magic in this case: they are
>> an
>> alias for the Optional enum.
>> For example, the type Int? and Optional<Int> are the same type.
>> This also means that declaring, for example, a variable with type Int?? is
>> legal, as this simply translates to Optional<Optional<Int>>.
>> This approach is better suited for Java as we already have a very similar
>> class that we can leverage: java.util.Optional.
>> Implementing the same concept in Java wouldn't have been previously
>> possible because the Optional wrapper, being a class, is too slow.
>> Thanks to Project Valhalla, though, the Optional wrapper can be
>> transformed
>> in a primitive class.
>> Non-null values assigned to Optional types should be autoboxed by using
>> Optional<T>#of(T), nullable values should be autoboxed by using
>> Optional<T>#ofNullable(T) while the null literal should be an alias of
>> Optional<T>#empty().
>> This can either be done with some compiler magic, like we currently do
>> with
>> primitive wrappers, or by introducing a full-fledged feature to support
>> boxing and unboxing for classes, but this conversation is not relevant
>> here
>> I think.
>> This means that the performance hit should no longer be a concern.
>> NOTE: An instance of a primitive class doesn't allow for null
>> initializers,
>> though, considering the previously described boxing and unboxing mechanism
>> this should be syntactically legal. If this approach introduced ambiguity,
>> though, a better approach would be to use a value class instead.
>>
>> **CHALLENGES**
>>
>> As previously mentioned, Java's legacy makes implementing such a feature
>> quite hard when considering backwards compatibility.
>> The first step is obviously to make Optional into a value class and this
>> is
>> possible without breaking anything, as far as I know.
>> Furthermore, it's also necessary to make sure that a developer using a
>> Java
>> version with this new hypothetical type system can, for example, use a
>> library built using a version of Java that doesn't.
>> This can be technically done by treating older types as equivalent to
>> platform types and allowing javac to place null checks(NULLCHK operator)
>> when needed.
>> While platform types as a concept aren't strictly necessary, I think that
>> having older types be nullable by default is not the right approach as if
>> the programmer wants to declare the variable as non-null requires implicit
>> conversion(from ExampleType? to ExampleType), which goes against the
>> purpose of this type system, or a manual cast which can be perceived as
>> very verbose and redundant.
>> This challenge was already faced by Google when they transitioned to Dart
>> 2.12 which introduced the concept of nullable and non-null types.
>> Finally, the only challenge would be to make migrating a project from an
>> older Java version as painless as possible.
>> For this new type system to work effectively, types should be non-null by
>> default.
>> Though, Java's current type system has object types nullable by default.
>> This means that if a developer were to transition to this new hypothetical
>> version, everything in the codebase would break.
>> To face this issue, I think that introducing a flag, for example,
>> sound(sound null safety), in the module-info, just like open, is the best
>> approach.
>> The default flag would be, for example, unsound, so that backwards
>> compatibility is preserved even if a module-info is already present.
>> Some may argue though that migrating a whole module to sound null safety
>> is
>> unlikely in major, especially enterprise, applications all at once.
>> This is the reason why we might consider having this flag also on a
>> package
>> level by using the package info.
>> An idea would be to have the flag in front of the package declaration,
>> though there are currently no flags available for packages in package
>> info,
>> having different type systems in a single module is not optimal for a
>> developer working on a codebase and there should also be a hierarchical
>> resolution of flags(for example a package is unsound but its parent module
>> is sound, what do we do?).
>> Apart from these implementation details, this approach ensures backwards
>> compatibility and an easy transition for developers to newer versions.
>>
>> **COMPLEMENTARY OPERATORS**
>>
>> To support such a type system, at least the null-safe de-referencing
>> operator(?.) would need to be introduced.
>> This should not break any existing application because the question mark
>> is
>> not a legal character for variables' names, so no ambiguities can occur.
>> Other useful operators would be the Elvis operator(?:) which is a binary
>> operator that returns the left operand if it's not null otherwise the
>> right
>> one and the double bang operator which is a unary operator that throws a
>> NullPointerException if its operand is null.
>> These aren't strictly needed, but they certainly improve the developer
>> experience.
>>
>> **COMPLEMENTARY KEYWORDS**
>>
>> Introducing an equivalent to the var keyword, introduced in Java 10, for
>> nullable types could be appropriate.
>> Let's say that the initializer of a variable is a literal, but the
>> developer later wants to assign a new value to this variable: how should
>> the compiler know if the type to infer is ExampleType or ExampleType?.
>> The var? keyword would only allow for nullable types, while the var
>> keyword
>> for both nullable and non-nullable.
>> This, though, should probably be discussed.
>>
>>
>> **EXAMPLES**
>>
>> Module Flags:
>>
>> module com.example.project { // The module has unsound null safety
>>
>> }
>>
>> sound module com.example.project { // The module has sound null safety
>>
>> }
>>
>> unsound module com.example.project { // Unsound is redundant, the module
>> has unsound null safety
>>
>> }
>>
>> Example interaction between legacy code and modern code:
>>
>> public void someMethod(){
>>    String? someValue = OldAPI.getSomeValue(); // The return type of
>> getSomeValue is specified by the programmer to be nullable, no checks
>> needed
>>    String someOtherValue = OldAPI.getSomeOtherValue(); // The return type
>> of getSomeValue is specified by the programmer to be non-null, the
>> compiler
>> should include a NULLCHK
>>    var inferredValue = OldAPI.getSomeValue(); // The return type of
>> getSomeValue is inferred to be nullable by var considering it's a platform
>> type
>> }
>>
>> **CONCLUSION**
>>
>> Discussing the technical changes needed for javac and for the JLS is
>> necessary, though this proposal will probably need a lot of discussion and
>> opinions before we get to this stage in my opinion.
>> The effort would probably be considerable though in the case of the
>> compiler.
>>
>
>
> --
>
> *Mariell Hoversholm *(she/her)
>
> Software Developer
>
> Integrations (Slack #integration-team-public)
>
>
> Paf
>
> Mobile: +46 73 329 40 18
>
> Bråddgatan 11 SE602 22, Norrköping
>
> Sweden
>
> *Working remote from Uppsala*
>
>
>
>
> This email is confidential and may contain legally privileged information.
> If you are not the intended recipient, please contact the sender and delete
> the email from your system without producing, distributing or retaining
> copies thereof. Thank you.
>