Null-restricted types: Why so complicated?

[-]

Hi John,

As Quân said, you should only declare your array with ! if you know the
zero instance is safe. Otherwise, don't use the ! null-restriction
declaration, which will still fill the array with nulls and throw NPE on an
undefined array element. It is still possible technically for Valhalla to
inline any value type into an array, null-restricted or not, except
non-atomicity and implicit construction makes inlining easier (given the
hardware restraints), and correctness should always come before performance
in your designs.

On Fri, Jan 19, 2024 at 12:06 PM John Bossons <jbossons at gmail.com> wrote:

> Hi again,
>
> What you suggest is one solution. My counter: It's unsafe. The JVM should
> be able to protect a developer from invoking doSomethingWith(a[5]) on an
> undefined array element (or method parameter).
>
> On Fri, Jan 19, 2024 at 12:43 PM Quân Anh Mai <anhmdq at gmail.com> wrote:
>
>> Why not just
>>
>> Range[] a = new Range[100]; // allocate with null values
>> System.out.println(a[5]); // NullPointerException
>>
>> By using Range![] you accept that a zero value is acceptable, similar to
>> how an int[] works. If you do not want the uninitialized values to be
>> usable then do not use null-restricted type.
>>
>> Cheer,
>> Quan Anh
>>
>> On Sat, 20 Jan 2024 at 01:34, John Bossons <jbossons at gmail.com> wrote:
>>
>>> Hi Quan Anh,
>>>
>>> We're talking past each other.
>>>
>>> ME:  But in Java idiom that means that a developer can invoke the public
>>> implicit constructor, which will cause confusion.
>>> YOU:  An implicit constructor can be invoked like any other constructor,
>>> and it will return an all-zero instance of the corresponding class.
>>>
>>>
>>> Precisely. Which will often not be valid in the application context. It
>>> should be possible for a constructor to exclude an all-zero instance, such
>>> as a zero-length Range or an all-null Name (to use two examples in the
>>> draft spec) without giving up the ability to specify that it is
>>> null-restricted. Or for the 'real' constructor to be private, invoked from
>>> a factory method, which is effectively made useless as a protective feature
>>> if a public constructor is also provided. (If the implicit constructor
>>> could be specified as private, that would take care of the problem.
>>> Extending a marker interface is simpler.)
>>>
>>> ME:  > My further suggestion is that appending ! to a type should mean
>>> that the default initialized value of an instance (all fields zero) is
>>> equivalent to null, so that
>>> >       Range![] a = new Range![100];  // allocated with zero values
>>> >       System.out.println(a[5]);     // throws NullPointerException
>>> (zero fields)
>>> > This better conforms to current idiom, where the initial
>>> initialization is with nulls and the println invocation on a null array
>>> element or field throws a NPE.
>>> YOU:  What is the value of this proposal? If you want the all-zero
>>> instance to be equivalent to null, just do not have any constructor that
>>> initializes an instance to that state. The whole point of null-restricted
>>> fields/variables is to indicate that the field/variable is always valid.
>>>
>>>
>>> The issue here is not what the constructor does -- it hasn't been
>>> invoked yet for the element on which println is invoked -- but rather that
>>> the fact that the array element is undefined should be capable of being
>>> caught.
>>>
>>> On Fri, Jan 19, 2024 at 11:48 AM Quân Anh Mai <anhmdq at gmail.com> wrote:
>>>
>>>> I forgot to cc valhalla-dev
>>>>
>>>> ---------- Forwarded message ---------
>>>> From: Quân Anh Mai <anhmdq at gmail.com>
>>>> Date: Sat, 20 Jan 2024 at 00:33
>>>> Subject: Re: Null-restricted types: Why so complicated?
>>>> To: John Bossons <jbossons at gmail.com>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> > But in Java idiom that means that a developer can invoke the public
>>>> implicit constructor, which will cause confusion.
>>>>
>>>> An implicit constructor can be invoked like any other constructor, and
>>>> it will return an all-zero instance of the corresponding class.
>>>>
>>>> > My further suggestion is that appending ! to a type should mean that
>>>> the default initialized value of an instance (all fields zero) is
>>>> equivalent to null, so that
>>>> >       Range![] a = new Range![100];  // allocated with zero values
>>>> >       System.out.println(a[5]);      // throws NullPointerException
>>>> (zero fields)
>>>> > This better conforms to current idiom, where the initial
>>>> initialization is with nulls and the println invocation on a null array
>>>> element or field throws a NPE.
>>>>
>>>> What is the value of this proposal? If you want the all-zero instance
>>>> to be equivalent to null, just do not have any constructor that initializes
>>>> an instance to that state. The whole point of null-restricted
>>>> fields/variables is to indicate that the field/variable is always valid.
>>>>
>>>> I think you are having some confusion. Null-restriction is the property
>>>> of a variable/field, i.e. the property of the holder, not of the class
>>>> itself. The class having implicit constructors simply means that it allows
>>>> the existence of null-restricted fields/variables. The class can be used as
>>>> normal with non-null-restricted types. (e.g Range r = null;)
>>>>
>>>> Regards,
>>>> Quan Anh
>>>>
>>>> On Sat, 20 Jan 2024 at 00:09, John Bossons <jbossons at gmail.com> wrote:
>>>>
>>>>> Thanks for your comments. I was not sufficiently explicit.
>>>>>
>>>>> Let me focus on implicit. I guess my dislike is of introducing a
>>>>> 'fake' constructor into the definition of a class. I say 'fake' because, as
>>>>> I understand it, the only purpose of the implicit constructor is to
>>>>> indicate to the JVM/compiler that a never-null instance can be created. But
>>>>> in Java idiom that means that a developer can invoke the public implicit
>>>>> constructor, which will cause confusion.
>>>>>
>>>>> Maybe it would be better to require a potentially null-restricted
>>>>> class to extend a marker interface ('extends NeverNullPossible'? Or maybe,
>>>>> looking ahead to my next comment, 'extends AllZerosIsNull'?). That would
>>>>> enable the compiler to catch an invalid use of the ! marker in a
>>>>> declaration, just as the proposed implicit constructor does, while
>>>>> conforming better to common Java idiom.
>>>>>
>>>>> My further suggestion is that appending ! to a type should mean that
>>>>> the default initialized value of an instance (all fields zero) is
>>>>> equivalent to null, so that
>>>>>       Range![] a = new Range![100];  // allocated with zero values
>>>>>       System.out.println(a[5]);      // throws NullPointerException
>>>>> (zero fields)
>>>>> This better conforms to current idiom, where the initial
>>>>> initialization is with nulls and the println invocation on a null array
>>>>> element or field throws a NPE.
>>>>>
>>>>> As you say, my suggestion means runtime testing to determine if all
>>>>> fields are zero, which has a performance cost. This will only occur if the
>>>>> JVM implements the ! specification, which it presumably will only do if the
>>>>> object is small. And the cost will be small (I am presuming) relative to
>>>>> savings from allowing the memory footprint to match that of primitives. Am
>>>>> I wrong? There is value in conforming to current idiom.
>>>>>
>>>>> Turning to the LooselyConsistentValue, I withdraw my comments. I
>>>>> mistakenly presumed that its use would be required, which is false. It
>>>>> simply enables a single-threaded (or volatile-protected) application to
>>>>> allow additional inlining, which is harmless.
>>>>>
>>>>> John
>>>>>
>>>>> On Thu, Jan 18, 2024 at 4:56 PM - <liangchenblue at gmail.com> wrote:
>>>>>
>>>>>> Hi John,
>>>>>>
>>>>>> On Thu, Jan 18, 2024 at 2:30 PM John Bossons <jbossons at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Maybe I am missing something, but the proposal seems to be trying to
>>>>>>> do too much.
>>>>>>>
>>>>>>> Specifically: Why not simply provide that appending ! to a type
>>>>>>> specification for an object (field, array element, or parameter) means that
>>>>>>> that the object is not only null-restricted but also never zero and
>>>>>>> necessarily non-atomic unless small?
>>>>>>>
>>>>>> First, a reminder that some objects cannot be non-atomic, mostly when
>>>>>> fields have dependencies/constraints on each other: if you have a range,
>>>>>> you cannot allow its lower bound to be larger than its upper bound.
>>>>>> Non-atomic representations cannot avoid this pitfall. Also you seem
>>>>>> to misunderstand non-atomic: if an object is non-atomic, each of its fields
>>>>>> can update independently from each other, so a 3-d position can be
>>>>>> non-atomic, but not so for a range. Non-atomicity is dangerous, and it
>>>>>> should not be the default. However, if an atomic class is small enough,
>>>>>> like OptionalInt (as now many architecture has like atomic handling of 16
>>>>>> bytes etc.) JVM may choose to apply non-atomic optimizations to them for
>>>>>> better performance without violating their object constraints.
>>>>>>
>>>>>>>
>>>>>>> Why complicate the specification with an implicit constructor that a
>>>>>>> developer will never explicitly invoke? Why permit a developer to 'opt in'
>>>>>>> to non-atomic?
>>>>>>>
>>>>>> The implicit constructor can always be called; its existence asks
>>>>>> programmers to affirm that the zero-filled inlined instance is a valid
>>>>>> instance. And this instance is different from a null, as null is a pointer,
>>>>>> yet the zero-instance has a different size defined by the class layout in
>>>>>> the stack/heap.
>>>>>>
>>>>>>>
>>>>>>> Sure, that means trying to read a zero value triggers a NPE. That
>>>>>>> just means that a type that can legitimately have a zero value cannot be
>>>>>>> specified as null-restricted, since a zero value (e.g. a {null, null} Name)
>>>>>>> is the equivalent of a null unrestricted value object. Why go beyond that?
>>>>>>> If a non-null zero value is possible, the type cannot be null-restricted
>>>>>>> and so can only be an unrestricted JEP 401 value type. End of story.
>>>>>>>
>>>>>> You see the inlined zero instance and the null pointer have different
>>>>>> sizes, and thus they are not exchangeable. Converting the inlined zero
>>>>>> instance to null to throw NPE is complex and hurtful to performance as you
>>>>>> will scan unrelated bits for almost every field access.
>>>>>>
>>>>>> And for unrestricted value type, yes, they exist and can possibly be
>>>>>> inlined as well if the restricted type is small enough (i.e. has space for
>>>>>> extra bit indicating nullity) But reminder, the nullity bit itself isn't
>>>>>> even non-atomic with (depends on) the rest of the object! You don't want
>>>>>> the nullity to indicate null while the rest of the object indicate some
>>>>>> sort of non-null value, which can happen in a non-atomic context.
>>>>>>
>>>>>>>
>>>>>>> With respect to non-atomic, what is new? Yes, unexpected  instances
>>>>>>> may occur without synchronization if the object is larger than the word
>>>>>>> size of the implementation. Why do we need to extend a
>>>>>>> LooselyConsistentValue interface to know/permit that?
>>>>>>>
>>>>>> Unexpected instances don't occur without synchronization if you use
>>>>>> finals, such as in Java's String or immutable List.of(). These APIs may
>>>>>> capture any "permitted value" from the arrays passed in, but once
>>>>>> constructed, the captured value remains constant no matter which thread
>>>>>> observes the String/List object reference. (Technically, JVM implements
>>>>>> this with a store-store fence between end of field writes in the
>>>>>> constructor and object reference is shared anywhere, and a load-load fence
>>>>>> between object reference read and field read) Value classes is about the
>>>>>> safety of final fields in programming instead of the close encounter of
>>>>>> third kinds of synchronization, volatiles, and fences.
>>>>>>
>>>>>>>
>>>>>>> Can we not keep this 'simple' (if that word has meaning in this
>>>>>>> context)? What am I missing?
>>>>>>>
>>>>>> I think you are missing a bit about how the layout (inlining is
>>>>>> represented in memory) and value classes (the thread safety its final
>>>>>> offers) work, and what "non-atomic" means. Feel free to question more.
>>>>>>
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Phone:  (416) 450-3584 (cell)
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Phone:  (416) 450-3584 (cell)
>>>>>
>>>>
>>>
>>> --
>>> Phone:  (416) 450-3584 (cell)
>>>
>>
>
> --
> Phone:  (416) 450-3584 (cell)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/valhalla-dev/attachments/20240119/2652cd02/attachment.htm>