What we have lost ?

[forax at univ-mlv.fr]

----- Original Message -----
> From: "Paul Sandoz" <paul.sandoz at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "Brian Goetz" <brian.goetz at oracle.com>, "valhalla-spec-experts" <valhalla-spec-experts at openjdk.java.net>
> Sent: Thursday, September 8, 2022 1:10:26 AM
> Subject: Re: What we have lost ?

> The Vector API is I think a proof point that ref projection can be acceptable
> and can work.
> 
> The Vector API implementation currently behaves approximately as if a vector
> instance (whose concrete class is private) is a ref to a value object. The C2
> compiler performs aggressive unboxing and mapping "payloads" (analogous to
> value objects) vectors hold to vector registers, rather than scalarizing the
> element “overlay” or view over the payload described by the vector’s shape and
> element type (the species).
> 
> In this current model we generally don’t recommend storing vector instances in
> the heap (there are exceptions to that rule for constants), and instead vectors
> are loaded from or stored to memory containers such as arrays, byte buffers, or
> memory segments. After a few iterations of the API and implementation its
> starting to work rather well at generating efficient vectorized code on
> supporting hardware.
> 
> 
> We are currently early in the process of exploring alignment of the Vector API
> with Valhalla where:
> 
> 1. the private concrete vector classes are value classes whose .val type is
> private; and
> 2. a vector value class has a "payload" field whose type is a value type.
> 
> The "payload" is a non-atomic "bag of bits" e.g. values types of Bits256.val,
> Bits128.val etc, which would not be exposed in the public Vector API. So no
> utterance of .val is expected in this model. The API remains the same as it
> does today.
> 
> We could decide the value classes of those “bag of bits” could be public
> classes, and developers may interact with those using the .val. Unsure yet. As
> we explore and learn more the approach could change. I hope the exploration
> will be informative.


The problem of the current API is that it only works if everything is inlined by C2, otherwise performance is awful.
If a method is not inlined because its assembly code is too big, performance is awful, if a species is not a constant (some students just forget "final" when declaring the species as static final), performance is awful, if an operator (ADD, DIV, etc) is not a constant, performance is awful.

I'm pretty sure, i've already said that but i believe there is a better API design that can work even if a method is not inlined, because being sure that everything is inlined is a hard goal to attain once the code starts to grow.

For that a vector should be a generic value class parametrized by the vector species (so the species is always a constant) with the actual species been private, not visible from the language, an Int128Vector becomes an IntVector at compile time and an IntVector<Int128Species> at runtime.

In that world if a method is not inlined and a value class if a ref, you have nullchecks appearing and those will be hard to see apart taking a look to the assembly code.
If an IntVector has no ref companion, you eliminate such kind of performance hiccup.

> 
> Paul.

Rémi

> 
> 
> 
>> On Sep 7, 2022, at 2:27 PM, forax at univ-mlv.fr wrote:
>> 
>> 
>> 
>> From: "Brian Goetz" <brian.goetz at oracle.com>
>> To: "Remi Forax" <forax at univ-mlv.fr>, "valhalla-spec-experts"
>> <valhalla-spec-experts at openjdk.java.net>
>> Sent: Wednesday, September 7, 2022 8:38:46 PM
>> Subject: Re: What we have lost ?
>> I would summarize this mail as "no new observations, but growing discomfort with
>> ref-as-default."  All of these points amount to "there are classes for which
>> most uses will prefer val to varying degrees, but users may forget to say .val
>> / avoid saying .val because it is clunky", which may turn into performance
>> potholes.  These are surely known, so I will take this as Remi worrying that
>> this will be a bigger and more persistent irritant than we think.
>> 
>> If we take a step back, one of the main reason of introducing value types is to
>> have a way to be closer to the actual CPU architecture which can also be
>> described as solving Java performance potholes.
>> So i think we should shy away that term.
>> 
>> Dan help us to categorize the different issues,
>> - singleton types and vector representing SIMD registers are similar cases where
>> using the ref projection is far worst than the val projection, to the point
>> where author of those classes may prefer to not have a ref projection.
>>   In the case of an empty value class, storing the val projection use no memory,
>>   so the difference in term of the behavior is big,
>>   In the case of a SIMD register, if the VM loose track of the creation of
>>   something which is typed as the reference projection, a nullcheck will to be
>>   emitted, while usually we don't care about the cost of a nullcheck, having a
>>   supplementary nullcheck in a thigh loop is a far bigger issue to the point
>>   where having a reference projection for such types is counter productive.
>> 
>> - Atomic (or any class that play the role of a modifier like Stable, etc) is
>> when the ref projection have the wrong semantics so should not exist and there
>> is no need for a lightweight boxing given that you can always extract the value
>> from the monad (and use its reference projection if necessary).
>> 
>> - the last case is a case where the way the value classes are defined in
>> Scala/Kotlin can not be retrofitted to use the VM value classes because
>> getClass() always returns the class of the ref projection. This is also a case
>> where those value classes should not have a ref projection.
>> 
>> 
>> (I think the idea of "class for which there is no ref type" is a non-starter;
>> for all of the types you talk about (vectors, etc), you could make the same
>> argument for `int`, but no one is saying we don't need `Integer`.  I think what
>> you are really getting at here is coming back to some form of "I want
>> val-default".)
>> 
>> There are types which should never be null for different reasons, for those
>> types having a reference projection is an issue.
>> A nullable int is something useful hence we need Integer, sadly not all the
>> value types are like this.
>> 
>> 
>> A control question I would ask (though let's keep the bikeshedding to a dull
>> roar) is how much of this is about the undeniable clumsiness of the locution
>> "Point.val".  If, for example, the val type were called "point" or "Point!", as
>> some people have already publicly wished, does this change your concern that
>> "users will get it wrong all the time"?
>> 
>> I really believe that not all (value) types should have a reference projection.
>> 
>> Rémi
>> 
>> 
>> 
>> 
>> On 9/6/2022 4:32 AM, Remi Forax wrote:
>> Hi everybody,
>> it seems to me that the current design has reached a kind of local maximum,
>> which is nice and not nice at the same time,
>> so i would like to take the time to reflect on what we have sacrificed when
>> moving to the current design.
>> 
>> What is missing/not supported by the current model is value classes that should
>> not be used by reference,
>> either because it will cause performance issues or because the user will not get
>> the semantics he think he will get.
>> 
>> Here is a list of such value types:
>> - unit types, value types like by example Nothing (which mean that a method
>> never returns) with no fields.
>>   Because creating a ref on it creates something :)
>> 
>> - wrappers/monads that modify the semantics, by example a generic value class
>> Atomic that plays the same role as an AtomicReference, AtomicInteger, etc
>>   the problem here is that the default semantics is not the semantics the user
>>   want.
>>   
>> - SIMD vectors, if those are nullable, the VM/JIT will insert implicit null
>> checks which are not usually a problem apart in thigh loop like users write
>> with SIMD vectors.
>> 
>> - existing value classes in Scala or Kotlin, those are not nullable by default
>> but in the current design, getClass() will happily reflect them with a nullable
>> class making Scala/Kotlin second class citizens of the Java platform.
>> 
>> Those are my 4 tent poles, they are maybe others, but currently we fail to
>> provide a good answer for those cases.
>> 
>> regards,
>> Rémi
>> 
>> 
>> 
>> 
>>