From daniel.smith at oracle.com  Mon May  3 21:58:42 2021
From: daniel.smith at oracle.com (Dan Smith)
Date: Mon, 3 May 2021 21:58:42 +0000
Subject: Classfile artifacts to support compilation/reflection
In-Reply-To: <07f9880a-758c-689a-dc07-5b7decc33df5@oracle.com>
References: <07f9880a-758c-689a-dc07-5b7decc33df5@oracle.com>
Message-ID: <E133DAC5-2738-4982-8756-E7F63A50E64D@oracle.com>



> On Apr 28, 2021, at 2:12 PM, Brian Goetz <brian.goetz at oracle.com> wrote:
> 
> I'm updating the SoV documents and it raises a few questions about what classfile surface we need for capturing the language model.  The good news is that with the single-classfile model, the translation complexity collapses almost to zero.  But there are a few questions of "what do we retain in the classfile."  
> 
> 1.  Ref-favoring vs val-favoring.  Whether a primitive class P is ref-favoring or val-favoring no longer affects translation of the classfile (yay), it only affects translation of _type uses_ of the unadorned name.  But, this has to be capture somewhere in the classfile, so that the compiler can read in P.class and know what the name `P` means.  There are a few choices here:
> 
>  - An ACC_ bit.  Meh, these are pretty expensive.  
>  - An attribute, which only javac and reflection would need to pay attention to.
>  - A supertype (implements RefFavoring).  
> 
> My preference is an attribute; this feels closest to `Signature` to me.  Reflection might want to reflect the ref-favoring bit.

Agree. Note that this is a Java-language-only attribute, irrelevant to the JVM.

> 2.  Whether abstract classes are primitive superclass candidates.  The static compiler will check this at compilation time when it sees a superclass of a primitive class, but the JVM will want to recheck anyway.  There are two sensible ways to handle this in the classfile:
> 
>  - An attribute that says "I am a primitive superclass candidate."  The static compiler puts it there, and the JVM         checks it.  
>  - Infer and tag.  If an abstract class is loaded that is not a primitive superclass candidate, the JVM injects IdentityObject as a superinterface of the newly loaded class; when we go to load a primitive subclass, this will fail because primitive classes cannot implement both IdentityObject and PrimitiveObject.  
> 
> Reflection probably doesn't have to reflect whether a class is primitive superclass candidate; it already reflects the things needed to make this determination.  

This one, on the other hand, conveys a core property of a JVM class.

To review the story for the JVM, it goes like this: there are two channels for subclass instance creation. The first channel, for identity subclasses, is the mutation-based '<init>' route, with subclasses required by the verifier to call up the '<init>' chain (and supply appropriate arguments) to get usable class instances. '<init>' declarations can limit access. The second channel, for primitive subclasses, is an opt-in flag in the parent that says "I'll allow primitive children without performing any initialization", and then 'defaultvalue' and 'withfield' can just ignore the superclass. This route also needs accessibility restrictions.

Here's how the *language behavior* of abstract classes is defined in JEP 401:

"An interface can explicitly extend either IdentityObject or PrimitiveObject if the author determines that all implementing objects are expected to have or not have identity. It is an error if a class ends up implementing both interfaces implicitly, explicitly, or by inheritance. By default, an interface extends neither of these interfaces and can be implemented by both kinds of concrete classes."

"An abstract class can similarly be declared to implement either IdentityObject or PrimitiveObject; or, if it declares a field, an instance initializer, a non-empty constructor, or a synchronized method, it implicitly implements IdentityObject. Otherwise, it extends neither interface and can be extended by both kinds of concrete classes."

We decided that detecting a "non-empty constructor" is not a job the JVM should be expected to perform, and so the JVM needs an explicit signal for the second-channel flavor of instance creation. (And note, BTW, that the first channel *also* has an explicit opt-in in the JVM, even though there's an implicit constructor in the language. You can declare a JVM class without instance creation support.) javac is responsible for generating that opt-in signal; legacy classes don't get it until recompilation.

Given that signal, the JVM can do some error checks (again quoting JEP 401):

"An abstract class that allows primitive subclasses declares this capability in its classfile (details TBD). At class load time, an error occurs if the class is not abstract, declares an instance field, declares a synchronized method, or implements?directly or indirectly?IdentityObject."

Concretely, the natural fit for encoding is a class attribute ('PrimitiveInstantiation', say) that carries an access flag.

> 3.  T.ref.  In generic code, we can say `T.ref`, which is a         total operator on types; if T is already a reference type, then T.ref = T, and if it is a primitive value type P.val., then T.ref = P.ref.  The Signature attribute should be extended to support the distinction between a use of `T` and a use of `T.ref`.  (T.val is partial, so doesn't make sense in the general case, and in the specific cases where it does make sense, does not currently look worth supporting.)

+1

> 4.  Other flavors, as needed.  We've considered a "null-default" primitive class; if so, this has to be captured in a similar way as (1).  These can probably all be folded into a single PrimitiveClass attribute.  

Currently, the JEP proposes bundling up all default/integrity-related constraints under a single flag:

"Tentative feature: If it is important for correctness, a primitive class may declare that instances must be validated through a constructor call. In this case, the compiler and JVM will ensure that backdoor instance creation is either prevented or detected before any instance methods of the class are executed."

This would affect JVM behavior (e.g., atomicity guarantees). It's binary, so could make sense as an ACC_* flag. Or it could go in an attribute. Or a marker interface.


From brian.goetz at oracle.com  Mon May  3 22:21:41 2021
From: brian.goetz at oracle.com (Brian Goetz)
Date: Mon, 3 May 2021 18:21:41 -0400
Subject: Classfile artifacts to support compilation/reflection
In-Reply-To: <E133DAC5-2738-4982-8756-E7F63A50E64D@oracle.com>
References: <07f9880a-758c-689a-dc07-5b7decc33df5@oracle.com>
 <E133DAC5-2738-4982-8756-E7F63A50E64D@oracle.com>
Message-ID: <2f79b3c3-b9bf-7052-096e-0eae53aaadca@oracle.com>


>> 2.  Whether abstract classes are primitive superclass candidates.  The static compiler will check this at compilation time when it sees a superclass of a primitive class, but the JVM will want to recheck anyway.  There are two sensible ways to handle this in the classfile:
>>
>>   - An attribute that says "I am a primitive superclass candidate."  The static compiler puts it there, and the JVM         checks it.
>>   - Infer and tag.  If an abstract class is loaded that is not a primitive superclass candidate, the JVM injects IdentityObject as a superinterface of the newly loaded class; when we go to load a primitive subclass, this will fail because primitive classes cannot implement both IdentityObject and PrimitiveObject.
>>
>> Reflection probably doesn't have to reflect whether a class is primitive superclass candidate; it already reflects the things needed to make this determination.
> This one, on the other hand, conveys a core property of a JVM class.

John's notes in the SotV suggests that the JVM is comfortable just 
"figuring it out" and not requiring an attribute.? So this is the "infer 
and tag" option; the VM infers this at runtime.? Not clear if there is a 
value to having the static compiler capture something that wasn't 
explicit in the source and that has to be validated at runtime anyway.



From daniel.smith at oracle.com  Mon May  3 22:29:32 2021
From: daniel.smith at oracle.com (Dan Smith)
Date: Mon, 3 May 2021 22:29:32 +0000
Subject: Classfile artifacts to support compilation/reflection
In-Reply-To: <E133DAC5-2738-4982-8756-E7F63A50E64D@oracle.com>
References: <07f9880a-758c-689a-dc07-5b7decc33df5@oracle.com>
 <E133DAC5-2738-4982-8756-E7F63A50E64D@oracle.com>
Message-ID: <B9937A1A-AEBD-4807-9B61-10E4BD023640@oracle.com>


> On May 3, 2021, at 3:58 PM, Dan Smith <daniel.smith at oracle.com> wrote:
> 
> To review the story for the JVM, it goes like this: there are two channels for subclass instance creation. The first channel, for identity subclasses, is the mutation-based '<init>' route, with subclasses required by the verifier to call up the '<init>' chain (and supply appropriate arguments) to get usable class instances. '<init>' declarations can limit access. The second channel, for primitive subclasses, is an opt-in flag in the parent that says "I'll allow primitive children without performing any initialization", and then 'defaultvalue' and 'withfield' can just ignore the superclass. This route also needs accessibility restrictions.
> 
> Here's how the *language behavior* of abstract classes is defined in JEP 401:
> 
> "An interface can explicitly extend either IdentityObject or PrimitiveObject if the author determines that all implementing objects are expected to have or not have identity. It is an error if a class ends up implementing both interfaces implicitly, explicitly, or by inheritance. By default, an interface extends neither of these interfaces and can be implemented by both kinds of concrete classes."
> 
> "An abstract class can similarly be declared to implement either IdentityObject or PrimitiveObject; or, if it declares a field, an instance initializer, a non-empty constructor, or a synchronized method, it implicitly implements IdentityObject. Otherwise, it extends neither interface and can be extended by both kinds of concrete classes."
> 
> We decided that detecting a "non-empty constructor" is not a job the JVM should be expected to perform, and so the JVM needs an explicit signal for the second-channel flavor of instance creation. (And note, BTW, that the first channel *also* has an explicit opt-in in the JVM, even though there's an implicit constructor in the language. You can declare a JVM class without instance creation support.) javac is responsible for generating that opt-in signal; legacy classes don't get it until recompilation.
> 
> Given that signal, the JVM can do some error checks (again quoting JEP 401):
> 
> "An abstract class that allows primitive subclasses declares this capability in its classfile (details TBD). At class load time, an error occurs if the class is not abstract, declares an instance field, declares a synchronized method, or implements?directly or indirectly?IdentityObject."
> 
> Concretely, the natural fit for encoding is a class attribute ('PrimitiveInstantiation', say) that carries an access flag.

BTW, I'd suggest updating the State of Valhalla, Section 3 document to better reflect the above details. Here's what it currently says:

>> An abstract class is a primitive superclass candidate if it has no fields, its no-arg constructor is empty (marked ACC_ABSTRACT), it has no other <init> methods, and its superclass is either Object or a primitive superclass candidate.

We should validate no fields, etc., but "and" conjunctions in definitions are dangerous. Let's not fall into the trap of "it's a legal class, it just doesn't mean what it appears to mean." There should be one clear opt-in signal, and then anything inconsistent with that is an error.

I'm pretty down on the ACC_ABSTRACT route for expressing the opt-in, because it intertwines the identity and primitive instance creation channels. There's no particular reason for the JVM to prevent, say, a primitive-friendly class from including some logging code in its identity channel. (Whether a *language* wants to adopt the associated complexity is a separate decision.)

>> (The static factory of the primitive class does not call the superclass constructor. This can be safe only if the constructor is empty anyway.)


More fundamentally at the JVM level, the 'defaultvalue' and 'withfield' instructions do not call any superclass instance initialization logic. Whether identity-only logic in <init> methods is "safe" is a problem for the class's author/language to work out.

>> The properties of being a primitive superclass candidate are "contagious" up the superclass chain. That is, if some abstract class has an empty no-arg constructor, its immediate superclass must also have an empty no-arg constructor.

IdentityObject is the mechanism for this check. Per the JEP text quoted above, if you've got the opt-in flag, you're not allowed to implement IdentityObject. (And if you don't, you implicitly implement IdentityObject.)

>> It would seem to follow that even Objectmust have a empty (ACC_ABSTRACT) constructor, but Object is special, because it alone (of all classes) is allowed to have a truly empty concrete constructor method. Thus, Object is exempt from the "contagion" of the requirement of empty constructors.

Object can, and should, have the primitive subclasses opt-in. And, of course, it doesn't implement IdentityObject.


From daniel.smith at oracle.com  Mon May  3 22:32:45 2021
From: daniel.smith at oracle.com (Dan Smith)
Date: Mon, 3 May 2021 22:32:45 +0000
Subject: Classfile artifacts to support compilation/reflection
In-Reply-To: <2f79b3c3-b9bf-7052-096e-0eae53aaadca@oracle.com>
References: <07f9880a-758c-689a-dc07-5b7decc33df5@oracle.com>
 <E133DAC5-2738-4982-8756-E7F63A50E64D@oracle.com>
 <2f79b3c3-b9bf-7052-096e-0eae53aaadca@oracle.com>
Message-ID: <2D35A34F-4FD2-4CEE-A4BD-9475E99027FF@oracle.com>


> On May 3, 2021, at 4:21 PM, Brian Goetz <brian.goetz at oracle.com> wrote:
> 
> 
>>> 2.  Whether abstract classes are primitive superclass candidates.  The static compiler will check this at compilation time when it sees a superclass of a primitive class, but the JVM will want to recheck anyway.  There are two sensible ways to handle this in the classfile:
>>> 
>>>  - An attribute that says "I am a primitive superclass candidate."  The static compiler puts it there, and the JVM         checks it.
>>>  - Infer and tag.  If an abstract class is loaded that is not a primitive superclass candidate, the JVM injects IdentityObject as a superinterface of the newly loaded class; when we go to load a primitive subclass, this will fail because primitive classes cannot implement both IdentityObject and PrimitiveObject.
>>> 
>>> Reflection probably doesn't have to reflect whether a class is primitive superclass candidate; it already reflects the things needed to make this determination.
>> This one, on the other hand, conveys a core property of a JVM class.
> 
> John's notes in the SotV suggests that the JVM is comfortable just "figuring it out" and not requiring an attribute.  So this is the "infer and tag" option; the VM infers this at runtime.  Not clear if there is a value to having the static compiler capture something that wasn't explicit in the source and that has to be validated at runtime anyway.

Ha, I was just looking over this! (See email.)

SotV still has an opt-in. It just describes it as a ACC_ABSTRACT flag on an <init> method, rather than a class attribute or something else. (It also describes some additional requirements, like no instance fields, but I argued in the other email that those requirements are better handled as consistency checks, not separate components to the opt-in.)

From daniel.smith at oracle.com  Wed May  5 14:23:08 2021
From: daniel.smith at oracle.com (Dan Smith)
Date: Wed, 5 May 2021 14:23:08 +0000
Subject: EG meeting, 2021-05-05
Message-ID: <75212109-E4D7-44D0-82F0-17002156CA1F@oracle.com>

The next EG Zoom meeting is today at 4pm UTC (9am PDT, 12pm EDT).

Topics to discuss:

"consolidated VM notes for primitive classes": Updated section of State of Valhalla addressing JVM changes

"Classfile artifacts to support compilation/reflection": Brian listed properties of primitive classes and value types that need to be encoded in class files


From forax at univ-mlv.fr  Wed May  5 14:39:01 2021
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 5 May 2021 16:39:01 +0200 (CEST)
Subject: Meeting today: IdentityClass
Message-ID: <1847895792.1903234.1620225541253.JavaMail.zimbra@u-pem.fr>

If it's possible, i would like to discuss about IdentityClass again.

As noted in the last draft, adding IdentityClass automatically at runtime is not a compatible change if tests that uses Class::getInterfaces() are not written correctly,
and sadly, a lot of tests are written that way. I'm guilty to having written those tests too.
I don't believe that the solution is to tweak the reflection because adding IdentityClass at runtime is already a hack and introducing a hack? (hack square hack) is usually where we should stop before anyone sanity goes under the bus.

The purpose of IdentityClass is
- to have a classfile for the javadoc describing the behavior of all identity class
- to be used as type or bound of type parameter.

Apart the result of Class::getInterfaces() being hardcoded, IdentityClass has to other issues, as javadoc container, given that IdentityClass is inserted by the VM, it means there is no place in the Java code where someone can click to see the javadoc in an IDE, we have the same issue with java.lang.Record currently, the class is added by javac automatically and unlike java.lang.Enum, there is no method useful on java.lang.Record (equals/hashCode and toString are defined directly on the record) so very few of my student where able to understand why a record with a NaN value was equals to itself.
But there is a more serious issue, using IdentityClass is not backward compatible with Object.
When we have introduced IdentityClass, one scenario was to be able to declare that the type parameter corresponding to the keys (K) to only support identity class.
This is not possible using IdentityClass because the erasure of K will be IdentityClass instead of Object (IdentityClass also appears when the compiler will to a lub, so the common class of String and URI will be computed as IdentityClass instead of Object leading to source compatibility issues).

I think at that point, we should go back to our blackboard and see if there is no other solution.

I see two, one is to do *nothing* and do not add a type saying that only identity class is a corner case after all,
the other is to piggyback on erasure not unlike Scala does, i.e. IdentityClass is a fake class that is erased to Object at runtime.

regards,
R?mi


From forax at univ-mlv.fr  Wed May  5 14:40:14 2021
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 5 May 2021 16:40:14 +0200 (CEST)
Subject: EG meeting, 2021-05-05
In-Reply-To: <75212109-E4D7-44D0-82F0-17002156CA1F@oracle.com>
References: <75212109-E4D7-44D0-82F0-17002156CA1F@oracle.com>
Message-ID: <2045198336.1903722.1620225614521.JavaMail.zimbra@u-pem.fr>

----- Mail original -----
> De: "daniel smith" <daniel.smith at oracle.com>
> ?: "valhalla-spec-experts" <valhalla-spec-experts at openjdk.java.net>
> Envoy?: Mercredi 5 Mai 2021 16:23:08
> Objet: EG meeting, 2021-05-05

> The next EG Zoom meeting is today at 4pm UTC (9am PDT, 12pm EDT).
> 
> Topics to discuss:
> 
> "consolidated VM notes for primitive classes": Updated section of State of
> Valhalla addressing JVM changes
> 
> "Classfile artifacts to support compilation/reflection": Brian listed properties
> of primitive classes and value types that need to be encoded in class files

+ IdentityClass

R?mi


From heidinga at redhat.com  Wed May  5 16:05:13 2021
From: heidinga at redhat.com (Dan Heidinga)
Date: Wed, 5 May 2021 12:05:13 -0400
Subject: Classfile artifacts to support compilation/reflection
In-Reply-To: <2D35A34F-4FD2-4CEE-A4BD-9475E99027FF@oracle.com>
References: <07f9880a-758c-689a-dc07-5b7decc33df5@oracle.com>
 <E133DAC5-2738-4982-8756-E7F63A50E64D@oracle.com>
 <2f79b3c3-b9bf-7052-096e-0eae53aaadca@oracle.com>
 <2D35A34F-4FD2-4CEE-A4BD-9475E99027FF@oracle.com>
Message-ID: <CAJq4Gi7MDGr6XLJrPD34tuFu3WaSv1qvj=BR9tbgPJdMd+Ax4w@mail.gmail.com>

I've added some comments to the document and thought I'd try something
different.  Rather than copy/paste context here, I've opened a github
pull request against the repo so the context of the changes / comments
are directly inline.
https://github.com/openjdk/valhalla-docs/pull/1

Trying this but if there's a preference to have it be on list, I can
recreate the comments here.

--Dan

On Mon, May 3, 2021 at 6:33 PM Dan Smith <daniel.smith at oracle.com> wrote:
>
>
> > On May 3, 2021, at 4:21 PM, Brian Goetz <brian.goetz at oracle.com> wrote:
> >
> >
> >>> 2.  Whether abstract classes are primitive superclass candidates.  The static compiler will check this at compilation time when it sees a superclass of a primitive class, but the JVM will want to recheck anyway.  There are two sensible ways to handle this in the classfile:
> >>>
> >>>  - An attribute that says "I am a primitive superclass candidate."  The static compiler puts it there, and the JVM         checks it.
> >>>  - Infer and tag.  If an abstract class is loaded that is not a primitive superclass candidate, the JVM injects IdentityObject as a superinterface of the newly loaded class; when we go to load a primitive subclass, this will fail because primitive classes cannot implement both IdentityObject and PrimitiveObject.
> >>>
> >>> Reflection probably doesn't have to reflect whether a class is primitive superclass candidate; it already reflects the things needed to make this determination.
> >> This one, on the other hand, conveys a core property of a JVM class.
> >
> > John's notes in the SotV suggests that the JVM is comfortable just "figuring it out" and not requiring an attribute.  So this is the "infer and tag" option; the VM infers this at runtime.  Not clear if there is a value to having the static compiler capture something that wasn't explicit in the source and that has to be validated at runtime anyway.
>
> Ha, I was just looking over this! (See email.)
>
> SotV still has an opt-in. It just describes it as a ACC_ABSTRACT flag on an <init> method, rather than a class attribute or something else. (It also describes some additional requirements, like no instance fields, but I argued in the other email that those requirements are better handled as consistency checks, not separate components to the opt-in.)
>


From forax at univ-mlv.fr  Wed May  5 17:50:25 2021
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 5 May 2021 19:50:25 +0200 (CEST)
Subject: Meeting today: IdentityObject
In-Reply-To: <1847895792.1903234.1620225541253.JavaMail.zimbra@u-pem.fr>
References: <1847895792.1903234.1620225541253.JavaMail.zimbra@u-pem.fr>
Message-ID: <1843312453.1985533.1620237025001.JavaMail.zimbra@u-pem.fr>

There is a 3rd way, uses the TypeRestriction attribute to still be binary backward compatible.

Let suppose that we have a way to say that javac needs to use TypeRestriction when generating a specific type in the class file
like
  import ...

  __restrict__ java.lang.IdentityObject __to__ java.lang.Object;

R?mi

----- Mail original -----
> De: "Remi Forax" <forax at univ-mlv.fr>
> ?: "valhalla-spec-experts" <valhalla-spec-experts at openjdk.java.net>
> Envoy?: Mercredi 5 Mai 2021 16:39:01
> Objet: Meeting today: IdentityClass

> If it's possible, i would like to discuss about IdentityClass again.
> 
> As noted in the last draft, adding IdentityClass automatically at runtime is not
> a compatible change if tests that uses Class::getInterfaces() are not written
> correctly,
> and sadly, a lot of tests are written that way. I'm guilty to having written
> those tests too.
> I don't believe that the solution is to tweak the reflection because adding
> IdentityClass at runtime is already a hack and introducing a hack? (hack square
> hack) is usually where we should stop before anyone sanity goes under the bus.
> 
> The purpose of IdentityClass is
> - to have a classfile for the javadoc describing the behavior of all identity
> class
> - to be used as type or bound of type parameter.
> 
> Apart the result of Class::getInterfaces() being hardcoded, IdentityClass has to
> other issues, as javadoc container, given that IdentityClass is inserted by the
> VM, it means there is no place in the Java code where someone can click to see
> the javadoc in an IDE, we have the same issue with java.lang.Record currently,
> the class is added by javac automatically and unlike java.lang.Enum, there is
> no method useful on java.lang.Record (equals/hashCode and toString are defined
> directly on the record) so very few of my student where able to understand why
> a record with a NaN value was equals to itself.
> But there is a more serious issue, using IdentityClass is not backward
> compatible with Object.
> When we have introduced IdentityClass, one scenario was to be able to declare
> that the type parameter corresponding to the keys (K) to only support identity
> class.
> This is not possible using IdentityClass because the erasure of K will be
> IdentityClass instead of Object (IdentityClass also appears when the compiler
> will to a lub, so the common class of String and URI will be computed as
> IdentityClass instead of Object leading to source compatibility issues).
> 
> I think at that point, we should go back to our blackboard and see if there is
> no other solution.
> 
> I see two, one is to do *nothing* and do not add a type saying that only
> identity class is a corner case after all,
> the other is to piggyback on erasure not unlike Scala does, i.e. IdentityClass
> is a fake class that is erased to Object at runtime.
> 
> regards,
> R?mi

From brian.goetz at oracle.com  Wed May  5 18:20:07 2021
From: brian.goetz at oracle.com (Brian Goetz)
Date: Wed, 5 May 2021 14:20:07 -0400
Subject: Meeting today: IdentityClass
In-Reply-To: <1847895792.1903234.1620225541253.JavaMail.zimbra@u-pem.fr>
References: <1847895792.1903234.1620225541253.JavaMail.zimbra@u-pem.fr>
Message-ID: <f2638ecd-9e62-239f-d540-435ec8bf2fe8@oracle.com>

Here are some idioms I can imagine a use for with IdentityObject:

 ??? // parameter type
 ??? void withLock(IdentityObject monitor, Runnable task)

 ??? // type bound
 ??? Map<K extends IdentityObject, V> map = ...? // gonna lock on keys

 ??? // dynamic check
 ??? if (x instanceof IdentityObject) {
 ??????? sync (x) { task(); }
 ??? }
 ??? else {
 ??????? sync (GLOBAL_LOCK) { task(); }
 ??? }

 ??? // reflective check against class
 ??? Class<?> c = ...
 ??? if (IdentityObject.class.isAssignableFrom(c)) { ... }

The "fake type" approach handles the first two cases well enough.? But 
it doesn't address the latter two cases; for that we'd need to expose a 
Class::isIdentityClass (x.getClass().isIdentityClass()), which seems 
workable, though might feel gratuitously different from the first two.

Erasing to Object means that we don't get to do things like overloading:

 ??? m(IdentityObject o) { ... }
 ??? m(PrimitiveObject o) { ... }

or

 ??? m(IdentityObject o) { ... }
 ??? m(Object o) { ... }




On 5/5/2021 10:39 AM, Remi Forax wrote:
> If it's possible, i would like to discuss about IdentityClass again.
>
> As noted in the last draft, adding IdentityClass automatically at runtime is not a compatible change if tests that uses Class::getInterfaces() are not written correctly,
> and sadly, a lot of tests are written that way. I'm guilty to having written those tests too.
> I don't believe that the solution is to tweak the reflection because adding IdentityClass at runtime is already a hack and introducing a hack? (hack square hack) is usually where we should stop before anyone sanity goes under the bus.
>
> The purpose of IdentityClass is
> - to have a classfile for the javadoc describing the behavior of all identity class
> - to be used as type or bound of type parameter.
>
> Apart the result of Class::getInterfaces() being hardcoded, IdentityClass has to other issues, as javadoc container, given that IdentityClass is inserted by the VM, it means there is no place in the Java code where someone can click to see the javadoc in an IDE, we have the same issue with java.lang.Record currently, the class is added by javac automatically and unlike java.lang.Enum, there is no method useful on java.lang.Record (equals/hashCode and toString are defined directly on the record) so very few of my student where able to understand why a record with a NaN value was equals to itself.
> But there is a more serious issue, using IdentityClass is not backward compatible with Object.
> When we have introduced IdentityClass, one scenario was to be able to declare that the type parameter corresponding to the keys (K) to only support identity class.
> This is not possible using IdentityClass because the erasure of K will be IdentityClass instead of Object (IdentityClass also appears when the compiler will to a lub, so the common class of String and URI will be computed as IdentityClass instead of Object leading to source compatibility issues).
>
> I think at that point, we should go back to our blackboard and see if there is no other solution.
>
> I see two, one is to do *nothing* and do not add a type saying that only identity class is a corner case after all,
> the other is to piggyback on erasure not unlike Scala does, i.e. IdentityClass is a fake class that is erased to Object at runtime.
>
> regards,
> R?mi
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/valhalla-spec-experts/attachments/20210505/41dd3e50/attachment.htm>

From forax at univ-mlv.fr  Wed May  5 19:53:51 2021
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Wed, 5 May 2021 21:53:51 +0200 (CEST)
Subject: Meeting today: IdentityClass
In-Reply-To: <f2638ecd-9e62-239f-d540-435ec8bf2fe8@oracle.com>
References: <1847895792.1903234.1620225541253.JavaMail.zimbra@u-pem.fr>
 <f2638ecd-9e62-239f-d540-435ec8bf2fe8@oracle.com>
Message-ID: <112803847.1999643.1620244431005.JavaMail.zimbra@u-pem.fr>

> De: "Brian Goetz" <brian.goetz at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>, "valhalla-spec-experts"
> <valhalla-spec-experts at openjdk.java.net>
> Envoy?: Mercredi 5 Mai 2021 20:20:07
> Objet: Re: Meeting today: IdentityClass

> Here are some idioms I can imagine a use for with IdentityObject:

> // parameter type
> void withLock(IdentityObject monitor, Runnable task)

> // type bound
> Map<K extends IdentityObject, V> map = ... // gonna lock on keys

> // dynamic check
> if (x instanceof IdentityObject) {
> sync (x) { task(); }
> }
> else {
> sync (GLOBAL_LOCK) { task(); }
> }

> // reflective check against class
> Class<?> c = ...
> if (IdentityObject.class.isAssignableFrom(c)) { ... }

> The "fake type" approach handles the first two cases well enough. But it doesn't
> address the latter two cases; for that we'd need to expose a
> Class::isIdentityClass (x.getClass().isIdentityClass()), which seems workable,
> though might feel gratuitously different from the first two.
Erasure in the generic sense (sorry for the pun) is the way to retcon types as if they were present since the beginning in Java. 
So it fit well with the idea of IdentityObject/PrimitiveObject. 

Now, i think there are two kind of erasures, we have classic erasure by the compiler only like with generics and we have erasure with TypeRestriction which involve the VM. 
Note that the erasure using TypeRestriction does not necessarily implies any parametric type, this is a separate mechanism. 

As you said, classic erasure only supports the first two scenarios, while i believe that erasure by TypeRestriction support all of them. 

> Erasing to Object means that we don't get to do things like overloading:

> m(IdentityObject o) { ... }
> m(PrimitiveObject o) { ... }

> or

> m(IdentityObject o) { ... }
> m(Object o) { ... }
Sadly, this scenario is not supported by both kinds of erasure. 

And as i said two weeks ago, erasure by TypeRestriction is also something interesting for value based classes, at least some of them, the ones that do not allow null, by example for Optional. 
Optional the Q-type can be erased as Optional the L-type everywhere in the existing method signatures so we have binary backward compatibility at the price of 
- having the VM to lightweight box/unbox at the inlining horizons and 
- spurious NPEs where a null is sent as an Optional. 

Obviously, this is an incomplete proposal, but i think it's worth to take a look into that direction. 

R?mi 

> On 5/5/2021 10:39 AM, Remi Forax wrote:

>> If it's possible, i would like to discuss about IdentityClass again.

>> As noted in the last draft, adding IdentityClass automatically at runtime is not
>> a compatible change if tests that uses Class::getInterfaces() are not written
>> correctly,
>> and sadly, a lot of tests are written that way. I'm guilty to having written
>> those tests too.
>> I don't believe that the solution is to tweak the reflection because adding
>> IdentityClass at runtime is already a hack and introducing a hack? (hack square
>> hack) is usually where we should stop before anyone sanity goes under the bus.

>> The purpose of IdentityClass is
>> - to have a classfile for the javadoc describing the behavior of all identity
>> class
>> - to be used as type or bound of type parameter.

>> Apart the result of Class::getInterfaces() being hardcoded, IdentityClass has to
>> other issues, as javadoc container, given that IdentityClass is inserted by the
>> VM, it means there is no place in the Java code where someone can click to see
>> the javadoc in an IDE, we have the same issue with java.lang.Record currently,
>> the class is added by javac automatically and unlike java.lang.Enum, there is
>> no method useful on java.lang.Record (equals/hashCode and toString are defined
>> directly on the record) so very few of my student where able to understand why
>> a record with a NaN value was equals to itself.
>> But there is a more serious issue, using IdentityClass is not backward
>> compatible with Object.
>> When we have introduced IdentityClass, one scenario was to be able to declare
>> that the type parameter corresponding to the keys (K) to only support identity
>> class.
>> This is not possible using IdentityClass because the erasure of K will be
>> IdentityClass instead of Object (IdentityClass also appears when the compiler
>> will to a lub, so the common class of String and URI will be computed as
>> IdentityClass instead of Object leading to source compatibility issues).

>> I think at that point, we should go back to our blackboard and see if there is
>> no other solution.

>> I see two, one is to do *nothing* and do not add a type saying that only
>> identity class is a corner case after all,
>> the other is to piggyback on erasure not unlike Scala does, i.e. IdentityClass
>> is a fake class that is erased to Object at runtime.

>> regards,
>> R?mi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/valhalla-spec-experts/attachments/20210505/76d8d003/attachment-0001.htm>

From daniel.smith at oracle.com  Thu May  6 16:40:19 2021
From: daniel.smith at oracle.com (Dan Smith)
Date: Thu, 6 May 2021 16:40:19 +0000
Subject: Meeting today: IdentityClass
In-Reply-To: <1847895792.1903234.1620225541253.JavaMail.zimbra@u-pem.fr>
References: <1847895792.1903234.1620225541253.JavaMail.zimbra@u-pem.fr>
Message-ID: <49F889F4-292E-4DEB-B7CC-004E68A7D732@oracle.com>


> On May 5, 2021, at 8:39 AM, Remi Forax <forax at univ-mlv.fr> wrote:
> 
> If it's possible, i would like to discuss about IdentityClass again.
> 
> As noted in the last draft, adding IdentityClass automatically at runtime is not a compatible change if tests that uses Class::getInterfaces() are not written correctly,
> and sadly, a lot of tests are written that way. I'm guilty to having written those tests too.
> I don't believe that the solution is to tweak the reflection because adding IdentityClass at runtime is already a hack and introducing a hack? (hack square hack) is usually where we should stop before anyone sanity goes under the bus.

You're correct that this is a risk. I don't think it's a deal-breaker?there aren't that many use cases for using 'getInterfaces', and those that do should ideally be writing code that isn't hostile to new supers?but I agree it's something to watch out for. The current plan is to release a preview feature and get some feedback on how disruptive this is.

> Apart the result of Class::getInterfaces() being hardcoded, IdentityClass has to other issues, as javadoc container, given that IdentityClass is inserted by the VM, it means there is no place in the Java code where someone can click to see the javadoc in an IDE, we have the same issue with java.lang.Record currently, the class is added by javac automatically and unlike java.lang.Enum, there is no method useful on java.lang.Record (equals/hashCode and toString are defined directly on the record) so very few of my student where able to understand why a record with a NaN value was equals to itself.

That's interesting. In this case, less of a concern because IdentityObject doesn't specify any behaviors at all. It's just a marker. If you're confused about, say, 'equals' behavior for primitive objects, you can go to the Object.equals documentation to get an explanation.

Object also has this property?you extend it even if you don't mention it in source. And I think that works just because "everybody knows" that's how it works. Understanding which class declarations implicitly implement IdentityObject and what class declarations implicitly implement PrimitiveObject will similarly be something that people just need to understand as part of their language fluency.

(If you're browsing javadoc, we'll be sure to make this obvious, just like it's obvious in javadoc that everything extends Object.)

> But there is a more serious issue, using IdentityClass is not backward compatible with Object.
> When we have introduced IdentityClass, one scenario was to be able to declare that the type parameter corresponding to the keys (K) to only support identity class.
> This is not possible using IdentityClass because the erasure of K will be IdentityClass instead of Object (IdentityClass also appears when the compiler will to a lub, so the common class of String and URI will be computed as IdentityClass instead of Object leading to source compatibility issues).

As John mentioned yesterday: the simplest solution is to make sure erasure treats 'K extends Object & IdentityObject' differently than 'K extends IdentityObject'. A bit awkward, but workable for authors who need to preserve binary compatibility.

> 
> I think at that point, we should go back to our blackboard and see if there is no other solution.
> 
> I see two, one is to do *nothing* and do not add a type saying that only identity class is a corner case after all,
> the other is to piggyback on erasure not unlike Scala does, i.e. IdentityClass is a fake class that is erased to Object at runtime.

There are a few different pieces any solution should support (unless we're changing our requirements):

- Dynamic test for "is this object an identity object?" or "is this class an identity class?"
- Static type for a variable ('void m(IdentityObject o)')
- Type variable bound ('T extends IdentityObject')
- Concise way to constrain a class's subtypes to primitive/identity classes (both compile-time and runtime)

If you say IdentityObject is only a compile-time type, you've lost the dynamic tests, runtime guarantees about variable types, and runtime class subtyping restrictions. You haven't improved anything about javadoc discoverability.

Of course we could come up with other solutions for those problems, but that's a lot of complexity just to make life easier for 'getInterfaces' clients and library authors who want to compatibly apply IdentityObject as a bound. Doesn't seem like a good trade-off.


From daniel.smith at oracle.com  Fri May  7 22:06:59 2021
From: daniel.smith at oracle.com (Dan Smith)
Date: Fri, 7 May 2021 22:06:59 +0000
Subject: Making Object abstract
Message-ID: <C28E8B70-F0EA-4D19-AB24-46806AFFC710@oracle.com>

I notice a number of times in current design documents that we state a rule and then have a parenthetical "except for Object, which is special". A lot of this specialness is due to Object behaving like an abstract class, even though it is not abstract.

I think we'd have a cleaner design if we just updated Object to be abstract.

Implications:

- The expression 'new Object()' wouldn't normally be allowed, but we plan to special-case this anyway to produce an instance of IdentityObject.

- Similarly, a 'new java/lang/Object' instruction would normally cause a InstantiationError, but we plan to special-case it.

- Reflection would expose the ACC_ABSTRACT flag. <shrug>

- The JVM story for opting in to primitive subclasses would need to work for class Object just as well as other abstract classes. We can design for that.

- As long as this is a preview feature, we might need two different versions of Object.class, depending on whether --enable-preview is set. But we need to do that for Integer, etc., anyway, so should be doable.

Am I missing anything that would be particularly disruptive?

From brian.goetz at oracle.com  Fri May  7 22:38:00 2021
From: brian.goetz at oracle.com (Brian Goetz)
Date: Fri, 7 May 2021 18:38:00 -0400
Subject: Making Object abstract
In-Reply-To: <C28E8B70-F0EA-4D19-AB24-46806AFFC710@oracle.com>
References: <C28E8B70-F0EA-4D19-AB24-46806AFFC710@oracle.com>
Message-ID: <bff7c151-305d-de1f-3d93-6db5a243f9b0@oracle.com>

I think the question is: "cleaner for whom."? The parenthetical remarks 
"except for Object" are mostly our mess to corral; "new Object()" 
working, even when Object is seen to be abstract, will be extremely 
confusing for new developers learning what abstract means.? It's not 
just that reflection would expose this, but the source code and Javadoc 
would to.? User mental models are a stakeholder in Object too.



On 5/7/2021 6:06 PM, Dan Smith wrote:
> I notice a number of times in current design documents that we state a rule and then have a parenthetical "except for Object, which is special". A lot of this specialness is due to Object behaving like an abstract class, even though it is not abstract.
>
> I think we'd have a cleaner design if we just updated Object to be abstract.
>
> Implications:
>
> - The expression 'new Object()' wouldn't normally be allowed, but we plan to special-case this anyway to produce an instance of IdentityObject.
>
> - Similarly, a 'new java/lang/Object' instruction would normally cause a InstantiationError, but we plan to special-case it.
>
> - Reflection would expose the ACC_ABSTRACT flag. <shrug>
>
> - The JVM story for opting in to primitive subclasses would need to work for class Object just as well as other abstract classes. We can design for that.
>
> - As long as this is a preview feature, we might need two different versions of Object.class, depending on whether --enable-preview is set. But we need to do that for Integer, etc., anyway, so should be doable.
>
> Am I missing anything that would be particularly disruptive?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/valhalla-spec-experts/attachments/20210507/45b03729/attachment.htm>

From daniel.smith at oracle.com  Fri May  7 22:48:43 2021
From: daniel.smith at oracle.com (Dan Smith)
Date: Fri, 7 May 2021 22:48:43 +0000
Subject: Making Object abstract
In-Reply-To: <bff7c151-305d-de1f-3d93-6db5a243f9b0@oracle.com>
References: <C28E8B70-F0EA-4D19-AB24-46806AFFC710@oracle.com>
 <bff7c151-305d-de1f-3d93-6db5a243f9b0@oracle.com>
Message-ID: <580432CB-6575-4C4E-92B5-83CAB6C73467@oracle.com>

> On May 7, 2021, at 4:38 PM, Brian Goetz <brian.goetz at oracle.com> wrote:
> 
> I think the question is: "cleaner for whom."  The parenthetical remarks "except for Object" are mostly our mess to corral; "new Object()" working, even when Object is seen to be abstract, will be extremely confusing for new developers learning what abstract means.  It's not just that reflection would expose this, but the source code and Javadoc would to.  User mental models are a stakeholder in Object too.

It's a cleaner mental model for developers, too. For example, "primitive classes may only extend abstract classes" is a simple rule to remember. So is "concrete classes always extend IdentityObject or PrimitiveObject".

I would strongly discourage new developers from saying "new Object()", and I think javac/IDEs should warn about it. It's allowed purely for compatibility. If developers insist on using it, then whether we've decreed that Object is abstract or not, they're going to need to learn that "new Object()" is a special case that doesn't behave the way they expect "new Foo()" to behave.


From heidinga at redhat.com  Sat May  8 01:06:02 2021
From: heidinga at redhat.com (Dan Heidinga)
Date: Fri, 7 May 2021 21:06:02 -0400
Subject: Making Object abstract
In-Reply-To: <580432CB-6575-4C4E-92B5-83CAB6C73467@oracle.com>
References: <C28E8B70-F0EA-4D19-AB24-46806AFFC710@oracle.com>
 <bff7c151-305d-de1f-3d93-6db5a243f9b0@oracle.com>
 <580432CB-6575-4C4E-92B5-83CAB6C73467@oracle.com>
Message-ID: <CAJq4Gi6+KH4mAok0YyjcD2JD2oLa=M4rHzWmEbBY8dZBbWrcAQ@mail.gmail.com>

> I would strongly discourage new developers from saying "new Object()", and I think javac/IDEs should warn about it. It's allowed purely for compatibility. If developers insist on using it, then whether we've decreed that Object is abstract or not, they're going to need to learn that "new Object()" is a special case that doesn't behave the way they expect "new Foo()" to behave.
>

I had assumed that we were making `new Object()` illegal syntax when
the Primitive Classes jep landed.  We need to keep the bytecode
sequence working for legacy classfiles but don't need the language to
support writing it in the future.  Was that incorrect?

There was also a proposal to add a Object::newObject() helper method
as a replacement for using `new Object()` which may be why I assumed
the old syntax would be illegal in the future.

--Dan


From brian.goetz at oracle.com  Sat May  8 01:34:40 2021
From: brian.goetz at oracle.com (Brian Goetz)
Date: Fri, 7 May 2021 21:34:40 -0400
Subject: [External] : Re: Making Object abstract
In-Reply-To: <CAJq4Gi6+KH4mAok0YyjcD2JD2oLa=M4rHzWmEbBY8dZBbWrcAQ@mail.gmail.com>
References: <C28E8B70-F0EA-4D19-AB24-46806AFFC710@oracle.com>
 <bff7c151-305d-de1f-3d93-6db5a243f9b0@oracle.com>
 <580432CB-6575-4C4E-92B5-83CAB6C73467@oracle.com>
 <CAJq4Gi6+KH4mAok0YyjcD2JD2oLa=M4rHzWmEbBY8dZBbWrcAQ@mail.gmail.com>
Message-ID: <65df60bb-100a-25f8-1145-b7091518f0fb@oracle.com>

 From a compatibility perspective, we can't outlaw either `new Object()` 
or `new j/l/Object`, but we can (a) add Object::newInstance and (b) warn 
on recompilation to switch to `Object::newInstance`.

On 5/7/2021 9:06 PM, Dan Heidinga wrote:
>> I would strongly discourage new developers from saying "new Object()", and I think javac/IDEs should warn about it. It's allowed purely for compatibility. If developers insist on using it, then whether we've decreed that Object is abstract or not, they're going to need to learn that "new Object()" is a special case that doesn't behave the way they expect "new Foo()" to behave.
>>
> I had assumed that we were making `new Object()` illegal syntax when
> the Primitive Classes jep landed.  We need to keep the bytecode
> sequence working for legacy classfiles but don't need the language to
> support writing it in the future.  Was that incorrect?
>
> There was also a proposal to add a Object::newObject() helper method
> as a replacement for using `new Object()` which may be why I assumed
> the old syntax would be illegal in the future.
>
> --Dan
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/valhalla-spec-experts/attachments/20210507/ea63e5c0/attachment.htm>

From heidinga at redhat.com  Sat May  8 15:50:19 2021
From: heidinga at redhat.com (Dan Heidinga)
Date: Sat, 8 May 2021 11:50:19 -0400
Subject: [External] : Re: Making Object abstract
In-Reply-To: <65df60bb-100a-25f8-1145-b7091518f0fb@oracle.com>
References: <C28E8B70-F0EA-4D19-AB24-46806AFFC710@oracle.com>
 <bff7c151-305d-de1f-3d93-6db5a243f9b0@oracle.com>
 <580432CB-6575-4C4E-92B5-83CAB6C73467@oracle.com>
 <CAJq4Gi6+KH4mAok0YyjcD2JD2oLa=M4rHzWmEbBY8dZBbWrcAQ@mail.gmail.com>
 <65df60bb-100a-25f8-1145-b7091518f0fb@oracle.com>
Message-ID: <CAJq4Gi7Cg+OGzGxmTpmASJMhOhHftz-VwOyoYujq-Jvh3d392g@mail.gmail.com>

On Fri, May 7, 2021 at 9:34 PM Brian Goetz <brian.goetz at oracle.com> wrote:
>
> From a compatibility perspective, we can't outlaw either `new Object()` or `new j/l/Object`, but we can (a) add Object::newInstance and (b) warn on recompilation to switch to `Object::newInstance`.
>

We're in complete agreement on needing to support it at the bytecode
level.  The part I'm unclear on is why that requires continuing to
allow `new Object()` at the source level.  Removing it is, of course,
a source incompatible change with all its attendant pain, but it
results in the simpler model as suggested by Dan Smith and allows
making Object abstract.

Continuing to allow `new Object()` at the source level will encourage
puzzlers and confusion as expressions like this will be false only for
Object:
```
new Object().getClass() == Object.class
```
Isn't it better to make this illegal at the source level so the code
says what it means?

My (non-exhaustive) examination of uses of `new Object()` in code
bases I've looked at (which is a small, jvm-centric set of uses) tend
to fall into two buckets:
* tests generating objects to validate GCs occur, metrics report
additional allocations or free space correctly.  None of these cases
care about the Object itself.  It's just something to occupy heap
space.
* allocating a unique object to use as a lock

Is there some other pattern that suggests keeping `new Object()` at
the source level is worth the special cases in the model?

--Dan


> On 5/7/2021 9:06 PM, Dan Heidinga wrote:
>
> I would strongly discourage new developers from saying "new Object()", and I think javac/IDEs should warn about it. It's allowed purely for compatibility. If developers insist on using it, then whether we've decreed that Object is abstract or not, they're going to need to learn that "new Object()" is a special case that doesn't behave the way they expect "new Foo()" to behave.
>
> I had assumed that we were making `new Object()` illegal syntax when
> the Primitive Classes jep landed.  We need to keep the bytecode
> sequence working for legacy classfiles but don't need the language to
> support writing it in the future.  Was that incorrect?
>
> There was also a proposal to add a Object::newObject() helper method
> as a replacement for using `new Object()` which may be why I assumed
> the old syntax would be illegal in the future.
>
> --Dan
>
>


From brian.goetz at oracle.com  Sat May  8 17:02:53 2021
From: brian.goetz at oracle.com (Brian Goetz)
Date: Sat, 8 May 2021 13:02:53 -0400
Subject: [External] : Re: Making Object abstract
In-Reply-To: <CAJq4Gi7Cg+OGzGxmTpmASJMhOhHftz-VwOyoYujq-Jvh3d392g@mail.gmail.com>
References: <C28E8B70-F0EA-4D19-AB24-46806AFFC710@oracle.com>
 <bff7c151-305d-de1f-3d93-6db5a243f9b0@oracle.com>
 <580432CB-6575-4C4E-92B5-83CAB6C73467@oracle.com>
 <CAJq4Gi6+KH4mAok0YyjcD2JD2oLa=M4rHzWmEbBY8dZBbWrcAQ@mail.gmail.com>
 <65df60bb-100a-25f8-1145-b7091518f0fb@oracle.com>
 <CAJq4Gi7Cg+OGzGxmTpmASJMhOhHftz-VwOyoYujq-Jvh3d392g@mail.gmail.com>
Message-ID: <3f145839-0ad0-e3f0-f58f-958d8c42cc77@oracle.com>



> We're in complete agreement on needing to support it at the bytecode
> level.  The part I'm unclear on is why that requires continuing to
> allow `new Object()` at the source level.  Removing it is, of course,
> a source incompatible change with all its attendant pain, but it
> results in the simpler model as suggested by Dan Smith and allows
> making Object abstract.

I agree that if we were starting from scratch, making Object abstract 
would be the sensible choice.? But there's just sooooooo much code that 
does at least one `new Object()` at the source level, and this seems a 
questionable place to spend our incompatibility budget.? We don't want 
people's first experience of Valhalla -- before they even know what it 
is -- to be "nothing compiles anymore."? That's a way to get started on 
the wrong foot.

We can certainly discourage it by warning ("please use 
Object.newInstance() instead"), and start the migration early (we can 
start by adding Object::newInstance now, and add to the set of 
pre-valhalla optional warnings), but it seems like too big a leap to 
start erroring on `new Object()` immediately.

> Continuing to allow `new Object()` at the source level will encourage
> puzzlers and confusion as expressions like this will be false only for
> Object:
> ```
> new Object().getClass() == Object.class
> ```
> Isn't it better to make this illegal at the source level so the code
> says what it means?
>
> My (non-exhaustive) examination of uses of `new Object()` in code
> bases I've looked at (which is a small, jvm-centric set of uses) tend
> to fall into two buckets:
> * tests generating objects to validate GCs occur, metrics report
> additional allocations or free space correctly.  None of these cases
> care about the Object itself.  It's just something to occupy heap
> space.
> * allocating a unique object to use as a lock
>
> Is there some other pattern that suggests keeping `new Object()` at
> the source level is worth the special cases in the model?
>
> --Dan
Most cases are the lock.? And while Joe Java doesn't write this that 
often, nearly every library has this once or twice.

So, I agree on the goal, I guess I don't think we can get there in one 
go without avoidably burning some goodwill.



From daniel.smith at oracle.com  Sat May  8 18:01:31 2021
From: daniel.smith at oracle.com (Dan Smith)
Date: Sat, 8 May 2021 18:01:31 +0000
Subject: Making Object abstract
In-Reply-To: <CAJq4Gi7Cg+OGzGxmTpmASJMhOhHftz-VwOyoYujq-Jvh3d392g@mail.gmail.com>
References: <C28E8B70-F0EA-4D19-AB24-46806AFFC710@oracle.com>
 <bff7c151-305d-de1f-3d93-6db5a243f9b0@oracle.com>
 <580432CB-6575-4C4E-92B5-83CAB6C73467@oracle.com>
 <CAJq4Gi6+KH4mAok0YyjcD2JD2oLa=M4rHzWmEbBY8dZBbWrcAQ@mail.gmail.com>
 <65df60bb-100a-25f8-1145-b7091518f0fb@oracle.com>
 <CAJq4Gi7Cg+OGzGxmTpmASJMhOhHftz-VwOyoYujq-Jvh3d392g@mail.gmail.com>
Message-ID: <87B00510-0F0C-4961-B2EA-6816AEF810ED@oracle.com>



> On May 8, 2021, at 9:50 AM, Dan Heidinga <heidinga at redhat.com> wrote:
> 
> On Fri, May 7, 2021 at 9:34 PM Brian Goetz <brian.goetz at oracle.com> wrote:
>> 
>> From a compatibility perspective, we can't outlaw either `new Object()` or `new j/l/Object`, but we can (a) add Object::newInstance and (b) warn on recompilation to switch to `Object::newInstance`.
>> 
> 
> We're in complete agreement on needing to support it at the bytecode
> level.  The part I'm unclear on is why that requires continuing to
> allow `new Object()` at the source level.  Removing it is, of course,
> a source incompatible change with all its attendant pain, but it
> results in the simpler model as suggested by Dan Smith and allows
> making Object abstract.

One way or another, we've got a migration problem?we want people to update their code to use a new idiom. This could look like:

- With Primitive Objects, introduce an alternative API point & javac warnings. After a few versions, maybe drop support for the constructor call.

- Before Primitive Objects, introduce an alternative API point & javac warnings. With Primitive Objects (final), make the constructor call illegal.

There needs to be some sort of gradual pathway?can't just spring it on people when Primitive Objects are final. We did the latter with the primitive wrappers, but in that case the deprecation began in 9 (2017), and I think 'new Integer' is a more rare occurrence than 'new Object'. Not sure we could squeeze it in before Primitive Objects are final. (But if we want to, we'd better get moving on it!)

I see the question of whether Object is abstract as orthogonal: unless we remove support for 'new Object' before the change, we're going to have a period where 'new Object()' is special, and not necessarily subject to the "can't instantiate an abstract class" rule.

From brian.goetz at oracle.com  Sat May  8 19:20:35 2021
From: brian.goetz at oracle.com (Brian Goetz)
Date: Sat, 8 May 2021 15:20:35 -0400
Subject: Making Object abstract
In-Reply-To: <87B00510-0F0C-4961-B2EA-6816AEF810ED@oracle.com>
References: <C28E8B70-F0EA-4D19-AB24-46806AFFC710@oracle.com>
 <bff7c151-305d-de1f-3d93-6db5a243f9b0@oracle.com>
 <580432CB-6575-4C4E-92B5-83CAB6C73467@oracle.com>
 <CAJq4Gi6+KH4mAok0YyjcD2JD2oLa=M4rHzWmEbBY8dZBbWrcAQ@mail.gmail.com>
 <65df60bb-100a-25f8-1145-b7091518f0fb@oracle.com>
 <CAJq4Gi7Cg+OGzGxmTpmASJMhOhHftz-VwOyoYujq-Jvh3d392g@mail.gmail.com>
 <87B00510-0F0C-4961-B2EA-6816AEF810ED@oracle.com>
Message-ID: <50b40e1e-ac19-b97b-afc6-43de88f5ee66@oracle.com>

I agree that we can introduce the new API point immediately.? The 17 
window hasn't even closed yet!? But we'd have to get a move on.? But 
realistically, we can expect it to be several years before we are 
comfortable erroring on the `new Object()` constructor.



> One way or another, we've got a migration problem?we want people to update their code to use a new idiom. This could look like:
>
> - With Primitive Objects, introduce an alternative API point & javac warnings. After a few versions, maybe drop support for the constructor call.
>
> - Before Primitive Objects, introduce an alternative API point & javac warnings. With Primitive Objects (final), make the constructor call illegal.
>
> There needs to be some sort of gradual pathway?can't just spring it on people when Primitive Objects are final. We did the latter with the primitive wrappers, but in that case the deprecation began in 9 (2017), and I think 'new Integer' is a more rare occurrence than 'new Object'. Not sure we could squeeze it in before Primitive Objects are final. (But if we want to, we'd better get moving on it!)
>
> I see the question of whether Object is abstract as orthogonal: unless we remove support for 'new Object' before the change, we're going to have a period where 'new Object()' is special, and not necessarily subject to the "can't instantiate an abstract class" rule.


From forax at univ-mlv.fr  Sun May  9 14:15:17 2021
From: forax at univ-mlv.fr (Remi Forax)
Date: Sun, 9 May 2021 16:15:17 +0200 (CEST)
Subject: Make new Object() backward compatible
Message-ID: <240068393.1575497.1620569717740.JavaMail.zimbra@u-pem.fr>

I think there is a way to avoid all the issues we have with new Object().

The problem:
  We want to inject the interface IdentityObject on all classes that are neither a "no field" abstract class nor a primtive class.
  But java.lang.Object acts as a "no field" abstract class but it's not an abstract class.

Clearly, there are two java.lang.Object, the one which is the root of all classes (the "no field" abstract one) and the one which can be instantiated that should implements IdentityObject.

The logical conclusion is that java.lang.Object is a parametric class with a parameter saying if it should implement IdentityObject or not.

The "raw" version does not implement IdentityObject, that the one used when a class extends Object.
The parameterized version implements IdentityObject and is the one used by new Object()

So the classfile Object.class should contain a SpecializationAnchor, and being a "no field" class or not is an attribute that references the Anchor.

To maintain backward compatibility, when the VM sees the opcode NEW java/lang/Object, it adds a new constant pool entry of kind SpecializationLinkage that reference java/lang/Object with a constant asking for version that implements IdentityObject and rewrite NEW java/lang/Object to NEW Linkage(java/lang/Object).

Obviously, there is the bootstraping issue with the code in java.lang.invoke making the BSM working not be initialized at the time java.lang.Object need to be available, so the VM has to inject the binding early as if it was resolved by the BSM but i think it's workable.

When the VM starts, if there are some new Object() executedbefore the code of java.lang.invoke is ready, those code can be rewritten to use a special class defined in the JDK for that like
  package jdk.internal.whatever;
  class LockObject { }

Even if it's not strictly necessary, javac can be changed to emit the Linkage(java/lang/Object) each time a new Object() is needed to the rewriting by the VM described above need to be done only for the classfiles that are using previous versions.

I believe this change is fully source and binary compatible and nice property is that 
  new Object().getClass() == Object.class
is still true.

R?mi

From brian.goetz at oracle.com  Sun May  9 15:04:31 2021
From: brian.goetz at oracle.com (Brian Goetz)
Date: Sun, 9 May 2021 11:04:31 -0400
Subject: Make new Object() backward compatible
In-Reply-To: <240068393.1575497.1620569717740.JavaMail.zimbra@u-pem.fr>
References: <240068393.1575497.1620569717740.JavaMail.zimbra@u-pem.fr>
Message-ID: <253c0d47-7c48-5f09-c394-b53fa9870816@oracle.com>



> The logical conclusion is that java.lang.Object is a parametric class with a parameter saying if it should implement IdentityObject or not.

Magic hammer, meet nail :)

To be explicit, what you're suggesting is something like

 ??? class Object<ID : boolean> { ... }

where an identity class extends Object<true> and a primitive class 
extends Object<false> (and most interfaces remain parametric in ID.)

Its conceptually elegant, and we've discussed it before; in theory 
parametricity could range over things other than types, such as 
constants, identity-ness, etc (e.g., the ref and val projections could 
be species).? But to apply it to a problem as prosaic as this seems like 
cracking a walnut with a two-ton bomb.? (I don't thing this is what you 
want to teach on day 1.)



From forax at univ-mlv.fr  Sun May  9 16:09:06 2021
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Sun, 9 May 2021 18:09:06 +0200 (CEST)
Subject: Make new Object() backward compatible
In-Reply-To: <253c0d47-7c48-5f09-c394-b53fa9870816@oracle.com>
References: <240068393.1575497.1620569717740.JavaMail.zimbra@u-pem.fr>
 <253c0d47-7c48-5f09-c394-b53fa9870816@oracle.com>
Message-ID: <1779893749.1593659.1620576546861.JavaMail.zimbra@u-pem.fr>

----- Mail original -----
> De: "Brian Goetz" <brian.goetz at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>, "valhalla-spec-experts" <valhalla-spec-experts at openjdk.java.net>
> Envoy?: Dimanche 9 Mai 2021 17:04:31
> Objet: Re: Make new Object() backward compatible

>> The logical conclusion is that java.lang.Object is a parametric class with a
>> parameter saying if it should implement IdentityObject or not.
> 
> Magic hammer, meet nail :)

New tool, new tricks :)

Given that we want two classes for java.lang.Object, seeing them as one being a specialization of the other is not too outrageous.

> 
> To be explicit, what you're suggesting is something like
> 
> ??? class Object<ID : boolean> { ... }
> 
> where an identity class extends Object<true> and a primitive class
> extends Object<false> (and most interfaces remain parametric in ID.)

yes

> 
> Its conceptually elegant, and we've discussed it before; in theory
> parametricity could range over things other than types, such as
> constants, identity-ness, etc (e.g., the ref and val projections could
> be species).? But to apply it to a problem as prosaic as this seems like
> cracking a walnut with a two-ton bomb.? (I don't thing this is what you
> want to teach on day 1.)

It seems that you are thinking that i want to introduce a special syntax just for specializing java.lang.Object, I don't.

I'm not proposing to change to the source code of Object.java, only the bytecode.
The compiler will insert the SpecializationAnchor (and the attribute that adds the identity-ness) without needing any language support.

The VM can also dynamically insert the SpecializationAnchor, but modifying javac to get the bytecode we want seems an easier path.

R?mi

From daniel.smith at oracle.com  Thu May 13 23:42:45 2021
From: daniel.smith at oracle.com (Dan Smith)
Date: Thu, 13 May 2021 23:42:45 +0000
Subject: JEP 401 & 402 updates
Message-ID: <634439FB-D8E5-4DAC-87B4-FF9AB81C585C@oracle.com>

I've made some revisions to these two JEPs, mainly to update their treatment of L descriptors/$ref classes, and to revise the story for core reflection.

https://openjdk.java.net/jeps/401
https://openjdk.java.net/jeps/402


From daniel.smith at oracle.com  Wed May 19 14:54:13 2021
From: daniel.smith at oracle.com (Dan Smith)
Date: Wed, 19 May 2021 14:54:13 +0000
Subject: EG meeting, 2021-05-19
Message-ID: <C5E7F8ED-D7DB-404E-A7EC-9B995995F208@oracle.com>

The next EG Zoom meeting is today at 4pm UTC (9am PDT, 12pm EDT).

Topics to discuss:

"Making Object abstract": timeline and goals for evolving Object instantiation

"JEP 401 & 402 updates": revised documents available (may be nothing to discuss, bring your questions if you have any)