From gavin.bierman at oracle.com Wed Nov 4 17:50:23 2020 From: gavin.bierman at oracle.com (Gavin Bierman) Date: Wed, 4 Nov 2020 17:50:23 +0000 Subject: [sealed-classes] Spec for next version of Sealed Classes In-Reply-To: References: Message-ID: <8D8979A8-3C74-4D05-A749-DB29477408AC@oracle.com> The permanent location for this spec is now: http://cr.openjdk.java.net/~gbierman/jep397/latest/ (I have fixed the two issues that Chris spotted [1].) Comments still welcome! Gavin [1] http://mail.openjdk.java.net/pipermail/amber-spec-experts/2020-October/002624.html > On 23 Oct 2020, at 16:16, Gavin Bierman wrote: > > Dear all: > > Drafts of the specs for the Sealed Classes feature that we plan to preview for > a second time in JDK 16 are now available: > > http://cr.openjdk.java.net/~gbierman/8246775/latest/ > > [NB: The URL will change once we have a JEP number, and will be announced.] > > The changes are the same as those in the first preview that was released in Java > SE 15, except for minor editorial changes and the following: > > - Clarification the use of context when applying the lexical grammar, > particularly in the identification of contextual keywords (formerly described > as "restricted identifiers" and "restricted keywords"). This is detailed in a > companion document entitled ?Contextual Keywords". The keywords `sealed`, > `non-sealed`, and `permits` are now defined as new instances of contextual > keywords (3.9). > > - This spec now assumes that the changes detailed in companion documents > entitled "Consistent Class and Interface Terminology? and "Local and Nested > Static Declarations" have been applied (these are being introduced as part of > the Records JEP). In particular, this means that Java SE 16 will support > static declarations in two new positions: > > 1. Local, implicitly-static interfaces and enum classes > 2. Static members of inner classes > > This requires asserting that local interfaces are not permitted to be > `sealed`. (14.3) > > - To enhance narrowing reference conversion to allow for stricter checking of > cast conversions with respect to sealed type hierarchies (5.1.6.1). > > - Local classes are not considered when determining implicitly declared > permitted direct subclasses of a `sealed` class or `sealed` interface > (8.1.6, 9.1.4). > > > Comments welcome! > > Gavin -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Thu Nov 5 10:31:43 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 5 Nov 2020 10:31:43 +0000 Subject: The Record Attribute - What does it mean to be a record at runtime? Message-ID: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> This email has no concrete proposal describe within, but rather is intended to start a discussion relating to the finer-grain aspects of the Record Attribute and the primitives built atop. Records offers a new protocol that can be leveraged at runtime by frameworks and libraries; A record class has a known state, accessors of that state, and a canonical constructor to initialize the state. Taken together these properties offer a new protocol for libraries to interact with records. We already see this with serialization libraries, e.g. Java Serialization, Jackson, Kryo, etc. The language, as it should, provides very strong guarantees about what it means to be a record class; is final, j.l.Record is the direct superclass, a final field and accessor for each component, a canonical constructor, etc. The JVM, however, does not provide such strong guarantees, which is not surprising. There is, and it is desirable to have, flexibility between the language and the JVM. The last piece of the puzzle, which builds atop the runtime, is the reflective support in the Java Core Libraries, namely j.l.Class and friends. For frameworks and libraries to take advantage of the "records protocol", then they need to be able to introspect and determine the various properties of a record class. To date, the Class::isRecord and Class::getRecordComponents primitives have proved sufficient for this. Lastly, and highly desirable, is that we've been able to expand the notion of "trusted final fields" to the fields of record classes [1]. The impact of this on the Java Core Libraries is that the fields (backing that of the record components) are not writable through the various Core Reflection, MethodHandle and VarHandles, APIs. There is a clear and tight coupling between the Java Reflection APIs and that of the JVM - they don't necessarily need to operate or even behave in the exact same way, but the relationship should be clear and well-specified. When it is not, we often find undesirable and uncomfortable behavior in dark corners - "Here be Dragons"! A recent change, 8255342 [2], laxifies the JVM checking of the Record Attribute - the JVM will happily consider a non-final, abstract class, that is not a direct subclass of j.l.Record to be a record class. The JVM implementation stores the record-ness property which feeds into the consideration of trust-final-non-static-fields, which can then feed into potential behavior (and optimizations?) in the JVM. In the Java Reflection APIs, the "writability" of the field of the record class is determine by the JVM's notion of whether or not class C is a record, and not C.isRecord - which may disagree with the JVM. But the specification of writability in the Core Reflection APIs builds upon Class::isRecord (what else could it do!). Not to mention object field lookup in Unsafe. We have a JIRA issue [3] tracking the potential knock-on affect on the Java Core Reflection APIs resulting from the changes for 8255342. We could just "fix" that bug and move on, but I think there is something more fundamental at stake here, and it may be best to re-consider some prior decisions relating to the interpretation of the Record Attribute itself. I don't have all the answers, but it seems that at an early stage the Record Attribute was merely some meta-data carried in the class file, hence its position in the second of the predefined attribute categorizes, "not critical to correct interpretation of the class file by the Java Virtual Machine, but are either critical to correct interpretation of the class file by the class libraries of the Java SE Platform". Is that still the case? Does the JVM consider the record-ness of a class for other parts of it operation now (trusting final fields)? It seems highly desirable that both the JVM and the Java Reflection Libraries agree on what it means to be a record at runtime. And where they do not, then that should be explicitly understood and the implications assessed. Comments welcome. -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8247444 [2] https://bugs.openjdk.java.net/browse/JDK-8255342 [3] https://bugs.openjdk.java.net/browse/JDK-8255560 From chris.hegarty at oracle.com Thu Nov 5 11:18:27 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 5 Nov 2020 11:18:27 +0000 Subject: getPermittedSubclasses() on j.l.rClass returning an array of ClassDesc In-Reply-To: References: <1800371247.929985.1588978383614.JavaMail.zimbra@u-pem.fr> <6333A7B0-144A-40A0-80A2-AF6778BF24F6@oracle.com> <8efdb9b2-939f-7c44-0940-7aad547c4c3a@oracle.com> <2025935165.2103166.1603539326260.JavaMail.zimbra@u-pem.fr> <1649379363.2194178.1603578742907.JavaMail.zimbra@u-pem.fr> <5088088E-CBA5-4853-BA9B-3272F3C1F66F@oracle.com> <1491892860.769152.1603799502470.JavaMail.zimbra@u-pem.fr> Message-ID: > On 29 Oct 2020, at 21:38, Dan Smith wrote: > >> ... > > You're not wrong, but I'm not sure this is reason not to provide better typing information. It certainly is the contract of the 'getPermittedSubclasses' method not to pollute it with non-subclasses of T. > > That said, Class[] seems to be the precedent followed by other methods: > > Class getSuperclass() > vs. > Class[] getInterfaces() > > Constructor getConstructor(Class) > vs. > Constructor[] getConstructors() > > So I guess we should do the same with 'getPermittedSubclasses'. For reference, 8246278 "Refine API for sealing in java.lang.Class? [1], tracks this issue. An additional point for consideration, the return "container". Returning a Class[] or maybe an immutable List. The latter allows for sharper type information in the signature, the former does not. -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8246278 -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Thu Nov 5 12:39:46 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 5 Nov 2020 13:39:46 +0100 (CET) Subject: getPermittedSubclasses() on j.l.rClass returning an array of ClassDesc In-Reply-To: References: <1800371247.929985.1588978383614.JavaMail.zimbra@u-pem.fr> <1649379363.2194178.1603578742907.JavaMail.zimbra@u-pem.fr> <5088088E-CBA5-4853-BA9B-3272F3C1F66F@oracle.com> <1491892860.769152.1603799502470.JavaMail.zimbra@u-pem.fr> Message-ID: <530129863.307789.1604579986758.JavaMail.zimbra@u-pem.fr> > De: "Chris Hegarty" > ?: "amber-spec-experts" > Envoy?: Jeudi 5 Novembre 2020 12:18:27 > Objet: Re: getPermittedSubclasses() on j.l.rClass returning an array of > ClassDesc >> On 29 Oct 2020, at 21:38, Dan Smith < [ mailto:daniel.smith at oracle.com | >> daniel.smith at oracle.com ] > wrote: >>> ... >> You're not wrong, but I'm not sure this is reason not to provide better typing >> information. It certainly is the contract of the 'getPermittedSubclasses' >> method not to pollute it with non-subclasses of T. >> That said, Class[] seems to be the precedent followed by other methods: >> Class getSuperclass() >> vs. >> Class[] getInterfaces() >> Constructor getConstructor(Class) >> vs. >> Constructor[] getConstructors() >> So I guess we should do the same with 'getPermittedSubclasses'. > For reference, 8246278 "Refine API for sealing in java.lang.Class? [1], tracks > this issue. > An additional point for consideration, the return "container". Returning a > Class[] or maybe an immutable List. The latter allows for > sharper type information in the signature, the former does not. yes, but if getPermittedSubclasses() returns a List, we are back to the signature of getPermittedSubclasses() making it an outlier. > -Chris. R?mi > [1] [ https://bugs.openjdk.java.net/browse/JDK-8246278 | > https://bugs.openjdk.java.net/browse/JDK-8246278 ] -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Thu Nov 5 23:59:27 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 6 Nov 2020 00:59:27 +0100 (CET) Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> Message-ID: <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Chris Hegarty" > ?: "amber-spec-experts" > Envoy?: Jeudi 5 Novembre 2020 11:31:43 > Objet: The Record Attribute - What does it mean to be a record at runtime? > This email has no concrete proposal describe within, but rather is > intended to start a discussion relating to the finer-grain aspects of > the Record Attribute and the primitives built atop. > > Records offers a new protocol that can be leveraged at runtime by > frameworks and libraries; A record class has a known state, accessors > of that state, and a canonical constructor to initialize the state. > Taken together these properties offer a new protocol for libraries to > interact with records. We already see this with serialization libraries, > e.g. Java Serialization, Jackson, Kryo, etc. > > The language, as it should, provides very strong guarantees about what > it means to be a record class; is final, j.l.Record is the direct > superclass, a final field and accessor for each component, a canonical > constructor, etc. The JVM, however, does not provide such strong > guarantees, which is not surprising. There is, and it is desirable to > have, flexibility between the language and the JVM. > > The last piece of the puzzle, which builds atop the runtime, is the > reflective support in the Java Core Libraries, namely j.l.Class and > friends. For frameworks and libraries to take advantage of the "records > protocol", then they need to be able to introspect and determine the > various properties of a record class. To date, the Class::isRecord and > Class::getRecordComponents primitives have proved sufficient for this. > > Lastly, and highly desirable, is that we've been able to expand the > notion of "trusted final fields" to the fields of record classes [1]. > The impact of this on the Java Core Libraries is that the fields > (backing that of the record components) are not writable through the > various Core Reflection, MethodHandle and VarHandles, APIs. > > There is a clear and tight coupling between the Java Reflection APIs > and that of the JVM - they don't necessarily need to operate or even > behave in the exact same way, but the relationship should be clear and > well-specified. When it is not, we often find undesirable and > uncomfortable behavior in dark corners - "Here be Dragons"! > > A recent change, 8255342 [2], laxifies the JVM checking of the Record > Attribute - the JVM will happily consider a non-final, abstract class, > that is not a direct subclass of j.l.Record to be a record class. The > JVM implementation stores the record-ness property which feeds into the > consideration of trust-final-non-static-fields, which can then feed into > potential behavior (and optimizations?) in the JVM. In the Java > Reflection APIs, the "writability" of the field of the record class is > determine by the JVM's notion of whether or not class C is a record, and > not C.isRecord - which may disagree with the JVM. But the specification > of writability in the Core Reflection APIs builds upon Class::isRecord > (what else could it do!). Not to mention object field lookup in Unsafe. > > We have a JIRA issue [3] tracking the potential knock-on affect on > the Java Core Reflection APIs resulting from the changes for 8255342. > We could just "fix" that bug and move on, but I think there is something > more fundamental at stake here, and it may be best to re-consider some > prior decisions relating to the interpretation of the Record Attribute > itself. I don't have all the answers, but it seems that at an early > stage the Record Attribute was merely some meta-data carried in the > class file, hence its position in the second of the predefined attribute > categorizes, "not critical to correct interpretation of the class file > by the Java Virtual Machine, but are either critical to correct > interpretation of the class file by the class libraries of the Java SE > Platform". Is that still the case? Does the JVM consider the record-ness > of a class for other parts of it operation now (trusting final fields)? > > It seems highly desirable that both the JVM and the Java Reflection > Libraries agree on what it means to be a record at runtime. And where > they do not, then that should be explicitly understood and the > implications assessed. > > Comments welcome. What a record is, is defined by the JLS but not by the JVMS (like enums), so the JVM should not know what a record is given that it's an artifact of the language not an artifact of the VM. We are also trying to enforce more unmodifiability of the final fields in the VM, Reflection, Unsafe, VarHandle, etc. Because of backward compatibility, we can not just make all final fields unmodifiable, because some libraries (or worst some Jakarta EE specs) relies on the fact that a final field can be changed after construction. So we are selectively enabling a more stronger stance on the modifiability of final field only for new constructs, VM anonymous class, VM hidden class and record class. Here the VM doesn't really need to know what a record is but only that this is a new construct so the rules around final fields are stronger, stricter. So for me, we do not need to do more in the VM. Now, for the reflection part, the reflection is mostly the Java view of the world, it's not stricto sensu true because you can see the desugaring done by the compiler. The reflection knows what a record is, it has to directly inherits from java.lang.Record, be final and have a RecordComponents attribute (gently provided by the VM). So both isRecord and getRecordComponents should checks these properties. I hope that help. > > -Chris. R?mi > > [1] https://bugs.openjdk.java.net/browse/JDK-8247444 > [2] https://bugs.openjdk.java.net/browse/JDK-8255342 > [3] https://bugs.openjdk.java.net/browse/JDK-8255560 From chris.hegarty at oracle.com Tue Nov 10 14:23:22 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 10 Nov 2020 14:23:22 +0000 Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> Message-ID: Thanks for your reply Remi, comments inline. > On 5 Nov 2020, at 23:59, Remi Forax wrote: >> >> .. >> A recent change, 8255342 [2], laxifies the JVM checking of the Record >> > ... > What a record is, is defined by the JLS but not by the JVMS (like enums), > so the JVM should not know what a record is given that it's an artifact of the language not an artifact of the VM. Agreed. > We are also trying to enforce more unmodifiability of the final fields in the VM, Reflection, Unsafe, VarHandle, etc. Yes. And this is a good move. > Because of backward compatibility, we can not just make all final fields unmodifiable, because some libraries (or worst some Jakarta EE specs) relies on the fact that a final field can be changed after construction. > So we are selectively enabling a more stronger stance on the modifiability of final field only for new constructs, VM anonymous class, VM hidden class and record class. Yes, this is my understanding also. > Here the VM doesn't really need to know what a record is but only that this is a new construct so the rules around final fields are stronger, stricter. > So for me, we do not need to do more in the VM. I agree that The Abstract JVM does not need to concern itself with the details of a record beyond the currently specified checks for the Record Attribute in the JVMS. > Now, for the reflection part, the reflection is mostly the Java view of the world, it's not stricto sensu true because you can see the desugaring done by the compiler. > The reflection knows what a record is, it has to directly inherits from java.lang.Record, be final and have a RecordComponents attribute (gently provided by the VM). > So both isRecord and getRecordComponents should checks these properties. Currently the core Java Reflection implementations determine whether a field is a trusted final based on information served up by the VM. One of the criteria that the VM uses to determine this is whether, or not, the field's containing class is a record. The VM asserts that a class is a record if it has a parsable Record attribute. Core reflection, however, determines a class to be a record class if the class is 1) a direct subclass of j.l.Record, and 2) contains a Record attribute. One option is to just update Class::isRecord to ensure, along with the above #1 and #2 checks, that the class is also 3) final, and 4) non-abstract. But this leaves a kind of impedance mismatch between both the VM and the libraries, and would require Core Reflection, MethodHandles, and VarHandles to also be updated, since their implementation is based on the VM's trusted final determination, whereas their specification is built atop Class::isRecord. A better option would be to update the VM implementation to only assert that the fields of a record-like class are trusted if that class contains 1) a structurally sound Record attribute, 2) is a direct subclass of j.l.Record, 3) is final, and 4) is non-abstract. This would align Core Reflection and the VM in this respect, while the JVMS would remain unchanged - it's an implementation detail. The above option is actually a slight rework of the changes for 8255342. (it is not a reversal of 8255342). The VM would still continue to parse and structurally verify the Record attribute, but only determine the record-ness/trusted-finalness-of-fields if the class is truly a record class as envisaged by the JLS. This seems like a reasonable compromise position. ( Note: with this proposal, classes that contain a Record Attribute, but without the other properties of what is means to be JSL record class, would behave like normal classes - their fields would not be trusted finals - which, again, seems reasonable. -Chris. -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Tue Nov 10 19:36:17 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Tue, 10 Nov 2020 20:36:17 +0100 (CET) Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> Message-ID: <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> > De: "Chris Hegarty" > ?: "Remi Forax" > Cc: "amber-spec-experts" > Envoy?: Mardi 10 Novembre 2020 15:23:22 > Objet: Re: The Record Attribute - What does it mean to be a record at runtime? > Thanks for your reply Remi, comments inline. >> On 5 Nov 2020, at 23:59, Remi Forax < [ mailto:forax at univ-mlv.fr | >> forax at univ-mlv.fr ] > wrote: >>> .. >>> A recent change, 8255342 [2], laxifies the JVM checking of the Record >> ... >> What a record is, is defined by the JLS but not by the JVMS (like enums), >> so the JVM should not know what a record is given that it's an artifact of the >> language not an artifact of the VM. > Agreed. >> We are also trying to enforce more unmodifiability of the final fields in the >> VM, Reflection, Unsafe, VarHandle, etc. > Yes. And this is a good move. >> Because of backward compatibility, we can not just make all final fields >> unmodifiable, because some libraries (or worst some Jakarta EE specs) relies on >> the fact that a final field can be changed after construction. >> So we are selectively enabling a more stronger stance on the modifiability of >> final field only for new constructs, VM anonymous class, VM hidden class and >> record class. > Yes, this is my understanding also. >> Here the VM doesn't really need to know what a record is but only that this is a >> new construct so the rules around final fields are stronger, stricter. >> So for me, we do not need to do more in the VM. > I agree that The Abstract JVM does not need to concern itself with the > details of a record beyond the currently specified checks for the Record > Attribute in the JVMS. >> Now, for the reflection part, the reflection is mostly the Java view of the >> world, it's not stricto sensu true because you can see the desugaring done by >> the compiler. >> The reflection knows what a record is, it has to directly inherits from >> java.lang.Record, be final and have a RecordComponents attribute (gently >> provided by the VM). >> So both isRecord and getRecordComponents should checks these properties. > Currently the core Java Reflection implementations determine whether a > field is a trusted final based on information served up by the VM. One > of the criteria that the VM uses to determine this is whether, or not, > the field's containing class is a record. The VM asserts that a class is > a record if it has a parsable Record attribute. > Core reflection, however, determines a class to be a record class if the > class is 1) a direct subclass of j.l.Record, and 2) contains a Record > attribute. > One option is to just update Class::isRecord to ensure, along with the > above #1 and #2 checks, that the class is also 3) final, and > 4) non-abstract. But this leaves a kind of impedance mismatch between > both the VM and the libraries, and would require Core Reflection, > MethodHandles, and VarHandles to also be updated, since their > implementation is based on the VM's trusted final determination, whereas > their specification is built atop Class::isRecord. > A better option would be to update the VM implementation to only assert > that the fields of a record-like class are trusted if that class > contains 1) a structurally sound Record attribute, 2) is a direct > subclass of j.l.Record, 3) is final, and 4) is non-abstract. This would > align Core Reflection and the VM in this respect, while the JVMS would > remain unchanged - it's an implementation detail. No, it's pushing the JLS semantics into the JVM. What you want to know is if a field is trusted final or not, to disallow the creation of VarHandle, the use Unsafe, etc. So the JDK API should ask the VM if a field is trusted final or not instead of asking if the class is a record. It will also be easier to update each time you introduce a new kind of classes, by example when primitive objects will be introduced, you have only to update isTrustedFinalField and not at each place you test if a field is trusted. > -Chris. R?mi -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Tue Nov 10 20:51:38 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 10 Nov 2020 20:51:38 +0000 Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> Message-ID: Remi, A point of clarification ( which I realise may have been ambiguous in my last email ). > On 10 Nov 2020, at 19:36, forax at univ-mlv.fr wrote: > ... > > A better option would be to update the VM implementation to only assert > that the fields of a record-like class are trusted if that class > contains 1) a structurally sound Record attribute, 2) is a direct > subclass of j.l.Record, 3) is final, and 4) is non-abstract. This would > align Core Reflection and the VM in this respect, while the JVMS would > remain unchanged - it's an implementation detail. > > No, it's pushing the JLS semantics into the JVM. > > What you want to know is if a field is trusted final or not, to disallow the creation of VarHandle, the use Unsafe, etc. > So the JDK API should ask the VM if a field is trusted final or not instead of asking if the class is a record. Agreed. The VM exposes a (private) interface to the core JDK libraries to tell whether a field is trusted or not. No issue. ( I did not mean to say otherwise ) My issue is with how the VM determines whether a field is trusted or not. The VM trusts fields in (among other types) ?record? classes. So what is a record class to the VM? (that is the question that I am trying to resolve) - the answer is not in the JVMS ( which is fine ). -Chris. -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Tue Nov 10 21:09:07 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Tue, 10 Nov 2020 22:09:07 +0100 (CET) Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> Message-ID: <13707283.2054368.1605042547841.JavaMail.zimbra@u-pem.fr> > De: "Chris Hegarty" > ?: "Remi Forax" > Cc: "amber-spec-experts" > Envoy?: Mardi 10 Novembre 2020 21:51:38 > Objet: Re: The Record Attribute - What does it mean to be a record at runtime? > Remi, > A point of clarification ( which I realise may have been ambiguous in my last > email ). >> On 10 Nov 2020, at 19:36, [ mailto:forax at univ-mlv.fr | forax at univ-mlv.fr ] >> wrote: >>> ... >>> A better option would be to update the VM implementation to only assert >>> that the fields of a record-like class are trusted if that class >>> contains 1) a structurally sound Record attribute, 2) is a direct >>> subclass of j.l.Record, 3) is final, and 4) is non-abstract. This would >>> align Core Reflection and the VM in this respect, while the JVMS would >>> remain unchanged - it's an implementation detail. >> No, it's pushing the JLS semantics into the JVM. >> What you want to know is if a field is trusted final or not, to disallow the >> creation of VarHandle, the use Unsafe, etc. >> So the JDK API should ask the VM if a field is trusted final or not instead of >> asking if the class is a record. > Agreed. The VM exposes a (private) interface to the core JDK libraries to tell > whether a field is trusted or not. No issue. ( I did not mean to say otherwise > ) > My issue is with how the VM determines whether a field is trusted or not. The VM > trusts fields in (among other types) ?record? classes. So what is a record > class to the VM? (that is the question that I am trying to resolve) - the > answer is not in the JVMS ( which is fine ). For me having a Record attribute is enough, i.e. the current definition of what a record is for the VM is enough. The problem is that if a class has to be a subclass of java.lang.Record, it means that any languages that want to use the attribute Record has to agree to the contract of j.l.Record, by example, j.l.Record has a precise definition of how to compare floating point numbers or the fact that there is an order in between the record components. Those kind of constraints are fine for Java the language but it's not something that the VM should enforce for any other languages. > -Chris. R?mi -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.buckley at oracle.com Tue Nov 10 21:37:04 2020 From: alex.buckley at oracle.com (Alex Buckley) Date: Tue, 10 Nov 2020 13:37:04 -0800 Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <13707283.2054368.1605042547841.JavaMail.zimbra@u-pem.fr> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> <13707283.2054368.1605042547841.JavaMail.zimbra@u-pem.fr> Message-ID: <9d42ee54-0fa7-e4a2-563a-a276458f04e8@oracle.com> On 11/10/2020 1:09 PM, forax at univ-mlv.fr wrote: > *De: *"Chris Hegarty" > *?: *"Remi Forax" > *Cc: *"amber-spec-experts" > *Envoy?: *Mardi 10 Novembre 2020 21:51:38 > *Objet: *Re: The Record Attribute - What does it mean to be a record > at runtime? > > Remi, > > A point of clarification ( which I realise may have been ambiguous > in my last email ). > > On 10 Nov 2020, at 19:36, forax at univ-mlv.fr > wrote: > > ... > > A better option would be to update the VM implementation to > only assert > that the fields of a record-like class are trusted if that class > contains 1) a structurally sound Record attribute, 2) is a > direct > subclass of j.l.Record, 3) is final, and 4) is non-abstract. > This would > align Core Reflection and the VM in this respect, while the > JVMS would > remain unchanged - it's an implementation detail. > > > No, it's pushing the JLS semantics into the JVM. > > What you want to know is if a field is trusted final or not, to > disallow the creation of VarHandle, the use Unsafe, etc. > So the JDK API should ask the VM if a field is trusted final or > not instead of asking if the class is a record. > > > Agreed. The VM exposes a (private) interface to the core JDK > libraries to tell whether a field is trusted or not. No issue. ?( I > did not mean to say otherwise ) > > My issue is with how the VM determines whether a field is trusted or > not. The VM trusts fields in (among other types) ?record? classes. > So what is a record class to the VM? ? (that is the question that I > am trying to resolve) - the answer is not in the JVMS ( which is fine ). > > > For me having a Record attribute is enough, i.e. the current definition > of what a record is for the VM is enough. > > The problem is that if a class has to be a subclass of java.lang.Record, > it means that any languages that want to use the attribute Record has to > agree to the contract of j.l.Record, > by example, j.l.Record has a precise definition of how to compare > floating point numbers or the fact that there is an order in between the > record components. > Those kind of constraints are fine for Java the language but it's not > something that the VM should enforce for any other languages. The JLS, and therefore the Java SE Platform that incorporates the JLS, defines what a record class is. A record class is a class that extends j.l.Record, has no non-static fields, has no native methods, etc. The Record attribute indicates that a class file wishes to be treated as a record class, so a compiler and the reflection libraries must hold a class file with a Record attribute to the standard expected of a record class (extends j.l.Record, etc). In principle, a JVM implementation could also have this responsibility, but in practice, no-one wants to do these exhaustive checks at class load time. So, a non-Java compiler can happily spin a class file with a Record attribute and no j.l.Record superclass, but it doesn't represent a record class and the reflection libraries should not say it does, hence no "trusted" final fields. Alex From forax at univ-mlv.fr Tue Nov 10 21:46:19 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 10 Nov 2020 22:46:19 +0100 (CET) Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <9d42ee54-0fa7-e4a2-563a-a276458f04e8@oracle.com> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> <13707283.2054368.1605042547841.JavaMail.zimbra@u-pem.fr> <9d42ee54-0fa7-e4a2-563a-a276458f04e8@oracle.com> Message-ID: <1663515390.2080107.1605044779508.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Alex Buckley" > ?: "amber-spec-experts" > Envoy?: Mardi 10 Novembre 2020 22:37:04 > Objet: Re: The Record Attribute - What does it mean to be a record at runtime? > On 11/10/2020 1:09 PM, forax at univ-mlv.fr wrote: >> *De: *"Chris Hegarty" >> *?: *"Remi Forax" >> *Cc: *"amber-spec-experts" >> *Envoy?: *Mardi 10 Novembre 2020 21:51:38 >> *Objet: *Re: The Record Attribute - What does it mean to be a record >> at runtime? >> >> Remi, >> >> A point of clarification ( which I realise may have been ambiguous >> in my last email ). >> >> On 10 Nov 2020, at 19:36, forax at univ-mlv.fr >> wrote: >> >> ... >> >> A better option would be to update the VM implementation to >> only assert >> that the fields of a record-like class are trusted if that class >> contains 1) a structurally sound Record attribute, 2) is a >> direct >> subclass of j.l.Record, 3) is final, and 4) is non-abstract. >> This would >> align Core Reflection and the VM in this respect, while the >> JVMS would >> remain unchanged - it's an implementation detail. >> >> >> No, it's pushing the JLS semantics into the JVM. >> >> What you want to know is if a field is trusted final or not, to >> disallow the creation of VarHandle, the use Unsafe, etc. >> So the JDK API should ask the VM if a field is trusted final or >> not instead of asking if the class is a record. >> >> >> Agreed. The VM exposes a (private) interface to the core JDK >> libraries to tell whether a field is trusted or not. No issue. ?( I >> did not mean to say otherwise ) >> >> My issue is with how the VM determines whether a field is trusted or >> not. The VM trusts fields in (among other types) ?record? classes. >> So what is a record class to the VM? ? (that is the question that I >> am trying to resolve) - the answer is not in the JVMS ( which is fine ). >> >> >> For me having a Record attribute is enough, i.e. the current definition >> of what a record is for the VM is enough. >> >> The problem is that if a class has to be a subclass of java.lang.Record, >> it means that any languages that want to use the attribute Record has to >> agree to the contract of j.l.Record, >> by example, j.l.Record has a precise definition of how to compare >> floating point numbers or the fact that there is an order in between the >> record components. >> Those kind of constraints are fine for Java the language but it's not >> something that the VM should enforce for any other languages. > > The JLS, and therefore the Java SE Platform that incorporates the JLS, > defines what a record class is. A record class is a class that extends > j.l.Record, has no non-static fields, has no native methods, etc. The > Record attribute indicates that a class file wishes to be treated as a > record class, so a compiler and the reflection libraries must hold a > class file with a Record attribute to the standard expected of a record > class (extends j.l.Record, etc). In principle, a JVM implementation > could also have this responsibility, but in practice, no-one wants to do > these exhaustive checks at class load time. So, a non-Java compiler can > happily spin a class file with a Record attribute and no j.l.Record > superclass, but it doesn't represent a record class and the reflection > libraries should not say it does, hence no "trusted" final fields. I don't disagree if the notion of trusted fields is created in the reflection API, because currently, it's just a VM thing, not a reflection thing. > > Alex R?mi From daniel.smith at oracle.com Wed Nov 11 02:39:18 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Tue, 10 Nov 2020 19:39:18 -0700 Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> Message-ID: <5B61751C-9FEF-4760-8525-BE34DF4AAC88@oracle.com> > On Nov 10, 2020, at 1:51 PM, Chris Hegarty wrote: > > My issue is with how the VM determines whether a field is trusted or not. The VM trusts fields in (among other types) ?record? classes. So what is a record class to the VM? (that is the question that I am trying to resolve) - the answer is not in the JVMS ( which is fine ). Suggestion: the VM shouldn't bother to provide any definition for "record class". That's up to Java language compilers and reflection (which should be in agreement). Instead, can't we say the VM trusts fields in classes that have a Record attribute? Who cares whether those classes are "real" records or not? (I may be missing something because I don't fully understand the concept of "trusted final field", but it seems to me like HotSpot has a lot of freedom to decide to trust or not trust whatever fields it wants.) From maurizio.cimadamore at oracle.com Thu Nov 12 21:15:05 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 12 Nov 2020 21:15:05 +0000 Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <5B61751C-9FEF-4760-8525-BE34DF4AAC88@oracle.com> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> <5B61751C-9FEF-4760-8525-BE34DF4AAC88@oracle.com> Message-ID: <9cf0f7fd-5570-7fea-af54-35e29e87e3af@oracle.com> I think the point Chris is raising is a bit more subtle. The VM has a notion of classes in which final fields are trusted - when that happens, the value of final fields are constant folded by the JIT (yayy!). There is a problem though: in practice, final fields can be mutated e.g. by reflection, or via 292 (var handles) or via Unsafe. For this reason, when we trust final fields in class C, we typically do other changes to prevent fields in class C from being exposed (e.g. via reflection), so as to decrease the potential that JIT optimization would be invalidated, either accidentally or maliciously. I think the current state of affair for records has at least 3 issues: * The paths which controls which fields are trusted by VM and which fields are exposed via reflection/292/unsafe do not align 100%. For instance, Unsafe refuses to give up field offset based on the stricter Class::isRecord definition, thus creating a safety gap. * In an attempt to prevent changes to records fields via reflection, the specification of Field::set was updated to say that if the field holder is a record, the field update will fail. This is good, if it weren't that the javadoc has to define what it means for a field holder to be a record; and the way that's done is by referring to Class::isRecord, which is not what the implementation does (the implementation uses the broader "isRecord" definition based on the presence/absence of the record attribute - the same used by the VM). * All clients have now a way to enable the (fragile) final field optimization: generate a class, then attach a record attribute to it (e.g. with ASM). The VM will trust all fields in that class no matter what. I think this is probably not what was originally intended by the changes in JDK-8247444. In other words, we now have a mechanism which is kind of like the internal @ForceInline annotation - not as handy as an annotation perhaps, but much more public (because, any classfile can be augmented to contain an extra, maybe empty, record attribute). So, the way I see it, is that there are few ways out of this conundrum: 1) Back away from final field optimization for records; we could in fact get rid of all the asymmetries described above by simply _not_ trusting record fields - at which point VM support would become again very minimal. 2) Enforce a definition of "is a record" which makes sense - and then enable extra optimizations on top. Since we need to be able to specific reflection restriction (Field::set) I think the only defensible definition for what constitutes a record is to delegate to Class::isRecord. This doesn't mean that the JVM will have to start taking into account the JLS definition of a record at verification time - it simply means that the final field optimization, instead of being gated on the mere _presence_ of the record attribute, it would be governed by a more complex definition which matches the behavior of Class::isRecord. Note that this is all below the JVMS surface. What Chris is saying is (I think) that the current implementation is caught in some inconsistent place between (1) and (2) - a place where, on one hand, we claim VM neutrality w.r.t. the record definition contained in the JLS but where, at the same time, we apply extra optimizations to speed up access to record fields. Maurizio On 11/11/2020 02:39, Dan Smith wrote: >> On Nov 10, 2020, at 1:51 PM, Chris Hegarty wrote: >> >> My issue is with how the VM determines whether a field is trusted or not. The VM trusts fields in (among other types) ?record? classes. So what is a record class to the VM? (that is the question that I am trying to resolve) - the answer is not in the JVMS ( which is fine ). > Suggestion: the VM shouldn't bother to provide any definition for "record class". That's up to Java language compilers and reflection (which should be in agreement). > > Instead, can't we say the VM trusts fields in classes that have a Record attribute? Who cares whether those classes are "real" records or not? (I may be missing something because I don't fully understand the concept of "trusted final field", but it seems to me like HotSpot has a lot of freedom to decide to trust or not trust whatever fields it wants.) > From forax at univ-mlv.fr Thu Nov 12 21:41:33 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Thu, 12 Nov 2020 22:41:33 +0100 (CET) Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <9cf0f7fd-5570-7fea-af54-35e29e87e3af@oracle.com> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> <5B61751C-9FEF-4760-8525-BE34DF4AAC88@oracle.com> <9cf0f7fd-5570-7fea-af54-35e29e87e3af@oracle.com> Message-ID: <318528656.825731.1605217293068.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Maurizio Cimadamore" > ?: "daniel smith" , "Chris Hegarty" > Cc: "Remi Forax" , "amber-spec-experts" > Envoy?: Jeudi 12 Novembre 2020 22:15:05 > Objet: Re: The Record Attribute - What does it mean to be a record at runtime? > I think the point Chris is raising is a bit more subtle. > > The VM has a notion of classes in which final fields are trusted - when > that happens, the value of final fields are constant folded by the JIT > (yayy!). > > There is a problem though: in practice, final fields can be mutated e.g. > by reflection, or via 292 (var handles) or via Unsafe. > > For this reason, when we trust final fields in class C, we typically do > other changes to prevent fields in class C from being exposed (e.g. via > reflection), so as to decrease the potential that JIT optimization would > be invalidated, either accidentally or maliciously. > > I think the current state of affair for records has at least 3 issues: > > * The paths which controls which fields are trusted by VM and which > fields are exposed via reflection/292/unsafe do not align 100%. For > instance, Unsafe refuses to give up field offset based on the stricter > Class::isRecord definition, thus creating a safety gap. > > * In an attempt to prevent changes to records fields via reflection, the > specification of Field::set was updated to say that if the field holder > is a record, the field update will fail. This is good, if it weren't > that the javadoc has to define what it means for a field holder to be a > record; and the way that's done is by referring to Class::isRecord, > which is not what the implementation does (the implementation uses the > broader "isRecord" definition based on the presence/absence of the > record attribute - the same used by the VM). > > * All clients have now a way to enable the (fragile) final field > optimization: generate a class, then attach a record attribute to it > (e.g. with ASM). The VM will trust all fields in that class no matter > what. I think this is probably not what was originally intended by the > changes in JDK-8247444. In other words, we now have a mechanism which is > kind of like the internal @ForceInline annotation - not as handy as an > annotation perhaps, but much more public (because, any classfile can be > augmented to contain an extra, maybe empty, record attribute). > > So, the way I see it, is that there are few ways out of this conundrum: > > 1) Back away from final field optimization for records; we could in fact > get rid of all the asymmetries described above by simply _not_ trusting > record fields - at which point VM support would become again very minimal. > > 2) Enforce a definition of "is a record" which makes sense - and then > enable extra optimizations on top. Since we need to be able to specific > reflection restriction (Field::set) I think the only defensible > definition for what constitutes a record is to delegate to > Class::isRecord. This doesn't mean that the JVM will have to start > taking into account the JLS definition of a record at verification time > - it simply means that the final field optimization, instead of being > gated on the mere _presence_ of the record attribute, it would be > governed by a more complex definition which matches the behavior of > Class::isRecord. Note that this is all below the JVMS surface. > > What Chris is saying is (I think) that the current implementation is > caught in some inconsistent place between (1) and (2) - a place where, > on one hand, we claim VM neutrality w.r.t. the record definition > contained in the JLS but where, at the same time, we apply extra > optimizations to speed up access to record fields. For me, you are focusing on the wrong side of the time line, trusted final fields should be the new normal and plain old classes are the ones that are weird because they have a *weak* final fields semantics. So instead of saying that records, hidden classes, anonymous classes (and soon primitive object classes) are special, the VM should say that plain old classes are the outlier. With that, the JLS definition of a record doesn't need to leak into the JVMS. > > Maurizio R?mi > > On 11/11/2020 02:39, Dan Smith wrote: >>> On Nov 10, 2020, at 1:51 PM, Chris Hegarty wrote: >>> >>> My issue is with how the VM determines whether a field is trusted or not. The VM >>> trusts fields in (among other types) ?record? classes. So what is a record >>> class to the VM? (that is the question that I am trying to resolve) - the >>> answer is not in the JVMS ( which is fine ). >> Suggestion: the VM shouldn't bother to provide any definition for "record >> class". That's up to Java language compilers and reflection (which should be in >> agreement). >> >> Instead, can't we say the VM trusts fields in classes that have a Record >> attribute? Who cares whether those classes are "real" records or not? (I may be >> missing something because I don't fully understand the concept of "trusted >> final field", but it seems to me like HotSpot has a lot of freedom to decide to >> trust or not trust whatever fields it wants.) From maurizio.cimadamore at oracle.com Fri Nov 13 00:18:01 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 13 Nov 2020 00:18:01 +0000 Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <318528656.825731.1605217293068.JavaMail.zimbra@u-pem.fr> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> <5B61751C-9FEF-4760-8525-BE34DF4AAC88@oracle.com> <9cf0f7fd-5570-7fea-af54-35e29e87e3af@oracle.com> <318528656.825731.1605217293068.JavaMail.zimbra@u-pem.fr> Message-ID: <71a40023-d45c-16b1-c369-40e88841da18@oracle.com> > For me, you are focusing on the wrong side of the time line, > trusted final fields should be the new normal and plain old classes are the ones that are weird because they have a *weak* final fields semantics. > > So instead of saying that records, hidden classes, anonymous classes (and soon primitive object classes) are special, the VM should say that plain old classes are the outlier. In an ideal world, yes, although I don't think wishing that does anything for our current issue(s) with records. Btw, for the records, the work described here: https://bugs.openjdk.java.net/browse/JDK-8233873 iis what's required to enable final field optimization _safely_ (stress this word many times, please) for _all_ classes. But that's besides the point. The point in my email is that just attaching an attribute (the record attribute) to a class doesn't magically remove the safety concerns associated with trusting final fields (some of which are captured in the above RFE). With the current mechanism for trusted final fields we still have to double check that trusted finals cannot be updated. So there has to be a symmetry between the logic granting the trusts and the logic which keeps the platform on a safe ground by rejecting final fields updates. In the current situation, I'm not sure the symmetry is there (at least not in all the code paths), and the logic to prevent reflective field update is borderline unspecifiable (*) and, even w/o considering that, inconsistent with what the implementation does, so it should be fixed one way or another. Maurizio (*) Field::set doesn't fail when Class::isRecord returns true (which is what we currently specify) but when the class the field belongs to has the Record attribute. Should we add that to the javadoc? > > With that, the JLS definition of a record doesn't need to leak into the JVMS. > >> Maurizio > R?mi > >> On 11/11/2020 02:39, Dan Smith wrote: >>>> On Nov 10, 2020, at 1:51 PM, Chris Hegarty wrote: >>>> >>>> My issue is with how the VM determines whether a field is trusted or not. The VM >>>> trusts fields in (among other types) ?record? classes. So what is a record >>>> class to the VM? (that is the question that I am trying to resolve) - the >>>> answer is not in the JVMS ( which is fine ). >>> Suggestion: the VM shouldn't bother to provide any definition for "record >>> class". That's up to Java language compilers and reflection (which should be in >>> agreement). >>> >>> Instead, can't we say the VM trusts fields in classes that have a Record >>> attribute? Who cares whether those classes are "real" records or not? (I may be >>> missing something because I don't fully understand the concept of "trusted >>> final field", but it seems to me like HotSpot has a lot of freedom to decide to >>> trust or not trust whatever fields it wants.) From daniel.smith at oracle.com Fri Nov 13 04:34:51 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Thu, 12 Nov 2020 21:34:51 -0700 Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <9cf0f7fd-5570-7fea-af54-35e29e87e3af@oracle.com> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> <5B61751C-9FEF-4760-8525-BE34DF4AAC88@oracle.com> <9cf0f7fd-5570-7fea-af54-35e29e87e3af@oracle.com> Message-ID: <0FE05432-CB74-416B-9A73-D3D6F40F85C3@oracle.com> > On Nov 12, 2020, at 2:15 PM, Maurizio Cimadamore wrote: > > I think the current state of affair for records has at least 3 issues: > > * The paths which controls which fields are trusted by VM and which fields are exposed via reflection/292/unsafe do not align 100%. For instance, Unsafe refuses to give up field offset based on the stricter Class::isRecord definition, thus creating a safety gap. Agree that's a problem. All the components involved in trusted fields need to be in agreement about which final fields are trusted. > * In an attempt to prevent changes to records fields via reflection, the specification of Field::set was updated to say that if the field holder is a record, the field update will fail. This is good, if it weren't that the javadoc has to define what it means for a field holder to be a record; and the way that's done is by referring to Class::isRecord, which is not what the implementation does (the implementation uses the broader "isRecord" definition based on the presence/absence of the record attribute - the same used by the VM). Agree the spec and implementation should be consistent. That doesn't necessarily mean they need to use Class.isRecord as their test. It could just as easily be checking for the attribute, or testing that the class extends java.lang.Record, or maybe something else. There are multiple places to draw the line. But all I'm saying is that there are multiple possibilities. While I think having a super-simple test might be attractive, I'm not in a position to weigh that against other concerns. > * All clients have now a way to enable the (fragile) final field optimization: generate a class, then attach a record attribute to it (e.g. with ASM). The VM will trust all fields in that class no matter what. I think this is probably not what was originally intended by the changes in JDK-8247444. In other words, we now have a mechanism which is kind of like the internal @ForceInline annotation - not as handy as an annotation perhaps, but much more public (because, any classfile can be augmented to contain an extra, maybe empty, record attribute). Right. If there's a concern that *too many* classes will have trusted fields, then drawing the line more narrowly is useful. It's possible?I'm not sure?that saying "all subclasses of java.lang.Record" would raise fewer concerns here than "all classes with a Record attribute". Not because it's necessarily harder to physically put in the class file, but because extending a class is a strong signal that you intend to adopt any special contracts/treatment associated with that class. From forax at univ-mlv.fr Fri Nov 13 10:15:11 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 13 Nov 2020 11:15:11 +0100 (CET) Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <71a40023-d45c-16b1-c369-40e88841da18@oracle.com> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> <5B61751C-9FEF-4760-8525-BE34DF4AAC88@oracle.com> <9cf0f7fd-5570-7fea-af54-35e29e87e3af@oracle.com> <318528656.825731.1605217293068.JavaMail.zimbra@u-pem.fr> <71a40023-d45c-16b1-c369-40e88841da18@oracle.com> Message-ID: <968236546.1112230.1605262511263.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Maurizio Cimadamore" > ?: "Remi Forax" > Cc: "daniel smith" , "Chris Hegarty" , "amber-spec-experts" > > Envoy?: Vendredi 13 Novembre 2020 01:18:01 > Objet: Re: The Record Attribute - What does it mean to be a record at runtime? >> For me, you are focusing on the wrong side of the time line, >> trusted final fields should be the new normal and plain old classes are the ones >> that are weird because they have a *weak* final fields semantics. >> >> So instead of saying that records, hidden classes, anonymous classes (and soon >> primitive object classes) are special, the VM should say that plain old classes >> are the outlier. > > In an ideal world, yes, although I don't think wishing that does > anything for our current issue(s) with records. > > Btw, for the records, the work described here: > > https://bugs.openjdk.java.net/browse/JDK-8233873 > > iis what's required to enable final field optimization _safely_ (stress > this word many times, please) for _all_ classes. But that's besides the > point. > > The point in my email is that just attaching an attribute (the record > attribute) to a class doesn't magically remove the safety concerns > associated with trusting final fields (some of which are captured in the > above RFE). With the current mechanism for trusted final fields we still > have to double check that trusted finals cannot be updated. So there has > to be a symmetry between the logic granting the trusts and the logic > which keeps the platform on a safe ground by rejecting final fields updates. > > In the current situation, I'm not sure the symmetry is there (at least > not in all the code paths), and the logic to prevent reflective field > update is borderline unspecifiable (*) and, even w/o considering that, > inconsistent with what the implementation does, so it should be fixed > one way or another. I fully agree that the codes in Field.set(), lookup.findSetter/unreflectFiedl() have to be fixed, i disagree that this is a problem related to records. For enums, the VM and the reflection disagree about what an enum is, and that's not an issue. (in fact, isEnum() et getEnumConstants() also disagree of what an enum is). The fact that there is no record access bit in the header of a class is to avoid to use one of these precious bits. So from the VM POV, a class is a record because it has the Record attribute. As i told Chris, i believe that the best to fix this issue is to have the VM to export a method boolean hasWeakFinalFieldSemantics(Class) that says if a class allows final fields to be set or not. And to fix the spec of Field.set(), lookup.findSetter/unreflectFiedl() to say that only plain old classes and enums allow their final fields to be set. > > Maurizio > > (*) Field::set doesn't fail when Class::isRecord returns true (which is > what we currently specify) but when the class the field belongs to has > the Record attribute. Should we add that to the javadoc ? that only POJO and enums final fields can be set. R?mi > >> >> With that, the JLS definition of a record doesn't need to leak into the JVMS. >> >>> Maurizio >> R?mi >> >>> On 11/11/2020 02:39, Dan Smith wrote: >>>>> On Nov 10, 2020, at 1:51 PM, Chris Hegarty wrote: >>>>> >>>>> My issue is with how the VM determines whether a field is trusted or not. The VM >>>>> trusts fields in (among other types) ?record? classes. So what is a record >>>>> class to the VM? (that is the question that I am trying to resolve) - the >>>>> answer is not in the JVMS ( which is fine ). >>>> Suggestion: the VM shouldn't bother to provide any definition for "record >>>> class". That's up to Java language compilers and reflection (which should be in >>>> agreement). >>>> >>>> Instead, can't we say the VM trusts fields in classes that have a Record >>>> attribute? Who cares whether those classes are "real" records or not? (I may be >>>> missing something because I don't fully understand the concept of "trusted >>>> final field", but it seems to me like HotSpot has a lot of freedom to decide to > >>> trust or not trust whatever fields it wants.) From chris.hegarty at oracle.com Fri Nov 13 14:41:55 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 13 Nov 2020 14:41:55 +0000 Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <968236546.1112230.1605262511263.JavaMail.zimbra@u-pem.fr> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> <5B61751C-9FEF-4760-8525-BE34DF4AAC88@oracle.com> <9cf0f7fd-5570-7fea-af54-35e29e87e3af@oracle.com> <318528656.825731.1605217293068.JavaMail.zimbra@u-pem.fr> <71a40023-d45c-16b1-c369-40e88841da18@oracle.com> <968236546.1112230.1605262511263.JavaMail.zimbra@u-pem.fr> Message-ID: <6E8B122A-F957-4651-8109-04587D4AEEF0@oracle.com> > On 13 Nov 2020, at 10:15, forax at univ-mlv.fr wrote: > > ----- Mail original ----- >> De: "Maurizio Cimadamore" >> ?: "Remi Forax" >> Cc: "daniel smith" , "Chris Hegarty" , "amber-spec-experts" >> >> Envoy?: Vendredi 13 Novembre 2020 01:18:01 >> Objet: Re: The Record Attribute - What does it mean to be a record at runtime? > >>> ... > > I fully agree that the codes in Field.set(), lookup.findSetter/unreflectFiedl() have to be fixed, We seem to have arrived at a common understanding that there is at least a problem to be fixed - good ;-) > i disagree that this is a problem related to records. When the Hotspot VM was changed to trust non-static final fields in ?records? [1], there was only one notion of what that meant - it was the full JLS definition of a record class (since the Hotspot VM would throw CFE for classes with the record attribute while not extending j.l.Record, or non-final). The Hotspot VM implementation subsequently changed [2]. Given that the landscape has changed, it seems right and proper to revisit the decision to trust non-static final fields in ?records? - or, at least, the set of circumstances upon which to base that trust. I will add an additional piece of information that I realise hasn?t come up yet. I added the check for ?direct superclass is j.l.Record? to Class::isRecord because of a discomfort that arose during one of the records CSR?s (details not to hand), since the JVMS does no include such. At the time, given the Hotspot VM implementation, this was the minimum acceptable solution that we arrived at, so as to keep the Hotspot VM and other parts of the runtime in agreement. > For enums, the VM and the reflection disagree about what an enum is, and that's not an issue. > (in fact, isEnum() et getEnumConstants() also disagree of what an enum is). I respectfully disagree, but there is likely nothing to be gained by trashing this out here, other than to say that I don?t accept this in any way as precedent, or good design to follow. > The fact that there is no record access bit in the header of a class is to avoid to use one of these precious bits. > So from the VM POV, a class is a record because it has the Record attribute. Well, as discussion progresses, this seems to be crux upon which the discussion hinges - which is where I started :-) > As i told Chris, i believe that the best to fix this issue is to have the VM to export a method boolean hasWeakFinalFieldSemantics(Class) that says if a class allows final fields to be set or not. > And to fix the spec of Field.set(), lookup.findSetter/unreflectFiedl() to say that only plain old classes and enums allow their final fields to be set. You did say this, but I did not ( and still do not ) accept that this is the right way to solve the problem. >> .. >> (*) Field::set doesn't fail when Class::isRecord returns true (which is >> what we currently specify) but when the class the field belongs to has >> the Record attribute. Should we add that to the javadoc ? > > that only POJO and enums final fields can be set. That is not a reasonable assertion to make in the Java SE Specification. -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8247444 [2] https://bugs.openjdk.java.net/browse/JDK-8255342 > R?mi > >> >>> >>> With that, the JLS definition of a record doesn't need to leak into the JVMS. >>> >>>> Maurizio >>> R?mi >>> >>>> On 11/11/2020 02:39, Dan Smith wrote: >>>>>> On Nov 10, 2020, at 1:51 PM, Chris Hegarty wrote: >>>>>> >>>>>> My issue is with how the VM determines whether a field is trusted or not. The VM >>>>>> trusts fields in (among other types) ?record? classes. So what is a record >>>>>> class to the VM? (that is the question that I am trying to resolve) - the >>>>>> answer is not in the JVMS ( which is fine ). >>>>> Suggestion: the VM shouldn't bother to provide any definition for "record >>>>> class". That's up to Java language compilers and reflection (which should be in >>>>> agreement). >>>>> >>>>> Instead, can't we say the VM trusts fields in classes that have a Record >>>>> attribute? Who cares whether those classes are "real" records or not? (I may be >>>>> missing something because I don't fully understand the concept of "trusted >>>>> final field", but it seems to me like HotSpot has a lot of freedom to decide to >>>>> trust or not trust whatever fields it wants.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri Nov 13 15:51:39 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 13 Nov 2020 10:51:39 -0500 Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <0FE05432-CB74-416B-9A73-D3D6F40F85C3@oracle.com> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> <5B61751C-9FEF-4760-8525-BE34DF4AAC88@oracle.com> <9cf0f7fd-5570-7fea-af54-35e29e87e3af@oracle.com> <0FE05432-CB74-416B-9A73-D3D6F40F85C3@oracle.com> Message-ID: <818572ba-9975-ceda-b151-a8e2be3008d4@oracle.com> Taking a step back here ... I agree with pretty much everything Maurizio says about the interaction of records and trusted fields. But, I think we should also recognize that we fell, somewhat, for a siren song.? We all hate the fact that final fields are not really final.? So when another opportunity came along to strike a blow for a rational memory model, we were all over it. But perhaps we swung at the wrong pitch. It is easy to agree "yes, those fields should be trusted."? But the amount of work we had to do to support just this one case -- including JIT, reflection, unsafe, and var handles -- was already a lot.? Then we found out that we didn't even do it right, and we had to have this discussion about how to remediate it, and more work to fix it.? Reclaiming a sane memory model 1% at a time does not seem to be an expensive and slow path. While it is easy to say, Remi's comment that we are focusing on the wrong side of the timeline seems the right way to start this conversation -- how do we get to the point where it is simple to add new classes to the set which are immune to to such shenanigans. In part, that means gathering some requirements as to when to _not_ trust final fields.? Has anyone characterized the space where this is required? On 11/12/2020 11:34 PM, Dan Smith wrote: >> On Nov 12, 2020, at 2:15 PM, Maurizio Cimadamore wrote: >> >> I think the current state of affair for records has at least 3 issues: >> >> * The paths which controls which fields are trusted by VM and which fields are exposed via reflection/292/unsafe do not align 100%. For instance, Unsafe refuses to give up field offset based on the stricter Class::isRecord definition, thus creating a safety gap. > Agree that's a problem. All the components involved in trusted fields need to be in agreement about which final fields are trusted. > >> * In an attempt to prevent changes to records fields via reflection, the specification of Field::set was updated to say that if the field holder is a record, the field update will fail. This is good, if it weren't that the javadoc has to define what it means for a field holder to be a record; and the way that's done is by referring to Class::isRecord, which is not what the implementation does (the implementation uses the broader "isRecord" definition based on the presence/absence of the record attribute - the same used by the VM). > Agree the spec and implementation should be consistent. That doesn't necessarily mean they need to use Class.isRecord as their test. It could just as easily be checking for the attribute, or testing that the class extends java.lang.Record, or maybe something else. There are multiple places to draw the line. > > But all I'm saying is that there are multiple possibilities. While I think having a super-simple test might be attractive, I'm not in a position to weigh that against other concerns. > >> * All clients have now a way to enable the (fragile) final field optimization: generate a class, then attach a record attribute to it (e.g. with ASM). The VM will trust all fields in that class no matter what. I think this is probably not what was originally intended by the changes in JDK-8247444. In other words, we now have a mechanism which is kind of like the internal @ForceInline annotation - not as handy as an annotation perhaps, but much more public (because, any classfile can be augmented to contain an extra, maybe empty, record attribute). > Right. If there's a concern that *too many* classes will have trusted fields, then drawing the line more narrowly is useful. It's possible?I'm not sure?that saying "all subclasses of java.lang.Record" would raise fewer concerns here than "all classes with a Record attribute". Not because it's necessarily harder to physically put in the class file, but because extending a class is a strong signal that you intend to adopt any special contracts/treatment associated with that class. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From elizarov at gmail.com Sat Nov 14 15:28:04 2020 From: elizarov at gmail.com (Roman Elizarov) Date: Sat, 14 Nov 2020 18:28:04 +0300 Subject: The Record Attribute - What does it mean to be a record at runtime? In-Reply-To: <818572ba-9975-ceda-b151-a8e2be3008d4@oracle.com> References: <2AC4024E-876C-4739-8D1E-0DC433B724B4@oracle.com> <2023002252.677642.1604620767748.JavaMail.zimbra@u-pem.fr> <1944953924.2033713.1605036977888.JavaMail.zimbra@u-pem.fr> <5B61751C-9FEF-4760-8525-BE34DF4AAC88@oracle.com> <9cf0f7fd-5570-7fea-af54-35e29e87e3af@oracle.com> <0FE05432-CB74-416B-9A73-D3D6F40F85C3@oracle.com> <818572ba-9975-ceda-b151-a8e2be3008d4@oracle.com> Message-ID: Brian, I really like where this is going. Our software is full of totally encapsulated (thus, non-record) and yet truly immutable classes that would greatly benefit from additional optimizations that are possible if final fields are trusted. If it turns out to be infeasible to figure out which final fields can/cannot be trusted in the legacy code, I really hope that, at least, we will have an option to designate newly developed classes with final fields to be trusted in a way that is not tied to records. I believe that a significant fraction of Java community can share similar accounts. Sincerely, Roman Elizarov On Fri, Nov 13, 2020 at 6:51 PM Brian Goetz wrote: > Taking a step back here ... > > I agree with pretty much everything Maurizio says about the interaction of > records and trusted fields. > > But, I think we should also recognize that we fell, somewhat, for a siren > song. We all hate the fact that final fields are not really final. So > when another opportunity came along to strike a blow for a rational memory > model, we were all over it. But perhaps we swung at the wrong pitch. > > It is easy to agree "yes, those fields should be trusted." But the amount > of work we had to do to support just this one case -- including JIT, > reflection, unsafe, and var handles -- was already a lot. Then we found > out that we didn't even do it right, and we had to have this discussion > about how to remediate it, and more work to fix it. Reclaiming a sane > memory model 1% at a time does not seem to be an expensive and slow path. > > While it is easy to say, Remi's comment that we are focusing on the wrong > side of the timeline seems the right way to start this conversation -- how > do we get to the point where it is simple to add new classes to the set > which are immune to to such shenanigans. > > In part, that means gathering some requirements as to when to _not_ trust > final fields. Has anyone characterized the space where this is required? > > > > On 11/12/2020 11:34 PM, Dan Smith wrote: > > On Nov 12, 2020, at 2:15 PM, Maurizio Cimadamore wrote: > > I think the current state of affair for records has at least 3 issues: > > * The paths which controls which fields are trusted by VM and which fields are exposed via reflection/292/unsafe do not align 100%. For instance, Unsafe refuses to give up field offset based on the stricter Class::isRecord definition, thus creating a safety gap. > > Agree that's a problem. All the components involved in trusted fields need to be in agreement about which final fields are trusted. > > > * In an attempt to prevent changes to records fields via reflection, the specification of Field::set was updated to say that if the field holder is a record, the field update will fail. This is good, if it weren't that the javadoc has to define what it means for a field holder to be a record; and the way that's done is by referring to Class::isRecord, which is not what the implementation does (the implementation uses the broader "isRecord" definition based on the presence/absence of the record attribute - the same used by the VM). > > Agree the spec and implementation should be consistent. That doesn't necessarily mean they need to use Class.isRecord as their test. It could just as easily be checking for the attribute, or testing that the class extends java.lang.Record, or maybe something else. There are multiple places to draw the line. > > But all I'm saying is that there are multiple possibilities. While I think having a super-simple test might be attractive, I'm not in a position to weigh that against other concerns. > > > * All clients have now a way to enable the (fragile) final field optimization: generate a class, then attach a record attribute to it (e.g. with ASM). The VM will trust all fields in that class no matter what. I think this is probably not what was originally intended by the changes in JDK-8247444. In other words, we now have a mechanism which is kind of like the internal @ForceInline annotation - not as handy as an annotation perhaps, but much more public (because, any classfile can be augmented to contain an extra, maybe empty, record attribute). > > Right. If there's a concern that *too many* classes will have trusted fields, then drawing the line more narrowly is useful. It's possible?I'm not sure?that saying "all subclasses of java.lang.Record" would raise fewer concerns here than "all classes with a Record attribute". Not because it's necessarily harder to physically put in the class file, but because extending a class is a strong signal that you intend to adopt any special contracts/treatment associated with that class. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Sat Nov 14 22:54:05 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 14 Nov 2020 23:54:05 +0100 (CET) Subject: Adding Do notaion/For comprehension In-Reply-To: <876231605390259@mail.yandex.ru> References: <876231605390259@mail.yandex.ru> Message-ID: <1878590279.1942796.1605394445183.JavaMail.zimbra@u-pem.fr> Wrong mailing list. > De: "Nikita Eshkeev" > ?: "compiler-dev" > Envoy?: Samedi 14 Novembre 2020 22:58:53 > Objet: Adding Do notaion/For comprehension > Hi, > What do you think about adding a language feature that is similar to do-notation > in Haskell or for-comprehension in Scala to make it easier to work with nested > flatMap? Language construct like for(:) or try(...) {} use an interface as type support, so even if we wanted to add that feature, we first need to be able to abstract Optional or Stream to a common interface containing flatMap(), something we can not do in Java, you have to put the type system on steroid first. > In java 5 we extended the capabilities of the for loop to iterate over types > that implement Iterable now I propose to add a language feature that will > simplify the code with nested flatMap, for example, the following code: > Optional helloWorld = Optional.of("Hello") > .flatMap(hello -> Optional.of("World") > .flatMap(world -> Optional.of(hello + ", " + world + "!"))) > would be transformable a more cleaner version: > Optional helloWorld = for { > hello <- Optional.of("Hello"); > world <- Optional.of("World"); > Optional.of(hello + ", " + world + "!") > } > *Here the word "for" is used as an example, it can be something different. A more Java like syntax should be something like Optional helloWorld = do(var hello = Optional.of("Hello"); var world = Optional.of("World")) { yield Optional.of(hello + ", " + world + " !"; }; > I would like to hear your opinion on it. I'm not seeing flatMap() being used often enough in Java code to the point it requires it own syntax. It's conceptually cool to see that flatMap() on a Stream and a loop are two sides of the same coin, but is it that useful ? Also, in the case of Optional, we may also be able to express the same snippet using pattern matching Optional helloWorld = switch(Optional.of("Hello"), Optional.of("World")) { case (Optional.of(var hello), Optional.of(var world)) -> Optional.of(hello + ", " + world + " !"); default -> Optional.empty(); }; and even Optional helloWorld = switch(Optional.of("Hello"), Optional.of("World")) { case (of(var hello), of(var world)) -> Optional.of(hello + ", " + world + " !"); default -> Optional.empty(); }; If we decide that like you don't have to specify the type of the enum when you match an enum constant, you don't have to specify the type when you match using a "named" deconstructor. > Sincerely, > Nikita Eshkeev R?mi -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Tue Nov 17 14:03:39 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 17 Nov 2020 14:03:39 +0000 Subject: [statics] allowing static initializers in interfaces? Message-ID: <45928680-f4a2-d21e-665c-19de2d87e2c0@oracle.com> Hi, now that the work on allowing static members in nested declarations [1] is wrapping up, I'm wondering if we could add treatment for one more case: static initializers in interfaces. This seems a case where the static compiler is already inserting static blocks (to initialize any static fields an interface might declare), but where the JLS is currently forbidding interfaces from having an explicit static initializer block in the source code. This make e.g. initialization of fields which might throw exceptions (e.g. method handles) not possible inside interfaces. While I realize this is a corner case, I thought it would still be worth asking the question :-) Cheers Maurizio [1] - https://git.openjdk.java.net/jdk/pull/571 From brian.goetz at oracle.com Tue Nov 17 14:08:20 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 17 Nov 2020 09:08:20 -0500 Subject: [statics] allowing static initializers in interfaces? In-Reply-To: <45928680-f4a2-d21e-665c-19de2d87e2c0@oracle.com> References: <45928680-f4a2-d21e-665c-19de2d87e2c0@oracle.com> Message-ID: <4b047622-1fd9-b424-ea66-d609f87dc11a@oracle.com> The goal of this effort is to make nesting of static members more uniform by eliminating such "gratuitous" restrictions, so this seems within the spirit. On 11/17/2020 9:03 AM, Maurizio Cimadamore wrote: > Hi, > now that the work on allowing static members in nested declarations > [1] is wrapping up, I'm wondering if we could add treatment for one > more case: static initializers in interfaces. This seems a case where > the static compiler is already inserting static blocks (to initialize > any static fields an interface might declare), but where the JLS is > currently forbidding interfaces from having an explicit static > initializer block in the source code. This make e.g. initialization of > fields which might throw exceptions (e.g. method handles) not possible > inside interfaces. While I realize this is a corner case, I thought it > would still be worth asking the question :-) > > Cheers > Maurizio > > [1] - https://git.openjdk.java.net/jdk/pull/571 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Tue Nov 17 14:15:42 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 17 Nov 2020 15:15:42 +0100 (CET) Subject: [statics] allowing static initializers in interfaces? In-Reply-To: <45928680-f4a2-d21e-665c-19de2d87e2c0@oracle.com> References: <45928680-f4a2-d21e-665c-19de2d87e2c0@oracle.com> Message-ID: <2019969021.1583036.1605622542894.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Maurizio Cimadamore" > ?: "amber-spec-experts" > Envoy?: Mardi 17 Novembre 2020 15:03:39 > Objet: [statics] allowing static initializers in interfaces? > Hi, > now that the work on allowing static members in nested declarations [1] > is wrapping up, I'm wondering if we could add treatment for one more > case: static initializers in interfaces. This seems a case where the > static compiler is already inserting static blocks (to initialize any > static fields an interface might declare), but where the JLS is > currently forbidding interfaces from having an explicit static > initializer block in the source code. This make e.g. initialization of > fields which might throw exceptions (e.g. method handles) not possible > inside interfaces. While I realize this is a corner case, I thought it > would still be worth asking the question :-) +1 for me, Yes, it's time of heal the rift between a class and an interface, we stop halfway by just adding default methods and later private methods. So currently, there are 3 ad_hoc differences - as you said, no static block, even if you can initialize a static field with an initializer - no private class - no private static field. The last will also requires a change of the JVM spec. > > Cheers > Maurizio cheers, R?mi > > [1] - https://git.openjdk.java.net/jdk/pull/571 From john.r.rose at oracle.com Wed Nov 18 08:07:18 2020 From: john.r.rose at oracle.com (John Rose) Date: Wed, 18 Nov 2020 00:07:18 -0800 Subject: [statics] allowing static initializers in interfaces? In-Reply-To: <45928680-f4a2-d21e-665c-19de2d87e2c0@oracle.com> References: <45928680-f4a2-d21e-665c-19de2d87e2c0@oracle.com> Message-ID: +1 from me too, and generally to heal the rift. The inability to put private classes (and other ?stuff?) in an interface makes it hard to support canonical algorithms in default methods, if they need auxiliary types. > On Nov 17, 2020, at 6:03 AM, Maurizio Cimadamore wrote: > > Hi, > now that the work on allowing static members in nested declarations [1] is wrapping up, I'm wondering if we could add treatment for one more case: static initializers in interfaces. This seems a case where the static compiler is already inserting static blocks (to initialize any static fields an interface might declare), but where the JLS is currently forbidding interfaces from having an explicit static initializer block in the source code. This make e.g. initialization of fields which might throw exceptions (e.g. method handles) not possible inside interfaces. While I realize this is a corner case, I thought it would still be worth asking the question :-) > > Cheers > Maurizio > > [1] - https://git.openjdk.java.net/jdk/pull/571 > From forax at univ-mlv.fr Wed Nov 18 10:58:07 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 18 Nov 2020 11:58:07 +0100 (CET) Subject: [statics] allowing static initializers in interfaces? In-Reply-To: References: <45928680-f4a2-d21e-665c-19de2d87e2c0@oracle.com> Message-ID: <777327472.2212237.1605697087284.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "John Rose" > ?: "Maurizio Cimadamore" > Cc: "amber-spec-experts" > Envoy?: Mercredi 18 Novembre 2020 09:07:18 > Objet: Re: [statics] allowing static initializers in interfaces? > +1 from me too, and generally to heal the rift. > The inability to put private classes (and other ?stuff?) > in an interface makes it hard to support canonical > algorithms in default methods, if they need auxiliary > types. yes, another example, in the patch introducing the new random generator API currently under review, there is this method default float nextFloat() { return (nextInt() >>> 8) * 0x1.0p-24f; } which a good example of why we also need private static final fields in interface. R?mi > >> On Nov 17, 2020, at 6:03 AM, Maurizio Cimadamore >> wrote: >> >> Hi, >> now that the work on allowing static members in nested declarations [1] is >> wrapping up, I'm wondering if we could add treatment for one more case: static >> initializers in interfaces. This seems a case where the static compiler is >> already inserting static blocks (to initialize any static fields an interface >> might declare), but where the JLS is currently forbidding interfaces from >> having an explicit static initializer block in the source code. This make e.g. >> initialization of fields which might throw exceptions (e.g. method handles) not >> possible inside interfaces. While I realize this is a corner case, I thought it >> would still be worth asking the question :-) >> >> Cheers >> Maurizio >> >> [1] - https://git.openjdk.java.net/jdk/pull/571 From brian.goetz at oracle.com Thu Nov 19 20:47:59 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 19 Nov 2020 15:47:59 -0500 Subject: Relaxed assignment conversions for sealed types In-Reply-To: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> References: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> Message-ID: Bump -- hoping for feedback on this thought: > ??? BarImpl bi = (__static BarImpl) b; > > Pulling on this string some more ? I think there?s a connection > between this feature and total statement switches. ?We?ve been looking > for a way to make statement switches total, and here, what we?re > looking for is a way to make _casts_ total. ?Which suggests this is > one feature, not two. ?So perhaps: > > ? ? switch-total (x) { ? } ?// a switch, but with added bonus totality > checking > > ? ? BarImpl b = (total BarImpl) bar ?// a cast, but with added bonus > totality checking > > Obviously we can use another word besides `total`, but it?s a pretty > good straw man. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Thu Nov 19 21:13:07 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 19 Nov 2020 22:13:07 +0100 (CET) Subject: Relaxed assignment conversions for sealed types In-Reply-To: References: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> Message-ID: <1953464029.814172.1605820387156.JavaMail.zimbra@u-pem.fr> We have avoided in the past to mix the class restriction and the type system. By example, even if a class A is declared final, List is not equivalent to List. For me, you want to steer the type system to work more like a close world type system, so it's not very consistent with the existing semantics and the fact that we have chosen to add the keyword 'sealed' instead of introducing a special construct to define a sum type. regards, R?mi > De: "Brian Goetz" > ?: "Tagir Valeev" > Cc: "amber-spec-experts" > Envoy?: Jeudi 19 Novembre 2020 21:47:59 > Objet: Re: Relaxed assignment conversions for sealed types > Bump -- hoping for feedback on this thought: >> BarImpl bi = (__static BarImpl) b; >> Pulling on this string some more ? I think there?s a connection between this >> feature and total statement switches. We?ve been looking for a way to make >> statement switches total, and here, what we?re looking for is a way to make >> _casts_ total. Which suggests this is one feature, not two. So perhaps: >> switch-total (x) { ? } // a switch, but with added bonus totality checking >> BarImpl b = (total BarImpl) bar // a cast, but with added bonus totality >> checking >> Obviously we can use another word besides `total`, but it?s a pretty good straw >> man. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Sun Nov 22 18:32:38 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 22 Nov 2020 13:32:38 -0500 Subject: Relaxed assignment conversions for sealed types In-Reply-To: <1953464029.814172.1605820387156.JavaMail.zimbra@u-pem.fr> References: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> <1953464029.814172.1605820387156.JavaMail.zimbra@u-pem.fr> Message-ID: I guess I deserved that, because I wasn't being specific, but I was looking for slightly different feedback :) What I was trying to ask for feedback on was, that we have two problems in search of solutions, and I was proposing that they are both, at some level, the same problem -- and might benefit from a common approach. On 11/19/2020 4:13 PM, Remi Forax wrote: > We have avoided in the past to mix the class restriction and the type > system. > By example, even if a class A is declared final, List is > not equivalent to List. > > For me, you want to steer the type system to work more like a close > world type system, > so it's not very consistent with the existing semantics and the fact > that we have chosen to add the keyword 'sealed' instead of introducing > a special construct to define a sum type. > > regards, > R?mi > > ------------------------------------------------------------------------ > > *De: *"Brian Goetz" > *?: *"Tagir Valeev" > *Cc: *"amber-spec-experts" > *Envoy?: *Jeudi 19 Novembre 2020 21:47:59 > *Objet: *Re: Relaxed assignment conversions for sealed types > > Bump -- hoping for feedback on this thought: > > BarImpl bi = (__static BarImpl) b; > > > Pulling on this string some more ? I think there?s a > connection between this feature and total statement switches. > ?We?ve been looking for a way to make statement switches > total, and here, what we?re looking for is a way to make > _casts_ total. ?Which suggests this is one feature, not two. > ?So perhaps: > > ? ? switch-total (x) { ? } ?// a switch, but with added bonus > totality checking > > ? ? BarImpl b = (total BarImpl) bar ?// a cast, but with added > bonus totality checking > > Obviously we can use another word besides `total`, but it?s a > pretty good straw man. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Sun Nov 22 19:06:37 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Sun, 22 Nov 2020 20:06:37 +0100 (CET) Subject: Relaxed assignment conversions for sealed types In-Reply-To: References: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> <1953464029.814172.1605820387156.JavaMail.zimbra@u-pem.fr> Message-ID: <1521755791.2222132.1606071997499.JavaMail.zimbra@u-pem.fr> > De: "Brian Goetz" > ?: "Remi Forax" > Cc: "Tagir Valeev" , "amber-spec-experts" > > Envoy?: Dimanche 22 Novembre 2020 19:32:38 > Objet: Re: Relaxed assignment conversions for sealed types > I guess I deserved that, because I wasn't being specific, but I was looking for > slightly different feedback :) > What I was trying to ask for feedback on was, that we have two problems in > search of solutions, and I was proposing that they are both, at some level, the > same problem -- and might benefit from a common approach. A cast is equivalent to an if that throws a CCE in the else branch and a total-switch with one case (because there is only one subtype) throws an exception in its default branch. So they are roughly equivalent without squinting too much (the behavior for null is also different). One problem we have is how to declare a total switch when it's not a switch expression, so the problem is a little different than just how to declare a total switch. This problem can be seen as even more specific if we say that once there is a "case Type", then the switch is total, because it becomes how to declare a total switch when it's a switch statement on an enum (and maybe an enum on true/false). At that point, it can be seen as a corner of a corner case because of backward compatibility that we will not fix or choose to fix it using the cast to void syntax proposed earlier (the least bad proposed syntax IMO). About using total as a prefix for a cast, I suppose that the idea is to have a compile-time error message if the sealed type permits more than one class. While it's cool, i'm not sure we need that and an annotation at declaration site similar too @FunctionalInterface seems a better idea than using 'total' at each use site. regards, R?mi > On 11/19/2020 4:13 PM, Remi Forax wrote: >> We have avoided in the past to mix the class restriction and the type system. >> By example, even if a class A is declared final, List is not >> equivalent to List. >> For me, you want to steer the type system to work more like a close world type >> system, >> so it's not very consistent with the existing semantics and the fact that we >> have chosen to add the keyword 'sealed' instead of introducing a special >> construct to define a sum type. >> regards, >> R?mi >>> De: "Brian Goetz" [ mailto:brian.goetz at oracle.com | ] >>> ?: "Tagir Valeev" [ mailto:amaembo at gmail.com | ] >>> Cc: "amber-spec-experts" [ mailto:amber-spec-experts at openjdk.java.net | >>> ] >>> Envoy?: Jeudi 19 Novembre 2020 21:47:59 >>> Objet: Re: Relaxed assignment conversions for sealed types >>> Bump -- hoping for feedback on this thought: >>>> BarImpl bi = (__static BarImpl) b; >>>> Pulling on this string some more ? I think there?s a connection between this >>>> feature and total statement switches. We?ve been looking for a way to make >>>> statement switches total, and here, what we?re looking for is a way to make >>>> _casts_ total. Which suggests this is one feature, not two. So perhaps: >>>> switch-total (x) { ? } // a switch, but with added bonus totality checking >>>> BarImpl b = (total BarImpl) bar // a cast, but with added bonus totality >>>> checking >>>> Obviously we can use another word besides `total`, but it?s a pretty good straw >>>> man. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Mon Nov 23 21:34:12 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 23 Nov 2020 16:34:12 -0500 Subject: [sealed-classes] Spec for next version of Sealed Classes In-Reply-To: <8D8979A8-3C74-4D05-A749-DB29477408AC@oracle.com> References: <8D8979A8-3C74-4D05-A749-DB29477408AC@oracle.com> Message-ID: <725c390a-b1cf-9931-f5e5-84d985120225@oracle.com> I have reviewed the spec. Very nice how simply the disjointness part boiled down. On 11/4/2020 12:50 PM, Gavin Bierman wrote: > The permanent location for this spec is now: > > http://cr.openjdk.java.net/~gbierman/jep397/latest/ > > > (I have fixed the two issues that Chris spotted [1].) > > Comments still welcome! > > Gavin > > [1] > http://mail.openjdk.java.net/pipermail/amber-spec-experts/2020-October/002624.html > > >> On 23 Oct 2020, at 16:16, Gavin Bierman > > wrote: >> >> Dear all: >> >> Drafts of the specs for the Sealed Classes feature that we plan to >> preview for >> a second time in JDK 16 are now available: >> >> http://cr.openjdk.java.net/~gbierman/8246775/latest/ >> >> >> [NB: The URL will change once we have a JEP number, and will be >> announced.] >> >> The changes are the same as those in the first preview that was >> released in Java >> SE 15, except for minor editorial changes and the following: >> >> - Clarification the use of context when applying the lexical grammar, >> ?particularly in the identification of contextual keywords (formerly >> described >> ?as "restricted identifiers" and "restricted keywords"). This is >> detailed in a >> ?companion document entitled ?Contextual Keywords". The keywords >> `sealed`, >> ?`non-sealed`, and `permits` are now defined as new instances of >> contextual >> ?keywords (3.9). >> >> - This spec now assumes that the changes detailed in companion documents >> ?entitled "Consistent Class and Interface Terminology? and "Local and >> Nested >> ?Static Declarations" have been applied (these are being introduced >> as part of >> ?the Records JEP). In particular, this means that Java SE 16 will support >> ?static declarations in two new positions: >> >> ???1. Local, implicitly-static interfaces and enum classes >> ???2. Static members of inner classes >> >> ?This requires asserting that local interfaces are not permitted to be >> ?`sealed`. (14.3) >> >> - To enhance narrowing reference conversion to allow for stricter >> checking of >> ?cast conversions with respect to sealed type hierarchies (5.1.6.1). >> >> - Local classes are not considered when determining implicitly declared >> ?permitted direct subclasses of a `sealed` class or `sealed` interface >> ?(8.1.6, 9.1.4). >> >> >> Comments welcome! >> >> Gavin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hegarty at oracle.com Tue Nov 24 12:44:23 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 24 Nov 2020 12:44:23 +0000 Subject: [sealed-classes] Spec for next version of Sealed Classes In-Reply-To: <2DBCF829-E5CA-44B7-9C9C-81B093CB7704@oracle.com> References: <8B135BD4-664A-4D0B-A092-24D0CD7265F4@oracle.com> <2DBCF829-E5CA-44B7-9C9C-81B093CB7704@oracle.com> Message-ID: Dan, > On 29 Oct 2020, at 21:29, Dan Smith wrote: > >> ... > >> "C does not have its ACC_PUBLIC flag set (4.1) and the superclass belongs to a different run-time package than C.? - I get that there is a bidirectional accessibility relationship between the superclass and the subclass, but this seems at odds with JEP text: ?In particular, a subclass may be less accessible than the sealed class?. Why is this not that the superclass must have ACC_PUBLIC, and not the subclass? > > The goal of this rule is to simulate resolution without actually asking to perform it, since the class hasn't even been created yet. Part of resolution is access checking, so we simulate it here. > > The JEP says "a permitted subclass and its sealed superclass must be accessible to each other. However, permitted subclasses need not have the same accessibility as each other, or as the sealed class." > > Keep in mind we're talking now about language-level accessibility, which doesn't always map to JVM accessibility. Understood. My question is only related to accessibility in the JVM. > Anyway, some examples: > public super, non-public sub, same package > package-access super, private sub, same package > public *and exported* super, public sub > private super, public sub, same nest Thanks for these examples, they are helpful. Here is one more: public super, non-public sub, same module, DIFF package The draft text [*] seems to explicitly disallow this (which I believe is incorrect, or at least I don?t understand why). [*] ? ... derivation fails with a IncompatibleClassChangeError: ... * C does not have its ACC_PUBLIC flag set (4.1) and the superclass belongs to a different run-time package than C. ?" I believe that the intent is that ?SUPERCLASS does not have its ACC_PUBLIC flag set (4.1) and C belongs to a different run-time package than SUPERCLASS? - not the other way around. No? -Chris. From maurizio.cimadamore at oracle.com Tue Nov 24 14:10:35 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 24 Nov 2020 14:10:35 +0000 Subject: Relaxed assignment conversions for sealed types In-Reply-To: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> References: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> Message-ID: <55b14498-32a2-46e3-9d37-6c016522f3e0@oracle.com> On 31/10/2020 23:30, Brian Goetz wrote: > > >> On Oct 25, 2020, at 10:06 AM, Brian Goetz > > wrote: >> >> To make it clear that I'm not talking about the annoyance of typing >> the cast, let's pretend I'm suggesting to write it like this: >> >> ??? BarImpl bi = (__static BarImpl) b; > > Pulling on this string some more ? I think there?s a connection > between this feature and total statement switches. ?We?ve been looking > for a way to make statement switches total, and here, what we?re > looking for is a way to make _casts_ total. ?Which suggests this is > one feature, not two. ?So perhaps: > > ? ? switch-total (x) { ? } ?// a switch, but with added bonus totality > checking > > ? ? BarImpl b = (total BarImpl) bar ?// a cast, but with added bonus > totality checking I agree the latter is a common enough problem when writing implementation code where you have a sealed hierarchy and you know there's only one impl (Foreign API has this all over the place). To throw in the mix - how is some kind of pattern match assignment (we referred to as a "let expression" in some of the earlier docs [1]) would change the picture here? In other words, maybe it's overloading `=` which is at odds here, and we need to make it more explicit that this is more akin to an extraction/match? Maurizio [1] - https://cr.openjdk.java.net/~briangoetz/amber/pattern-semantics.html > > Obviously we can use another word besides `total`, but it?s a pretty > good straw man. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Tue Nov 24 14:45:48 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 24 Nov 2020 15:45:48 +0100 (CET) Subject: Relaxed assignment conversions for sealed types In-Reply-To: <55b14498-32a2-46e3-9d37-6c016522f3e0@oracle.com> References: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> <55b14498-32a2-46e3-9d37-6c016522f3e0@oracle.com> Message-ID: <783473703.1450038.1606229148222.JavaMail.zimbra@u-pem.fr> > De: "Maurizio Cimadamore" > ?: "Brian Goetz" , "Tagir Valeev" > Cc: "amber-spec-experts" > Envoy?: Mardi 24 Novembre 2020 15:10:35 > Objet: Re: Relaxed assignment conversions for sealed types > On 31/10/2020 23:30, Brian Goetz wrote: >>> On Oct 25, 2020, at 10:06 AM, Brian Goetz < [ mailto:brian.goetz at oracle.com | >>> brian.goetz at oracle.com ] > wrote: >>> To make it clear that I'm not talking about the annoyance of typing the cast, >>> let's pretend I'm suggesting to write it like this: >>> BarImpl bi = (__static BarImpl) b; >> Pulling on this string some more ? I think there?s a connection between this >> feature and total statement switches. We?ve been looking for a way to make >> statement switches total, and here, what we?re looking for is a way to make >> _casts_ total. Which suggests this is one feature, not two. So perhaps: >> switch-total (x) { ? } // a switch, but with added bonus totality checking >> BarImpl b = (total BarImpl) bar // a cast, but with added bonus totality >> checking > I agree the latter is a common enough problem when writing implementation code > where you have a sealed hierarchy and you know there's only one impl (Foreign > API has this all over the place). You think that not typechecking that BarImpl is the sole implementation of Bar everytime you write a cast is a problem ? I don't know for the Foreign API, but having a static helper method that takes a BarImpl and returns a Bar is not enough ? BarImpl impl(Bar bar) { return switch(bar) { case BarImpl impl -> impl; }; } Or am i missing something ? > To throw in the mix - how is some kind of pattern match assignment (we referred > to as a "let expression" in some of the earlier docs [1]) would change the > picture here? In other words, maybe it's overloading `=` which is at odds here, > and we need to make it more explicit that this is more akin to an > extraction/match? The match operator (as in let ... match), is more or less what C# as done by using switch as an operator, BarImpl impl(Bar bar) { return bar switch { case BarImpl impl -> impl; }; } It's still a possible syntax for total-switch. Anyway, it doesn't answer to your question, it seems you want something in the middle in between a cast and a switch, let impl = bar; > Maurizio R?mi > [1] - [ https://cr.openjdk.java.net/~briangoetz/amber/pattern-semantics.html | > https://cr.openjdk.java.net/~briangoetz/amber/pattern-semantics.html ] >> Obviously we can use another word besides `total`, but it?s a pretty good straw >> man. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Tue Nov 24 15:01:20 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 24 Nov 2020 15:01:20 +0000 Subject: Relaxed assignment conversions for sealed types In-Reply-To: <783473703.1450038.1606229148222.JavaMail.zimbra@u-pem.fr> References: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> <55b14498-32a2-46e3-9d37-6c016522f3e0@oracle.com> <783473703.1450038.1606229148222.JavaMail.zimbra@u-pem.fr> Message-ID: On 24/11/2020 14:45, Remi Forax wrote: > > > ------------------------------------------------------------------------ > > *De: *"Maurizio Cimadamore" > *?: *"Brian Goetz" , "Tagir Valeev" > > *Cc: *"amber-spec-experts" > *Envoy?: *Mardi 24 Novembre 2020 15:10:35 > *Objet: *Re: Relaxed assignment conversions for sealed types > > > On 31/10/2020 23:30, Brian Goetz wrote: > > > > On Oct 25, 2020, at 10:06 AM, Brian Goetz > > > wrote: > > To make it clear that I'm not talking about the annoyance > of typing the cast, let's pretend I'm suggesting to write > it like this: > > ??? BarImpl bi = (__static BarImpl) b; > > > Pulling on this string some more ? I think there?s a > connection between this feature and total statement switches. > ?We?ve been looking for a way to make statement switches > total, and here, what we?re looking for is a way to make > _casts_ total. ?Which suggests this is one feature, not two. > ?So perhaps: > > ? ? switch-total (x) { ? } ?// a switch, but with added bonus > totality checking > > ? ? BarImpl b = (total BarImpl) bar ?// a cast, but with added > bonus totality checking > > I agree the latter is a common enough problem when writing > implementation code where you have a sealed hierarchy and you know > there's only one impl (Foreign API has this all over the place). > > > You think that not typechecking that BarImpl is the sole > implementation of Bar everytime you write a cast is a problem ? > I don't know for the Foreign API, but having a static helper method > that takes a BarImpl and returns a Bar is not enough ? I'm saying that it's (in my humble experience) a common enough pattern that I've seen often enough, so what Brian is proposing doesn't seem completely crazy to me - e.g. having some more 1st class support in the language for this sort of stuff can be a good thing. > > ? BarImpl impl(Bar bar) { > ??? return switch(bar) { case BarImpl impl -> impl; }; > ? } > > Or am i missing something ? See first-class :-) Sure, there are other ways to get there. > > To throw in the mix - how is some kind of pattern match assignment > (we referred to as a "let expression" in some of the earlier docs > [1]) would change the picture here? In other words, maybe it's > overloading `=` which is at odds here, and we need to make it more > explicit that this is more akin to an extraction/match? > > > The match operator (as in let ... match), is more or less what C# as > done by using switch as an operator, > ? BarImpl impl(Bar bar) { > ??? return bar switch { case BarImpl impl -> impl; }; > ? } > > It's still a possible syntax for total-switch. > Anyway, it doesn't answer to your question, it seems you want > something in the middle in between a cast and a switch, let impl = bar; I think all your references to switch expression reinforce my feeling that having a pattern-matching based way to declare local variables (as the document I linked in [1] shows with its __let construct) might be one way to skin this cat. Maurizio > > > Maurizio > > > R?mi > > [1] - > https://cr.openjdk.java.net/~briangoetz/amber/pattern-semantics.html > > > Obviously we can use another word besides `total`, but it?s a > pretty good straw man. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Tue Nov 24 15:24:22 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 24 Nov 2020 10:24:22 -0500 Subject: Relaxed assignment conversions for sealed types In-Reply-To: <55b14498-32a2-46e3-9d37-6c016522f3e0@oracle.com> References: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> <55b14498-32a2-46e3-9d37-6c016522f3e0@oracle.com> Message-ID: <0b22c0dc-00d7-31da-1a88-6d91ba32e529@oracle.com> > To throw in the mix - how is some kind of pattern match assignment (we > referred to as a "let expression" in some of the earlier docs [1]) > would change the picture here? In other words, maybe it's overloading > `=` which is at odds here, and we need to make it more explicit that > this is more akin to an extraction/match? > So, I think "let statements" beat "total casts" to the overload here :) Originally, we imagined a "let" statement like: ??? let P = e [ else ... ] where P is an optimistically total pattern. We pulled on the "what can you do in the else, besides throw" string for a while, but didn't love where it ended up.? The obvious thing you might want to do besides throw or return, is to assign to the pattern variables and keep going: ??? let Point(var x, var y) ??? else { x = 0; y = 0; } This was messy when pattern variables were implicitly final, but now that we backed away from that siren song (whew), these become simple assignments to DU locals. But over time, the `let` started to rankle, not only as noise, but because it limited the usefulness.? The observation is that (again, now that we backed away from the siren song of pattern variables being final): ??? Point p = could either be the declaration of a local of type Point, _or_ a pattern match to the total pattern `Point p`.? In other words, what we historically? thought of as local variable declaration, could in fact be generalized to total pattern matching.? Not only is this aesthetically pleasing, but it means we can extend this idiom in two dimensions: other patterns, and other places. The "other patterns" is straightforward enough: ??? Point(var x, var y) = p The only challenge here is what to do with the remainder, but we've done enough work on characterizing the remainder to be comfortable enough with throwing in those cases (usually NPE.) The "other places" takes slightly longer to get used to, such as method declarations: ??? void foo(Point(var x, var y) p, int z) { ... } which is equivalent to: ??? void foo(Point p, int z) { Point(var x, var y) = p; ... } and a similar possibility with lambdas: ??? (Point(var x, var y) p) -> ... All of this is to say that (a) the "let" statement structure was getting in the way, and (b) there's a generalization of local declaration struggling to get out here, and (c) pattern totality is the key to it. Given all this, it would be justifiable to not require the cast at all!? The pattern `BarImpl b` is total on `Bar` (assuming Bar is abstract and sealed only to BarImpl), by virtue of the rules for totality that are outlined in [1].? So by this interpretation of "local declaration generalizes to total pattern matching", we could just say: ??? BarImpl b = bar; and be done, since the expression `bar` is of type Bar and the pattern `BarImpl b` is optimistically total on Bar.? (If the sealing changes between compile and run time, this could throw CCE.) But some people might find that uncomfortable, which is how we got here; they want the optimism to be explicit.? (Backing off to "let" seems a strict loser compared to "total cast", since total cast has much more leverage for the same incremental syntactic footprint (and requiring let where things are more obvious will be seen as noise, and the statement-ness of `let` gets in the way of using it in more places.))? If this is too aggressive, we would want to limit the `P = e` form to strict totality, or more likely, totality with only { null } remainder (as this would permit using deconstruction patterns on the LHS.) The blind cast is surely allowed, but its not something we want to encourage -- and as both you and I have found (perhaps the only people who have yet written significant libraries with sealing?), it is going to be almost irresistible to avoid embedding the blind-cast assumptions in your code. So this is how I got to a "blind cast with assertion", not unlike an @Override assertion -- cast, but raise a compile-time error if the cast is not optimistically total.? Instanceof and switch learned about totality; perhaps casting should too. [1] https://github.com/openjdk/amber-docs/blob/master/site/design-notes/type-patterns-in-switch.md From heidinga at redhat.com Tue Nov 24 15:23:58 2020 From: heidinga at redhat.com (Dan Heidinga) Date: Tue, 24 Nov 2020 10:23:58 -0500 Subject: Relaxed assignment conversions for sealed types In-Reply-To: References: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> <55b14498-32a2-46e3-9d37-6c016522f3e0@oracle.com> <783473703.1450038.1606229148222.JavaMail.zimbra@u-pem.fr> Message-ID: On Tue, Nov 24, 2020 at 10:02 AM Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > > On 24/11/2020 14:45, Remi Forax wrote: > > > > ------------------------------ > > *De: *"Maurizio Cimadamore" > > *?: *"Brian Goetz" , > "Tagir Valeev" > *Cc: *"amber-spec-experts" > > *Envoy?: *Mardi 24 Novembre 2020 15:10:35 > *Objet: *Re: Relaxed assignment conversions for sealed types > > > On 31/10/2020 23:30, Brian Goetz wrote: > > > > On Oct 25, 2020, at 10:06 AM, Brian Goetz wrote: > > To make it clear that I'm not talking about the annoyance of typing the > cast, let's pretend I'm suggesting to write it like this: > > BarImpl bi = (__static BarImpl) b; > > > Pulling on this string some more ? I think there?s a connection between > this feature and total statement switches. We?ve been looking for a way to > make statement switches total, and here, what we?re looking for is a way to > make _casts_ total. Which suggests this is one feature, not two. So > perhaps: > > switch-total (x) { ? } // a switch, but with added bonus totality > checking > > BarImpl b = (total BarImpl) bar // a cast, but with added bonus > totality checking > > I agree the latter is a common enough problem when writing implementation > code where you have a sealed hierarchy and you know there's only one impl > (Foreign API has this all over the place). > > > You think that not typechecking that BarImpl is the sole implementation of > Bar everytime you write a cast is a problem ? > I don't know for the Foreign API, but having a static helper method that > takes a BarImpl and returns a Bar is not enough ? > > I'm saying that it's (in my humble experience) a common enough pattern > that I've seen often enough, so what Brian is proposing doesn't seem > completely crazy to me - e.g. having some more 1st class support in the > language for this sort of stuff can be a good thing. > > > BarImpl impl(Bar bar) { > return switch(bar) { case BarImpl impl -> impl; }; > } > > Or am i missing something ? > > See first-class :-) Sure, there are other ways to get there. > > > To throw in the mix - how is some kind of pattern match assignment (we > referred to as a "let expression" in some of the earlier docs [1]) would > change the picture here? In other words, maybe it's overloading `=` which > is at odds here, and we need to make it more explicit that this is more > akin to an extraction/match? > > > The match operator (as in let ... match), is more or less what C# as done > by using switch as an operator, > BarImpl impl(Bar bar) { > return bar switch { case BarImpl impl -> impl; }; > } > > It's still a possible syntax for total-switch. > Anyway, it doesn't answer to your question, it seems you want something in > the middle in between a cast and a switch, let impl = bar; > > I think all your references to switch expression reinforce my feeling that > having a pattern-matching based way to declare local variables (as the > document I linked in [1] shows with its __let construct) might be one way > to skin this cat. > > Maurizio > > > Looking at Remi's examples makes me agree more strongly that we need first class support - can't you just hear the resounding "java's too verbose" complaints about having to write the switch helper method and remember to pepper your code with calls to "impl(bar)" rather than using casts as you would elsewhere in your code? I agree Brian's really onto something with suggesting the totality question is the same for the two features. I'm in favour of going even further and removing the cast to allow = to do double duty here for sealed types rather than introducing new syntax for it. The new syntax ("__ construct" or "(total BarImpl)") highlights "this is different" and I'm not sure we need to do that for sealed types. Letting ='s do the work feels like a better form of auto-boxing where javac does the compile-time analysis and errors out for non-total cases or inserts the required total cast for the user. This gives both compile time checking and runtime safety without needing to beat the reader over the head about it. We want to make it easier to read & write correct code - pushing the analysis of totality for sealed types into javac does both for us. --Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Tue Nov 24 15:35:20 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 24 Nov 2020 10:35:20 -0500 Subject: Relaxed assignment conversions for sealed types In-Reply-To: References: <4F38821E-C785-450D-BC2C-811A997B172E@oracle.com> <55b14498-32a2-46e3-9d37-6c016522f3e0@oracle.com> <783473703.1450038.1606229148222.JavaMail.zimbra@u-pem.fr> Message-ID: > > I agree Brian's really onto something with suggesting the totality > question is the same for the two features.? I'm in favour of going > even further and removing the cast to allow = to do double duty here > for sealed types rather than introducing new syntax for it.? The new > syntax ("__ construct" or "(total BarImpl)") highlights "this is > different" and I'm not sure we need to do that for sealed types.? > Letting ='s do the work feels like a better form of auto-boxing where > javac does the compile-time analysis and errors out for non-total > cases or inserts the required total cast for the user.? This gives > both compile time checking and runtime safety without needing to beat > the reader over the head about it. > > We want to make it easier to read & write correct code - pushing the > analysis of totality for sealed types into javac does both for us. > To connect to the mail that crossed with Dan's, Dan is voting for: ?- Allowing `P = e` when P is optimistically total on the static type of `e` In reviewing the discussions on this over the past few years, what I'm seeing is that this requires a certain degree of "getting comfortable" with it.? At first, locutions like this seemed woefully imprecise, and everyone got nervous, but over time, people have been coming around to it.? The "bargaining" stage involves things like "stick `let` in front of it", "allow it for locals, but not method parameters", and "require it to be strictly total (or total only with null remainder.")?? Over time, many of these proposed restrictions are seen to be "bargaining" with change, and we get comfortable with them. Dan's observation here is that, with a suitable interpretation of total pattern match as a generalization of local declaration, we don't need the cast at all, so we sidestep the question. That said, even if we do that, we're not totally out of the woods, for a minor and major reason. The minor reason is the use in expressions.? Suppose we have a context where we are limited to expressions, not statements (e.g., field initializers, constructor arguments, etc.) or where we just don't want to use a statement for aesthetic reasons. ? We still have to use a cast in these cases, and the cast is still blind.? But that's minor. THe major reason is that we still don't have a way to say "this statement switch is total, please typecheck that for me."? I don't see a way we get to inferring this, which means we need some syntax.? Being able to reuse the same syntax in other contexts (e.g., casts) may reinforce it in both places.