From daniel.smith at oracle.com Wed Nov 2 00:14:13 2022 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 2 Nov 2022 00:14:13 +0000 Subject: EG meeting *canceled*, 2022-11-02 Message-ID: Well, I was a little too optimistic about our progress. We've restarted some internal design discussions, hopefully will have something ready for wider consumption next time. But nothing new for this week. From daniel.smith at oracle.com Tue Nov 15 19:51:19 2022 From: daniel.smith at oracle.com (Dan Smith) Date: Tue, 15 Nov 2022 19:51:19 +0000 Subject: EG meeting, 2022-11-16 Message-ID: EG Zoom meeting November 16 at 5pm UTC (9am PDT, 12pm EDT). We're back with some new ideas about modeling flattening in the language! Brian has some slides to share, and hopefully that will prompt a good discussion. Hope everyone can make it. From john.r.rose at oracle.com Fri Nov 18 02:54:46 2022 From: john.r.rose at oracle.com (John Rose) Date: Thu, 17 Nov 2022 18:54:46 -0800 Subject: straw man for conversion bytecode (from EG meeting) In-Reply-To: References: Message-ID: <08519979-ADEC-4020-A708-C9C39D425FCA@oracle.com> Yesterday we bumped (for the Nth time) into the subject of bytecode shape for the new conversions (mostly implicit conversions) between primitives (both legacy and user-defined) and their (nullable) box types. (This connects to the recurring question, which recurred yesterday, of whether we need a third int type, of the form `Qjava/lang/Integer;`, to accompany `Ljava/lang/Integer;` and `I`, in various kinds of bytecode translating tricky Java code. To which I always say, ?No?, and ?but show me a real job for the Q-descriptor of int?, and then follow some suggestions which I debunk as unnecessary. Where we end is at this point, where we know we need conversion code at least between the remaining two VM types.) We know we can just use various JDK API points to do these jobs, possibly with extra `checkcast` instructions to clean up the verifier types. We asked, early in the Valhalla design process, ?do we want or need a general conversion bytecode?? and dubbed it `a2b` at some points. IMO, if such a bytecode were truly necessary in full generality, it would indicate that our verifier type system was unmanageably complex, so I?m glad to see we are not reaching for it. But I do want to write, for the record, a proposal (probably not to be adopted) for how we might fill a small and modest niche in the translation strategy, the one occupied by those library calls and `checkcast` uses, with a bytecode enhancement. So we can know more clearly what we are rejecting, or (if we were to adopt it) so we can know the cost sooner rather than later. So, here it is. I have tried to write it in a tone that is *very tentative*. Please do not try to read it as something I am enthusiastic about. But, have a look. enhanced checkcast for Valhalla type unification https://bugs.openjdk.org/browse/JDK-8297236 (I think the problem statement part is pretty good, actually.) P.S. We also discussed how to attach null-rejection to variables that are not naturally null-rejecting. The above proposal gives a nicer way to do it than we have today. And a variation on the ?lazy static? theme gives yet another way, which is attaching initialization-detection logic to fields and array elements. That is very expensive, so again I?m not proposing it for today, but rather just putting it on the table. Here is that note: low-level control of field initialization https://bugs.openjdk.org/browse/JDK-8297156 -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Wed Nov 23 13:03:40 2022 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 23 Nov 2022 14:03:40 +0100 (CET) Subject: Null restricted type and JSpecify Message-ID: <1508664657.51644988.1669208620662.JavaMail.zimbra@u-pem.fr> Hi all, i've taken a look to JSpecify [1] to see how it works with the recent proposal from Brian, seen value types as null restricted types. Obviously, i'm not an expert in the details of JSpecify so feel free to correct me. I really like JSpecify, the specification is simple and lean but yet at the same time guides users towards a null safe Java. JSpecify specifies two annotations, @NullMarked and @Nullable. @NullMarked (on a class, a package or a module) says that any types in that scope is not null (NO_CHANGE) by default. @Nullable indicates that a type is nullable (UNION_NULL), inside a @NullMarked scope or not. A type outside the the @NullMarked scope and not marked as @Nullable is unspecified (UNSPECIFIED) [2]. Compared to the null restricted type proposal, there are IMO two major differences, - the fact that JSpecify uses annotations implies that adding nullness information is a binary backward compatible change. - there is no @NonNull, a lack of @Nullable inside a @NullMarked scope encodes the fact that a type is non null. The former suggests that we should investigate erasing Q-type to L-type inside method/field descriptors. The later suggests that either we should embrace this approach too and only provides '?' or we should allow both '!' (NO_CHANGE) and '?' (UNION_NULL) but it makes the transition from actual Java to a null safe Java less clear. The other difference, which was noted by Kevin during our meeting last week, JSpecify is specified in term of subtyping relationship not in term of boxing/unboxing rules. For me, boxing/unboxing rules are less specific that subtyping in assignments but more specific in case of overriding, so i've no idea if this is an issue or not. regards, R?mi [1] https://jspecify.dev/ [2] there is a 4th state which is MINUS_NULL but it's for type variable, so not something which is directly related to value types. From kevinb at google.com Wed Nov 23 19:23:22 2022 From: kevinb at google.com (Kevin Bourrillion) Date: Wed, 23 Nov 2022 11:23:22 -0800 Subject: Null restricted type and JSpecify In-Reply-To: <1508664657.51644988.1669208620662.JavaMail.zimbra@u-pem.fr> References: <1508664657.51644988.1669208620662.JavaMail.zimbra@u-pem.fr> Message-ID: There were some offline musings related to this but no actual "proposal". Not sure what Brian will come up with. I'm very glad for more awareness of our work obviously... but I don't want to overstate how directly applicable it is to Valhalla. The relationship between the two could be more complex; I'm only interested that we can see some path toward future unification, and we can convince ourselves we're not stepping off that path. On Wed, Nov 23, 2022 at 5:04 AM Remi Forax wrote: - there is no @NonNull, a lack of @Nullable inside a @NullMarked scope > encodes the fact that a type is non null. (Well, we did eventually add it; the javadoc explains why. Sorry the spec isn't current. The javadoc and design faq are much more so.) JSpecify is specified in term of subtyping relationship not in term of > boxing/unboxing rules. >From our reading of the JLS, this was just the most convenient way to do it. If it goes wrong somehow, we missed that. -- Kevin Bourrillion | Java Librarian | Google, Inc. | kevinb at google.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Wed Nov 30 13:14:33 2022 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 30 Nov 2022 14:14:33 +0100 (CET) Subject: Null restricted type and JSpecify In-Reply-To: References: <1508664657.51644988.1669208620662.JavaMail.zimbra@u-pem.fr> Message-ID: <74421246.56408925.1669814073387.JavaMail.zimbra@u-pem.fr> > From: "Kevin Bourrillion" > To: "Remi Forax" > Cc: "valhalla-spec-experts" > Sent: Wednesday, November 23, 2022 8:23:22 PM > Subject: Re: Null restricted type and JSpecify > There were some offline musings related to this but no actual "proposal". Not > sure what Brian will come up with. yes, sorry, > I'm very glad for more awareness of our work obviously... but I don't want to > overstate how directly applicable it is to Valhalla. The relationship between > the two could be more complex; I'm only interested that we can see some path > toward future unification, and we can convince ourselves we're not stepping off > that path. That was my idea too, the question is not to make JSpecify be to driver of Valhalla or whatever but to be sure that Vahalla does not impede the existence of JSpecify. > On Wed, Nov 23, 2022 at 5:04 AM Remi Forax < [ mailto:forax at univ-mlv.fr | > forax at univ-mlv.fr ] > wrote: >> - there is no @NonNull, a lack of @Nullable inside a @NullMarked scope encodes >> the fact that a type is non null. > (Well, we did eventually add it; the [ > https://jspecify.dev/docs/api/org/jspecify/annotations/NonNull.html | javadoc ] > explains why. Sorry the spec isn't current. The javadoc and design faq are much > more so.) I see, for the case where a user want a non null type argument of a parameterized type declared with a nullable type parameter. >> JSpecify is specified in term of subtyping relationship not in term of >> boxing/unboxing rules. > From our reading of the JLS, this was just the most convenient way to do it. If > it goes wrong somehow, we missed that. I think i prefer the boxing relationship, even ignoring Valhalla, because it allows unsafe casting by example from a List<@Nullable T> to a List<@NonNull T> and vice versa. So adding a @Nullable / @NonNull is something that can always be done afterward to fix a mistake in how the code was nullified (== generified for null). R?mi -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.smith at oracle.com Wed Nov 30 15:30:21 2022 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 30 Nov 2022 15:30:21 +0000 Subject: EG meeting, 2022-11-30 Message-ID: <7854C67A-4677-41CA-AE6E-89539082E50A@oracle.com> EG Zoom meeting November 30 at 5pm UTC (9am PDT, 12pm EDT). At Oracle we're still noodling on null-free types as a way to enable flattening. A lot of details are still uncertain. So there may not be much to talk about, but we'll at least have a quick check in and see if there are followup questions from last time that can be addressed. From john.r.rose at oracle.com Wed Nov 30 17:01:42 2022 From: john.r.rose at oracle.com (John Rose) Date: Wed, 30 Nov 2022 09:01:42 -0800 Subject: Null restricted type and JSpecify In-Reply-To: <1508664657.51644988.1669208620662.JavaMail.zimbra@u-pem.fr> References: <1508664657.51644988.1669208620662.JavaMail.zimbra@u-pem.fr> Message-ID: Kevin & co won?t like this maybe, but I think this design and our Valhalla design do not conflict, precisely because they make completely distinct sets of assertions, and yet the assertions (taken together) are consistent. What we are talking about here is X! on non-type-vars. That means we are adding a state EXPLICIT_MINUS_NULL, which (a) has an obvious relation to the other states in JSpec., and (b) can have a semantic weight different from anything in JSpec.; it can and should translate to a different Q-descriptor. On 23 Nov 2022, at 5:03, Remi Forax wrote: > Hi all, > i've taken a look to JSpecify [1] to see how it works with the recent proposal from Brian, seen value types as null restricted types. > > Obviously, i'm not an expert in the details of JSpecify so feel free to correct me. > > I really like JSpecify, the specification is simple and lean but yet at the same time guides users towards a null safe Java. > > JSpecify specifies two annotations, @NullMarked and @Nullable. @NullMarked (on a class, a package or a module) says that any types in that scope is not null (NO_CHANGE) by default. @Nullable indicates that a type is nullable (UNION_NULL), inside a @NullMarked scope or not. > A type outside the the @NullMarked scope and not marked as @Nullable is unspecified (UNSPECIFIED) [2]. > > Compared to the null restricted type proposal, there are IMO two major differences, > - the fact that JSpecify uses annotations implies that adding nullness information is a binary backward compatible change. > - there is no @NonNull, a lack of @Nullable inside a @NullMarked scope encodes the fact that a type is non null. > > The former suggests that we should investigate erasing Q-type to L-type inside method/field descriptors. The later suggests that either we should embrace this approach too and only provides '?' or we should allow both '!' (NO_CHANGE) and '?' (UNION_NULL) but it makes the transition from actual Java to a null safe Java less clear. > > The other difference, which was noted by Kevin during our meeting last week, JSpecify is specified in term of subtyping relationship not in term of boxing/unboxing rules. For me, boxing/unboxing rules are less specific that subtyping in assignments but more specific in case of overriding, so i've no idea if this is an issue or not. > > regards, > R?mi > > [1] https://jspecify.dev/ > [2] there is a 4th state which is MINUS_NULL but it's for type variable, so not something which is directly related to value types. From forax at univ-mlv.fr Wed Nov 30 17:06:49 2022 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 30 Nov 2022 18:06:49 +0100 (CET) Subject: Null restricted type and JSpecify In-Reply-To: References: <1508664657.51644988.1669208620662.JavaMail.zimbra@u-pem.fr> Message-ID: <1991372969.56691342.1669828009619.JavaMail.zimbra@u-pem.fr> ----- Original Message ----- > From: "John Rose" > To: "Remi Forax" > Cc: "valhalla-spec-experts" > Sent: Wednesday, November 30, 2022 6:01:42 PM > Subject: Re: Null restricted type and JSpecify > Kevin & co won?t like this maybe, but I think this design and our Valhalla > design do not conflict, precisely because they make completely distinct sets of > assertions, and yet the assertions (taken together) are consistent. > > What we are talking about here is X! on non-type-vars. That means we are adding > a state EXPLICIT_MINUS_NULL, which (a) has an obvious relation to the other > states in JSpec., and (b) can have a semantic weight different from anything in > JSpec.; it can and should translate to a different Q-descriptor. if it's L-descriptor in descriptors but Q-descriptor in TypeRestriction + bytecodes, yes ! I think being able to add a '!' if you have overlook one should be an option. R?mi > > On 23 Nov 2022, at 5:03, Remi Forax wrote: > >> Hi all, >> i've taken a look to JSpecify [1] to see how it works with the recent proposal >> from Brian, seen value types as null restricted types. >> >> Obviously, i'm not an expert in the details of JSpecify so feel free to correct >> me. >> >> I really like JSpecify, the specification is simple and lean but yet at the same >> time guides users towards a null safe Java. >> >> JSpecify specifies two annotations, @NullMarked and @Nullable. @NullMarked (on a >> class, a package or a module) says that any types in that scope is not null >> (NO_CHANGE) by default. @Nullable indicates that a type is nullable >> (UNION_NULL), inside a @NullMarked scope or not. >> A type outside the the @NullMarked scope and not marked as @Nullable is >> unspecified (UNSPECIFIED) [2]. >> >> Compared to the null restricted type proposal, there are IMO two major >> differences, >> - the fact that JSpecify uses annotations implies that adding nullness >> information is a binary backward compatible change. >> - there is no @NonNull, a lack of @Nullable inside a @NullMarked scope encodes >> the fact that a type is non null. >> >> The former suggests that we should investigate erasing Q-type to L-type inside >> method/field descriptors. The later suggests that either we should embrace this >> approach too and only provides '?' or we should allow both '!' (NO_CHANGE) and >> '?' (UNION_NULL) but it makes the transition from actual Java to a null safe >> Java less clear. >> >> The other difference, which was noted by Kevin during our meeting last week, >> JSpecify is specified in term of subtyping relationship not in term of >> boxing/unboxing rules. For me, boxing/unboxing rules are less specific that >> subtyping in assignments but more specific in case of overriding, so i've no >> idea if this is an issue or not. >> >> regards, >> R?mi >> >> [1] https://jspecify.dev/ >> [2] there is a 4th state which is MINUS_NULL but it's for type variable, so not > > something which is directly related to value types. From john.r.rose at oracle.com Wed Nov 30 18:21:24 2022 From: john.r.rose at oracle.com (John Rose) Date: Wed, 30 Nov 2022 10:21:24 -0800 Subject: Null restricted type and JSpecify In-Reply-To: <1991372969.56691342.1669828009619.JavaMail.zimbra@u-pem.fr> References: <1508664657.51644988.1669208620662.JavaMail.zimbra@u-pem.fr> <1991372969.56691342.1669828009619.JavaMail.zimbra@u-pem.fr> Message-ID: <26824341-F4F6-4FE4-8888-39C0F7CD1C2D@oracle.com> On 30 Nov 2022, at 9:06, forax at univ-mlv.fr wrote: > ----- Original Message ----- >> From: "John Rose" >> To: "Remi Forax" >> Cc: "valhalla-spec-experts" >> Sent: Wednesday, November 30, 2022 6:01:42 PM >> Subject: Re: Null restricted type and JSpecify > >> Kevin & co won?t like this maybe, but I think this design and our Valhalla >> design do not conflict, precisely because they make completely distinct sets of >> assertions, and yet the assertions (taken together) are consistent. >> >> What we are talking about here is X! on non-type-vars. That means we are adding >> a state EXPLICIT_MINUS_NULL, which (a) has an obvious relation to the other >> states in JSpec., and (b) can have a semantic weight different from anything in >> JSpec.; it can and should translate to a different Q-descriptor. Disclaimer! By the way, my use of X! doesn?t imply a particular bikeshed for the syntax. JSpec. spells it @NonNull, one draft of Valhalla spells it .val (as in Point.val), and ?everybody knows? what X! means, kind of. Except they don?t: Different languages give very different meanings (sometimes in very subtle details) to their emo-type syntaxes. There! > > if it's L-descriptor in descriptors but Q-descriptor in TypeRestriction + bytecodes, yes ! > > I think being able to add a '!' if you have overlook one should be an option. As we discussed in the EG just now, this is a distinct proposal which should be taken up independently of anything we do regarding JSpecify. At the VM level, we are planning to do (for generics) some sort of type restrictions which are semantically enforces but DO NOT appear in the descriptor; it is a side channel to the descriptor. So L-Point could be a translation of Point.val (aka Point! but see above) instead of Q-Point, as long as there is a type restriction lurking somewhere near that specific occurrence of L-Point. Technically we could do this at some level. But I?d like to challenge this proposal, by a counter-proposal that (I think) would first have to hold water, in order to translate Point.val as L-Point+TR. That counter proposal is, experimentally amend today?s Java translation strategy so that every use the descriptor I (for int) in APIs is changed to L-Integer plus a TR, so that (basically) int erases to Integer, with a TR. Suppose some reasonable JVM support for TRs of this form (just for primitive types, for now). Now, can you implement the JLS? Should you do it this way? For bonus points (as you suggested in the meeting) see if you can erase ALL TYPES to L-Object, with associated TRs. Does it work? Would it be smart? If these experiments would go wrong somehow, I suggest, strongly, that a similar voyage into erasure for Q-Point into L-Point+TR would also go wrong, in similar ways. For starters, some overloaded functions would be broken in either of those thought experiments. That should make us a little bit scared to do that same thing for the new Q-types. I suspect there are a number of ?gotchas? like this. We spent the first couple of years on the Valhalla project enumerating such gotchas, for various proposals. Generally speaking it has turned out that erasing Q to L seems OK about 80% of the way through the details, and then something jumps out and bites, related to details of the JLS, or JVM optimizations, or binary compatibility, or all three at the same time. ? John