From brian.goetz at oracle.com Thu Feb 6 17:53:25 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 6 Feb 2020 12:53:25 -0500 Subject: Superclasses for inline classes In-Reply-To: <145A4BBE-4972-4CEB-A149-95C4D6E5C519@oracle.com> References: <145A4BBE-4972-4CEB-A149-95C4D6E5C519@oracle.com> Message-ID: I've been thinking about this, and gone around in circles a few times.? The properties laid out here are understandable and easily checkable at both compile and run time.? (I might add "no synchronization", or at least "no synchronized methods".) I agree that having some sort of class-level modifier gives this concept too much importance.? And normally, I would take the position that a property like this would have to appear in the declaration somewhere, to capture the fact that conformance to these properties is not merely accidental and subject to change tomorrow.? But, lately I've been questioning this. Reasons to have some sort of indicator of "I am an initialization-free class": ?- Allows author to capture design intent ?- Allows compiler to type-check conformance to requirements, based on design intent ?- Propagates design intent into documentation, where subclasses can see that this is safe to extend But it occurs to me that all of these are within the bounds of what _annotations_ are allowed to do; we did the same with `@FunctionalInterface` that, while not required, turns on additional compile-time type checking and documentation. I am currently thinking that we can adopt Dan's rules at the language level, and have an annotation @InitializationFree, which: (a) declares intent to conform to the contract of initialization-freedom, (b) unleashes compile-time type checking for same, and (c) causes a blurb to be emitted into the Javadoc along the lines of "This is an initialization-free class" -- but that the language specification works entirely in terms of structural properties.? We can say that "inline classes may extend initialization-free inline classes", and be done. Regardless of compile-time type checking, we will likely want to do the analysis and write the result into the classfile (e.g., an InitializationFree class attribute), which can be seen as a necessary-but-not-sufficient signal for the VM to allow the class to be used as a super for an inline class.? When we go to load a class with this attribute, we can verify that it actually conforms: ?- Fields: no fields ?- Constructors/initializers: no ctor/initializer ?- Class declaration: it is abstract, and its super is i-free ?- Synchronization: No methods have ACC_SYCHRONIZED When we go to load an inline class, we check the i-free bit on the superclass. None of these checks seem all that burdensome to the VM. It does mean that dusty abstract classes need to be recompiled before they can be bases for inline classes, which, on balance, seems OK. > Language model > > An inline class may extend another class, as long as the superclass has the following properties: > - It has no instance fields > - It has no constructors > - It has no instance initializers > - It is abstract* or Object > - It extends another class with these properties > > Subtype polymorphism works the same for superclasses as it does for superinterfaces. > > (*Remi points out that we could drop the 'abstract' restriction, > meaning there may be identity instances of the superclass. Given the > restriction on fields, though, I'm struggling to envision a use case; > the consensus is that 'new Object()' is probably something we want to > *stop* supporting.) > > Call a class that satisfies these constraints an "initialization-free class" (bikeshedding on this term is welcome!). Like an interface, its value set may include references to both inline class instances and identity class instances. > > We*do *not* want the initialization-free property to be expressed as a class modifier?this feature is too obscure to deserve that much prominence, encouraging every class author to consider one more degree of freedom; and we don't want every class to have to manually opt in. > > But we*do* need the initialization-free property to be part of the public information about the class. For example, the javadoc should say something like "this is an initialization-free class". Otherwise, it's impossible to tell the difference between, e.g., a class with no fields and a class with private fields. > > In the past, a Java class declaration that lacks a constructor always got a default constructor. In this model, however, an initialization-free class has no constructor at all. A 'super()' call directed at such a class is a no-op. From brian.goetz at oracle.com Fri Feb 7 17:34:48 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 7 Feb 2020 12:34:48 -0500 Subject: Reference-default style In-Reply-To: <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> Message-ID: <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> I want to combine this discussion with the question about whether inline classes can extend abstract classes, and with the "reference projection" story in general. Our initial inclination was to say "no abstract class" supertypes (largely on the basis that interfaces are a cleaner way to extend contracts) and therefore that the reference projection would always be an interface.? This felt clean, but opened up a pile of questions, since there are all sorts of things that interfaces can't express (such as package-private methods.) For example, here's the sort of challenge this generates. Suppose we want to translate: ??? inline class C implements I { /* methods */ } as ??? // TRANSLATION A ??? ACC_INLINE class C implements C$ref, I, InlineObject { ??????? /* methods */ ??? } ??? sealed interface C$ref extends I permits C { ??????? /* abstract versions of the public methods */ ??? } where we lift the public methods onto the interface.? But what about the non-public methods?? We can't represent them in an interface.? Well, one trick we have in our bag is that, when we encounter the use of the ref projection as the receiver `ref.m()`, the compiler can cast it to the concrete type and invoke on that: `((C) ref).m()`.? This is safe because a value of the ref projection is either a value of the concrete class, or null; in both cases, the two yield identical results.? But, that's kind of yucky, not because the trick is inherently yucky, but because we're only doing it some of the time, which seems likely to cause other problems down the road.? (Also, this trick depends on the inline and reference classes being equally accessible. That's not the case in the current draft, but there's a way to get there, let's put a pin in that.) If we assume that, _at least at the VM level_, inline classes can extend suitable abstract classes, this offers us an alternate translation story: the reference projection is an abstract class (one that is still sealed only to permit the inline class.)? Then this anomaly goes away, which is nice: // TRANSLATION B ??? ACC_INLINE class C extends C$ref implements I, InlineObject { ??????? /* methods */ ??? } ??? sealed abstract class C$ref implements I permits C { ??????? /* abstract versions of the methods */ ??? } This avoids the asymmetry, but now, here comes John to say "Yuck, you're repeating yourself.? Why lift the methods to the interface at all?"? Which brings us to the philosophical question Dan raises, which is: what is the reference projection, really?? Is it a fundamental part of the language model, or something that is just a convenience, to be produced by compiler sugar and/or design patterns? The reference projection is largely a _language fiction_, like generics or checked exceptions.? (At the VM level, `QC <: LObject`; this is what makes our inline narrowing and widening conversions cheap.)? The more we can make these two types work together, the better.? (Sealing helps; nestmates help; we can do more.)? John would say: let's make the abstract class _empty_: ??? // TRANSLATION C ??? sealed abstract class C$ref implements I permits C { ??????? /* no methods */ ??? } and then handle _all_ uses of C$ref as a receiver by casting to C.? This is more uniform, and therefore less error-prone -- and is more in line (heh, inline) with its role as a language fiction. Let's put this down for a minute and talk about reference-default.? Dan makes the argument that migration is not the only reason why we might want reference-default inline classes.? In the model currently on the table, we have an uneasy combination of "if you want inline default, you get nice language sugar, if you want reference-default, you code it yourself and the language tries to guess at what you mean."? Dan's (3), which lets you declare inline classes one way and pick which way the "good name" goes seems a lot simpler: ??? // C and C.val refer to the same class, C ??? // compiler generates C and C$ref ??? /* val-default */ inline class C { } ??? // D and D.ref refer to the same class, D // compiler generates D and D$val ref-default inline class D { } This allows us to bring the reference projection closer to the inline class uniformly, always generating them both.? This eliminates potential anomalies and guesswork, and also eliminates any chance that the ref and val flavors could have different accessibilities, allowing the cast trick to work uniformly.? This seems a significant consolidation; every inline class has a ref projection; the ref projection is always generated at the same time as the val; we can safely assume useful relationships between them.? (It is worth noting that migrating between ref-default and val-default will not be a compatible move, so choose wisely.) OK, now let's come back to translation.? A primary use case, but not the only, for ref-default is migration. In this case, we know that there will be classfiles out there that do `invokevirtual C.foo()`.? And under translation (C), these won't work.? We can refine our translation accordingly, where C is a val-default inline class and D is ref-default: ??? // TRANSLATION D ACC_INLINE class C extends C$ref implements I, InlineObject { ??????? /* fields and methods */ ??? } ??? sealed abstract class C$ref implements I permits C { ??????? /* nothing */ ??? } ACC_INLINE class D$val extends D implements I, InlineObject { ??????? /* fields, but no methods */ ??? } ??? sealed abstract class C$ref implements I permits C { ??????? /* methods (which cast D to D$val internally to access fields) */ ??? } This means that the fields will always be on the val class, and the methods will always be on the _default_ projection. Alternately, we can pick translation (E), where we _always_ put the fields on the val class and _always_ put the methods on the reference projection, and the methods are just inherited by the val projection.? Or, translation (F), where for ref-default we duplicate the methods onto the ref projection. So, summary: ?- Yes, we should figure out how to support abstract class supertypes of inline classes, if only at the VM level; ?- There should be one way to declare an inline class, with a modifier saying which projection gets the good name; ?- Both the ref and val projections should have the same accessibility, in part so that the compiler can freely use inline widening/narrowing as convenient; ?- We would prefer to avoid duplication of the methods on both projections, where possible; ?- The migration case requires that, for ref-default inline classes, we translate so that the methods appear on the ref projection. On 12/20/2019 3:04 PM, Brian Goetz wrote: > >> 1) As a design pattern > > This was the strawman starting point, shortly after the JVMLS meeting, > which kicked off the "eclair" notion.? While this one seems like "the > simplest thing that could work", it strikes me as too simple. > > When some version of this approach was floated much earlier, Stephen > commented "I'm not looking forward to making up new names for the > inline flavor of LocalDateTime and friends."? I share this concern, > but 100x so on behalf of the clients -- I don't want to force clients > to have to keep a mental database of "what is the inline flavor of > this called."? So I think its basically a forced move that there is > some mechanical way to say "the other flavor of T". > > > Several folks have come out vocally in favor of the Foo / foo naming > convention, which could conceivably satisfy this requirement.? But, I > see this as a move we will likely come to regret.? (Among other > things, there goes our source of conditional keywords, forever.? On > its own, that's a lot of damage to the future evolution of the language.) > > > The "mechanical way to describe the reference > companion/projection/pair/whatever" becomes even stronger when we get > to specialized generics, as we'll need to be able to say `T.ref` for a > type variable `T` (this is, for example, the return type of > `Map::get`.)? The other direction is plausible too (when `T extends > InlineObject`), though I don't have compelling examples of this in > mind right now, so its possible that this is only a one-way requirement. > >> 2) As an "advanced" feature of inline classes >> >> This is the State of Valhalla strategy: inline classes are designed >> to be inline-default, but as a special-case feature, you can also >> declare the 'Foo.ref' interface, give it a name, and wire it up to >> the inline class declaration. >> >> In reference-default style, the programmer gives the "good name" to >> the reference projection, and either gives an alternate name to the >> inline class or is able to elide it entirely (in that case, clients >> use 'Foo.inline'). >> >> Ways this is different than (1): >> - The 'Foo.inline' type operator >> - Implicit conversions (although sealed types can get us there in (1)) >> - There are two types, not three (and two JVM classes, not three) >> - Opportunities for "boilerplate reduction" in the two declarations > > Much of the generality of (2) comes from the goals of migrating > primitives to just be declared classes, while retaining the spelling > `Integer` for the ref projection, and not having _two_ box types. If > we're willing to special-case the primitives, then we may be able to > do better here. > >> 3) As an equal partner with inline-default >> >> An inline class declaration introduces two types, an inline type and >> a reference type. But a modifier on the declaration determines >> whether the "good name" goes to the inline type or the reference >> type. The other type can be derived using an operator ('Foo.ref' or >> 'Foo.inline'). There's never a need for an alternate name. >> >> In this case, the language isn't biased to one style or the other; >> each declaration picks one. The trade-off is that clients need to >> keep track of one more bit when thinking about the inline class ("Is >> this a *foo* inline class or a *bar* inline class?" Actual >> terminology to be bikeshedded...) > > In a previous iteration, we had an LV/QV duality at the VM level, > which corresponded to a null-default/zero-default duality at the > language level.? We hated both of these (too much complexity for too > little gain), so we ditched them.? What you're proposing is to > reintroduce a new duality, `ref-default` vs `inline-default`, which > would arbitrate custody of "the good name". > > What I like about this is that _both_ `Foo.ref` and `Foo.inline` > become true projections from the class declaration Foo; there's no > "write a bunch of classes and wire up their relationship". (Though > some degree of special pleading and auto-wiring would be needed for > primitives, which seems like it is probably acceptable.)? It is a more > principled position, and not actually all that different in practice > from (2), in that the default is still inline. > > What I don't like is that (a) the author has to pick a polarity at > development time (and therefore can pick wrong), and (b) to the extent > ref-default is common, the client now has to maintain a mental > database of the polarity of every inline class, and (c) if the > polarity is not effectively a forced move (as in (2), where we only > use it for migration), switching polarities will (at least) not be > binary compatible.? So the early choice (made with the least > information) is permanent. From a user perspective, we are introducing > _two_ new kinds of top level abstractions; in (2), we are introducing > one, and leaning on interfaces/abstract classes for the other.? On the > other other hand, having more ref-default classes than the migrated > ones will make `.inline` stick out less. > > > Do we want to step back away from the experiment that is `inline`, and > go back to `Foo.ref` and `Foo.val`?? If we're looking to level the > playing field, giving them equally fussy/unfussy names is a leveler... > > > >> 4) As the only supported style >> >> An inline class declaration always gives the "good name" to the >> reference type, and you always use an operator to get to the inline >> type ('Foo.inline'?but we're gonna need better syntax.) >> >> This one would represent a significant shift in the design center of >> the feature. If you want flattening everywhere, you're going to need >> to make liberal use of the '.inline' operator. But if you just want >> to declare that a bunch of your classes don't have identity, and >> hopefully get a cheap performance boost as a result, it's simple. The >> burden of learning something new is shifted to "advanced" users and >> APIs to whom flattening is important. > > I can't really see this being a winner. > >> Conclusion: >> >> I'm not ready to completely dismiss any of these designs, but my >> preferences at the moment are (1) and (3). Options (4) and (5) are >> more ambitious, discarding some of our assumptions and taking things >> in a different direction. >> >> Like many design patterns, (1) suffers from boilerplate overhead ((2) >> too, without some language help). It also risks some missed >> opportunities for optimization or language convenience, because the >> relationship between the inline and reference type is incidental. >> (I'd like to get a clearer picture of whether this really matters or >> not.) > > The main knock on (1) is that it leans on an ad-hoc convention, and to > the extent this convention is not universally adhered to, user > confusion abounds.? (Think about how many brain cycles you've spent > being even mildly miffed that the box for `long` is `Long` but the box > for `char` is `Character`.? If it's more than zero, that's a waste of > cycles.) > > I really have a hard time seeing (1) as leading where we want. > >> (5) feels like something fundamentally new in Java, although if you >> squint it's "just" a variation on name resolution. What originally >> prompted this idea was seeing a similar approach in attempts to >> introduce nullability type operators?legacy code has the "wrong" >> default, so you need some lightweight way to pick a different default. > (5) could be achieved with another long-standing requests, aliased > imports: > > ??? import Foo.inline as Foo; > > Not saying that makes it better, but a lot of people sort of want > import to work this way anyway. > > From brian.goetz at oracle.com Fri Feb 7 22:05:35 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 7 Feb 2020 17:05:35 -0500 Subject: Reference-default style In-Reply-To: <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> Message-ID: <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> > > So, summary: > > ?- Yes, we should figure out how to support abstract class supertypes > of inline classes, if only at the VM level; > ?- There should be one way to declare an inline class, with a modifier > saying which projection gets the good name; > ?- Both the ref and val projections should have the same > accessibility, in part so that the compiler can freely use inline > widening/narrowing as convenient; > ?- We would prefer to avoid duplication of the methods on both > projections, where possible; > ?- The migration case requires that, for ref-default inline classes, > we translate so that the methods appear on the ref projection. Let me flesh this out some more, since the previous mail was a bit of a winding lead-up. #### Abstract class supertypes It is desirable, both for the migration story and for the language in general, for inline classes to be able to extend abstract classes.? There are restrictions: no fields, no constructors, no instance initializers, no synchronized methods.? These can be checked by the compiler at compile time, and need to be re-checked by the VM at load time. The VM folks have indicated that their preferred way to say "inline-friendly abstract class" is to have only a no-arg constructor which is ACC_ABSTRACT.? For abstract classes that meet the inline-friendly requirement, the static compiler can replace the default constructor we generate now with an abstract one.? The VM would have to be able to deal with subclasses doing `invokespecial ` super-calls on these. My current bikeshed preference for how to indicate these is to do just the test structurally, with good error messages, and back it up with annotation support similar to `@FunctionalInterface` that turns on stricter type checking and documentation support.? (The case we would worry about, which stronger declaration-site indication would help with, would be: a public accidentally-inline-friendly abstract class in one maintenance domain, extended by an inline class in another maintenance domain, and then subsequently the abstract class is modified to, say, add a field.? This could happen, but likely would not happen that often; we can warn users of the risks by additionally issuing a warning on the subclass when the superclass is not marked with the annotation.) #### Val and ref projections An inline class `C` declaration defines three types; `C`, `C.ref`, and `C.val`.? One of the latter two is always an alias for the first.? `C.ref` is always a reference type, and we extend the `.ref` operator to extend to all classes (thus allowing us to speak of `T.ref` in generic code.) Which of the two projections (.ref and .val) is aliased to C is determined by a modifier, either `ref-default` or `val-default` (the latter being the, er, default.) The three types are defined to have the same members, type variables, and accessibility; the only differences between `C.ref` and `C.val` are: ?- The value set of `C.val` consists of values, whereas the value set of `C.ref` consists of references to values, and includes null; ?- `C.ref <: Object` and `C.ref <: I` for all implemented interfaces `I`. ?- `C.val` can be _converted_ to `Object` or `I` via an _inline widening conversion_ (but, this conversion is cheaper than you might think.) ?- Variables of type `C.val` are routinely flattened by the VM, whereas variables of type `C.ref` are not. #### Translation -- classfiles A val-default inline class `C` is translated to two classfiles, `C` (val projection) and `C$ref` (ref projection).? A ref-default inline class `D` is translated to two classfiles, `D` (ref projection) and `D$val` (val projection), as follows: ?- The two classfiles are members of the same nest. ?- The ref projection is a sealed abstract class that permits only the val projection. ?- Instance fields are lifted onto the val projection. ?- Supertypes, methods (including static methods, and including the static "constructor" factory), and static fields are lifted onto the ref projection.? Method bodies may internally require downcasting to `C.val` to access fields. #### Translation -- uses Variables of type `C.ref` are translated as L types (`LC` or `LC$ref`, depending); variables of type `C.val` are translated as Q types (`QC` or `QC$val`, depending.) `C.val` is widened to `C.ref` by direct assignment, since in the VM, an inline class is related to its supertypes by subtyping. `C.ref` is narrowed to `C.val` by casting, which the VM can optimize to a null check. Instance field accesses on `C.val` are translated to `getfield`; field accesses on `C.ref` are translated by casting to `C.val` and `getfield`. Method invocation on `C.val` or `C.ref` can be translated directly, except for private methods, which would require casting `C.val` to `C.ref` first (not because they are inaccessible, but because they are not inherited.)? Same for static fields. Conversion of `C.ref` to supertypes is ordinary subtyping; conversion of `C.val` goes through widening to `C.ref`. Similarly, `instanceof` on an operand of type `C.val` goes through casting to `C.ref`. There are other stackings, of course, but this is a starting point, chosen for simplicity and compatibility. From daniel.smith at oracle.com Fri Feb 7 22:39:08 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Fri, 7 Feb 2020 15:39:08 -0700 Subject: Reference-default style In-Reply-To: <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> Message-ID: <9A0B3673-49C5-4D49-ACBE-E6C7B80CA290@oracle.com> > On Feb 7, 2020, at 3:05 PM, Brian Goetz wrote: > > - Supertypes, methods (including static methods, and including the static "constructor" factory), and static fields are lifted onto the ref projection. This deserves highlighting: in theory, there's nothing wrong with putting all of an inline class's methods in an abstract superclass. In practice, I'm curious to know whether we take a performance hit, and if so whether it's straightforward to optimize away. (And if there *is* a significant performance penalty, is it so bad that it's impractical to *ever* make serious use of superclass or superinterface methods in performance-sensitive contexts? That may be okay, but it's something authors will want to understand.) I also kind of wonder if there are some code paths in Hotspot that will never get seriously exercised if javac never invokes a method declared in an inline class. (Not really an argument against this strategy, just an observation.) From john.r.rose at oracle.com Fri Feb 7 23:17:53 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 7 Feb 2020 15:17:53 -0800 Subject: Reference-default style In-Reply-To: <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> Message-ID: On Feb 7, 2020, at 2:05 PM, Brian Goetz wrote: >> >> So, summary: >> >> - Yes, we should figure out how to support abstract class supertypes of inline classes, if only at the VM level; >> - There should be one way to declare an inline class, with a modifier saying which projection gets the good name; >> - Both the ref and val projections should have the same accessibility, in part so that the compiler can freely use inline widening/narrowing as convenient; >> - We would prefer to avoid duplication of the methods on both projections, where possible; >> - The migration case requires that, for ref-default inline classes, we translate so that the methods appear on the ref projection. Abstract classes, check. User control over good name, check. Co-accessibility of both projections, check. No schema duplication, check. Methods on ref projection for migration, check. Awesome! I?m relieved that we are embracing abstract classes, because (a) the JVM processes them a little more easily than interfaces, and (b) they have fewer nit-picky limitations than interfaces (toString/equals/hashCode, package access members). Thanks, Dan and whoever else agitated for abstract classes; the JVM thanks you. I have a tiny reservation about the co-accessibility of both projections, although it?s a good principle overall. There might be cases (migration and maybe new code) where the nullable type has wider access than the inline type, where the type?s contract somehow embraces nullability to the extent that the .val projection is invisible. But we can cross that bridge when and if we come to it; I can?t think of compelling examples. > Let me flesh this out some more, since the previous mail was a bit of a winding lead-up. > > #### Abstract class supertypes > > It is desirable, both for the migration story and for the language in general, for inline classes to be able to extend abstract classes. There are restrictions: no fields, no constructors, no instance initializers, no synchronized methods. These can be checked by the compiler at compile time, and need to be re-checked by the VM at load time. (Nitpick: The JVM *fully* checks synchronization of such things dynamically; it cannot fully check at load time. Given that, it is not a good idea to partially check for evidence of synchronization; that just creates the semblance of an invariant where one does not exist. The JVM tries hard to make static checks that actually prove things, rather than just ?catch user errors?. So, please, no JVM load-time checks for synchronized methods, except *maybe* within the inline classes themselves.) > The VM folks have indicated that their preferred way to say "inline-friendly abstract class" is to have only a no-arg constructor which is ACC_ABSTRACT. For abstract classes that meet the inline-friendly requirement, the static compiler can replace the default constructor we generate now with an abstract one. The VM would have to be able to deal with subclasses doing `invokespecial ` super-calls on these. More info, from a JVM perspective: In that case, and that case alone, the JVM would validly look up the superclass chain for a non-abstract method, and link to that instead. This is a very special case of inheritance where a constructor is inherited and used as-is, rather than wrapped by a subclass constructor. It?s a valid operation precisely because the abstract constructor is provably a no-op. The Object constructor is the initial point of this inheritance process, and the end of the upward search. I?m leaning towards keeping that as non-abstract, both for compatibility, and as a physical landing place for the upward search past abstract constructors. For inlines, we say that the inline class constructor is required to inherit the Object constructor, with no non-abstract constructors in intervening supers, and furthermore that the JVM is allowed to omit the call to the Object constructor. This amounts to a special pleading that ?everybody knows Object. does nothing?. Actually in HotSpot it does something: For a class with a finalizer it registers something somewhere. But that?s precisely irrelevant to inlines. > > My current bikeshed preference for how to indicate these is to do just the test structurally, with good error messages, and back it up with annotation support similar to `@FunctionalInterface` that turns on stricter type checking and documentation support. (The case we would worry about, which stronger declaration-site indication would help with, would be: a public accidentally-inline-friendly abstract class in one maintenance domain, extended by an inline class in another maintenance domain, and then subsequently the abstract class is modified to, say, add a field. This could happen, but likely would not happen that often; we can warn users of the risks by additionally issuing a warning on the subclass when the superclass is not marked with the annotation.) That seems OK, even under restrictions about the effects of annotations. Annotations which cause the compiler to exit with an error don?t change the runtime semantics. And then the translation strategy can say: ?I?ve got a new trick up my sleeve! If the constructor is truly empty, with just a delegating call to my super , then I can express this condition as an abstract constructor, rather than some classfile boilerplate.? As a JVM person, I?m always itchy when somebody pours boilerplate into classfiles. Maybe I need to write a ?boilerplate considered harmful? manifesto about classfiles and translation strategies. > #### Val and ref projections > > ? (Yay!) > #### Translation -- classfiles > > A val-default inline class `C` is translated to two classfiles, `C` (val projection) and `C$ref` (ref projection). A ref-default inline class `D` is translated to two classfiles, `D` (ref projection) and `D$val` (val projection), as follows: > > - The two classfiles are members of the same nest. > - The ref projection is a sealed abstract class that permits only the val projection. > - Instance fields are lifted onto the val projection. > - Supertypes, methods (including static methods, and including the static "constructor" factory), and static fields are lifted onto the ref projection. Method bodies may internally require downcasting to `C.val` to access fields. This is a little like MVT, in that inline classes end up containing very little other than fields. This is the right move, IMO, for migrated classes. Hollowing out *all* inline classes strikes me as over-rotation for the sake of migration. I see how it allows both cases to have the same translation strategy, *except for the name*. That?s a pleasing property on paper. Maybe I can get used to it, but I?m uncomfortable with loading everything into the ref class even in the val-default case. I?d prefer (if consistency were not an issue) to make the ref class be completely empty (except for an abstract constructor), just like a marker interface, for the common case of a val-default. In the case of reflection, I think we can afford to show a consistent view for both kinds of inlines, by making all fields and methods appear on both projections. In other words, core reflection doesn?t require you to hunt around through both projections to find some API point; all API points are present on both projections. Does anybody see a downside to that? If we put API points just where they appear in the classfile, then people have to hunt around, which is bad, since it?s a translation strategy option which might conceivably change. If we put API points only on the val projection, legacy code will fail for migrated classes. If we put API points only on the ref projection, then users of val-default classes will be always fumbling around to fetch the ref projection when they reflect API points. So reflecting everything in both places looks OK to me. If we support non-sealed abstract supers of inlines (records!) then the hack on core reflection should copy the API points from the inline *only* if the super is sealed uniquely to the inline. > #### Translation -- uses > > Variables of type `C.ref` are translated as L types (`LC` or `LC$ref`, depending); variables of type `C.val` are translated as Q types (`QC` or `QC$val`, depending.) The Q-descriptor gives a necessary and sufficient signal to the JVM to load the inline class and determine its layout. The JVM is free to reject QR; where R fails to be an inline class, and the JVM is free to treat LV;, where V is an inline class, as an ill-defined descriptor, like L__noSuchClass;. (Note that the JVM does not *reject* ill-defined descriptors; it?s physically impossible, except in special cases like the resolution of C_MethodType. Resolving a C_MT of ()LV; should fail, though, if V is an inline. It was the compiler?s responsibility to say ()QV; in such a case.) > `C.val` is widened to `C.ref` by direct assignment, since in the VM, an inline class is related to its supertypes by subtyping. `C.ref` is narrowed to `C.val` by casting, which the VM can optimize to a null check. +1 > Instance field accesses on `C.val` are translated to `getfield`; field accesses on `C.ref` are translated by casting to `C.val` and `getfield`. +1 Construction requests on either type are translated to calls to a factory C.val::. > Method invocation on `C.val` or `C.ref` can be translated directly, except for private methods, which would require casting `C.val` to `C.ref` first (not because they are inaccessible, but because they are not inherited.) Same for static fields. +0.5; I see this is a place where consistency pays off, just a little, but I?m still annoyed that the ref class gets all the members except fields and constructors. If we flip the other way, then it?s like this: Method invocation on `C.val` or `C.ref` can be translated directly, except for private methods, which may require casting first *to the class holding the method* (not because they are inaccessible, but because they are not inherited.) Same for static fields. This to say ?the class holding the method? instead of ?the C.ref?, we preserve the immediate goal of supporting migration of Optional etc., but we incur some migration debt, because it?s harder to move from val-default to ref-default. This, I think, is best fixed by adding auto-bridging of some sort later, rather than over-rotating towards the migration case right now. (Did I miss some other reason for putting everything on C.ref?) > Conversion of `C.ref` to supertypes is ordinary subtyping; conversion of `C.val` goes through widening to `C.ref`. Similarly, `instanceof` on an operand of type `C.val` goes through casting to `C.ref`. Casting (actually, unboxing) conversion of C.ref to C.val is a regular checkcast. Conversion (via cast or anything else) of C.val to C.ref is a no-op. Instanceof never needs a checkcast, because the JVM treats the operand of instanceof as an untyped reference; there?s nothing new here for instanceof. > There are other stackings, of course, but this is a starting point, chosen for simplicity and compatibility. I like it, very very much, with the one reservation harped on above. ? John From john.r.rose at oracle.com Fri Feb 7 23:26:17 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 7 Feb 2020 15:26:17 -0800 Subject: Reference-default style In-Reply-To: <9A0B3673-49C5-4D49-ACBE-E6C7B80CA290@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> <9A0B3673-49C5-4D49-ACBE-E6C7B80CA290@oracle.com> Message-ID: <9C41E9F3-2A61-498D-8B20-700CC953EAF3@oracle.com> On Feb 7, 2020, at 2:39 PM, Dan Smith wrote: > >> On Feb 7, 2020, at 3:05 PM, Brian Goetz wrote: >> >> - Supertypes, methods (including static methods, and including the static "constructor" factory), and static fields are lifted onto the ref projection. > > This deserves highlighting: in theory, there's nothing wrong with putting all of an inline class's methods in an abstract superclass. In practice, I'm curious to know whether we take a performance hit, and if so whether it's straightforward to optimize away. > > (And if there *is* a significant performance penalty, is it so bad that it's impractical to *ever* make serious use of superclass or superinterface methods in performance-sensitive contexts? That may be okay, but it's something authors will want to understand.) Putting something in a surprising place has surprising penalties, sometimes. But, the JVM is designed from the ground up to manage single inheritance code sharing extremely well. I don?t think there?s much of a performance penalty here for putting code on a superclass rather than a subclass. That said, putting code in a surprising place sometimes leads to surprises. Using interface default methods is significantly riskier, as I implied in my previous comment. > I also kind of wonder if there are some code paths in Hotspot that will never get seriously exercised if javac never invokes a method declared in an inline class. (Not really an argument against this strategy, just an observation.) Good observation. We could see ?surprises?. Basically, the two classfiles are logically one compilation unit, so it should be OK to put anything from the source file in either half-bucket. (To remind everyone: We are using two half-buckets rather than one bucket mainly so that Optional can be migrated. If it were just supporting V? then we?d use an empty marker type, I think, probably just an interface.) I?m not sure exactly why I feel queasy about hollowing out the V.ref, and that?s probably because my ?anti-surprise heuristic? (aka. spidey sense) is responsible, and such heuristics can?t every say much more than ?I?ve got a bad feeling about this?. But a rarely used code path is one door that surprises can pop through. :-S ? John From john.r.rose at oracle.com Fri Feb 7 23:36:16 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 7 Feb 2020 15:36:16 -0800 Subject: Reference-default style In-Reply-To: <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> Message-ID: (Replying second to your first.) On Feb 7, 2020, at 9:34 AM, Brian Goetz wrote: > > I want to combine this discussion with the question about whether inline classes can extend abstract classes, and with the "reference projection" story in general. Thanks for your ?John would say??; I?ll take those as already said! The only other thing is I guess I prefer your Translation D over Translation E, and either of those over the other options. That is, given two half-buckets, put every API chunk in exactly one half, and put most of the chunks in the side of the bucket with the good name. (And have Core Reflection copy all chunks to both names.) ? John From brian.goetz at oracle.com Fri Feb 7 23:37:42 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 7 Feb 2020 18:37:42 -0500 Subject: Reference-default style In-Reply-To: References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> Message-ID: <897b2f37-6eb3-7c05-1c1b-e55dad68dd92@oracle.com> > I have a tiny reservation about the co-accessibility of both projections, > although it?s a good principle overall. There might be cases (migration > and maybe new code) where the nullable type has wider access than > the inline type, where the type?s contract somehow embraces nullability > to the extent that the .val projection is invisible. But we can cross that > bridge when and if we come to it; I can?t think of compelling examples. I was worried about this too, wanting to support patterns like "public interface, private implementation."? But then I realized I was thinking like a compiler rather than a programmer!? If we want that, we can just write: ??? public interface Foo { ... } ??? private inline class FooImpl implements Foo { ... } just like we always did. > > (Nitpick: The JVM *fully* checks synchronization of such things dynamically; > it cannot fully check at load time. Given that, it is not a good idea to partially > check for evidence of synchronization; that just creates the semblance of an > invariant where one does not exist. The JVM tries hard to make static checks > that actually prove things, rather than just ?catch user errors?. So, please, > no JVM load-time checks for synchronized methods, except *maybe* within > the inline classes themselves.) Sure, that's your call.? The static compiler can do its own thing. > >> #### Translation -- classfiles >> >> A val-default inline class `C` is translated to two classfiles, `C` (val projection) and `C$ref` (ref projection). A ref-default inline class `D` is translated to two classfiles, `D` (ref projection) and `D$val` (val projection), as follows: >> >> - The two classfiles are members of the same nest. >> - The ref projection is a sealed abstract class that permits only the val projection. >> - Instance fields are lifted onto the val projection. >> - Supertypes, methods (including static methods, and including the static "constructor" factory), and static fields are lifted onto the ref projection. Method bodies may internally require downcasting to `C.val` to access fields. > This is a little like MVT, in that inline classes end up containing very little > other than fields. This is the right move, IMO, for migrated classes. > > Hollowing out *all* inline classes strikes me as over-rotation for the sake > of migration. I see how it allows both cases to have the same translation > strategy, *except for the name*. That?s a pleasing property on paper. It's more than a pleasing property (though it is that.)? Having translation strategies with big switches in them is where surprising migration compatibility constraints come from; would we want (say) changing a method from private to public to be a binary-incompatible change?? These are the kinds of things that happen when the translation strategy gets too squirrely and ad-hoc. > In the case of reflection, I think we can afford to show a consistent view > for both kinds of inlines, by making all fields and methods appear on > both projections. The language does this; while reflection is usually classfile-based, we'd like to treat these two classfiles as being mostly one artifact, so this seems reasonable to consider. > This to say ?the class holding the method? instead of ?the C.ref?, we preserve > the immediate goal of supporting migration of Optional etc., but we incur > some migration debt, because it?s harder to move from val-default to ref-default. > This, I think, is best fixed by adding auto-bridging of some sort later, rather > than over-rotating towards the migration case right now. > > (Did I miss some other reason for putting everything on C.ref?) Putting stuff on C.ref means we can lean harder on inheritance/subtyping in the translation scheme; putting stuff on C.val means we're always casting.? That might be OK, since there are probably more derefs of the val than of the ref. From brian.goetz at oracle.com Fri Feb 7 23:39:06 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 7 Feb 2020 18:39:06 -0500 Subject: Reference-default style In-Reply-To: <9C41E9F3-2A61-498D-8B20-700CC953EAF3@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> <9A0B3673-49C5-4D49-ACBE-E6C7B80CA290@oracle.com> <9C41E9F3-2A61-498D-8B20-700CC953EAF3@oracle.com> Message-ID: > (To remind everyone: We are using two half-buckets rather than one bucket > mainly so that Optional can be migrated. If it were just supporting V? then > we?d use an empty marker type, I think, probably just an interface.) The two half buckets also exist because it is how we get primitives and inlines to be the same thing, and not end up with THREE kinds of types. From john.r.rose at oracle.com Fri Feb 7 23:43:57 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 7 Feb 2020 15:43:57 -0800 Subject: Reference-default style In-Reply-To: References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> <9A0B3673-49C5-4D49-ACBE-E6C7B80CA290@oracle.com> <9C41E9F3-2A61-498D-8B20-700CC953EAF3@oracle.com> Message-ID: <226F747C-21D7-4F9F-A469-B965441FE447@oracle.com> On Feb 7, 2020, at 3:39 PM, Brian Goetz wrote: > > >> (To remind everyone: We are using two half-buckets rather than one bucket >> mainly so that Optional can be migrated. If it were just supporting V? then >> we?d use an empty marker type, I think, probably just an interface.) > > The two half buckets also exist because it is how we get primitives and inlines to be the same thing, and not end up with THREE kinds of types. Good point; thanks. In the case of primitives, it might turn out to be more than two half-buckets, depending on if and how we choose to support identity-bearing primitive wrappers (today?s new Integer(42)). Towards the baroque end of things, I can imagine a three-bucket solution: nest { abstract class Integer permits intThePrimitive, intTheBox { ?migration support here? } inline class intThePrimitive extends Integer { ? } final class intTheBox extends Integer { private final intThePrimitive value; ... } } From john.r.rose at oracle.com Fri Feb 7 23:47:04 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 7 Feb 2020 15:47:04 -0800 Subject: Reference-default style In-Reply-To: <897b2f37-6eb3-7c05-1c1b-e55dad68dd92@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> <897b2f37-6eb3-7c05-1c1b-e55dad68dd92@oracle.com> Message-ID: So, unless someone else comes up with something surprising, I think we know enough to prototype vigorously on your proposal, Brian. (Brian would say, ?why don?t we run with this for now?.) My queasiness about hollowing out C.val will either gain some practical shape from experience in prototyping, or will turn out to be something like coronavirus. ? John On Feb 7, 2020, at 3:37 PM, Brian Goetz wrote: > > >> I have a tiny reservation about the co-accessibility of both projections, >> although it?s a good principle overall. There might be cases (migration >> and maybe new code) where the nullable type has wider access than >> the inline type, where the type?s contract somehow embraces nullability >> to the extent that the .val projection is invisible. But we can cross that >> bridge when and if we come to it; I can?t think of compelling examples. > > I was worried about this too, wanting to support patterns like "public interface, private implementation." But then I realized I was thinking like a compiler rather than a programmer! If we want that, we can just write: > > public interface Foo { ... } > private inline class FooImpl implements Foo { ... } > > just like we always did. > >> >> (Nitpick: The JVM *fully* checks synchronization of such things dynamically; >> it cannot fully check at load time. Given that, it is not a good idea to partially >> check for evidence of synchronization; that just creates the semblance of an >> invariant where one does not exist. The JVM tries hard to make static checks >> that actually prove things, rather than just ?catch user errors?. So, please, >> no JVM load-time checks for synchronized methods, except *maybe* within >> the inline classes themselves.) > > Sure, that's your call. The static compiler can do its own thing. > >> >>> #### Translation -- classfiles >>> >>> A val-default inline class `C` is translated to two classfiles, `C` (val projection) and `C$ref` (ref projection). A ref-default inline class `D` is translated to two classfiles, `D` (ref projection) and `D$val` (val projection), as follows: >>> >>> - The two classfiles are members of the same nest. >>> - The ref projection is a sealed abstract class that permits only the val projection. >>> - Instance fields are lifted onto the val projection. >>> - Supertypes, methods (including static methods, and including the static "constructor" factory), and static fields are lifted onto the ref projection. Method bodies may internally require downcasting to `C.val` to access fields. >> This is a little like MVT, in that inline classes end up containing very little >> other than fields. This is the right move, IMO, for migrated classes. >> >> Hollowing out *all* inline classes strikes me as over-rotation for the sake >> of migration. I see how it allows both cases to have the same translation >> strategy, *except for the name*. That?s a pleasing property on paper. > > It's more than a pleasing property (though it is that.) Having translation strategies with big switches in them is where surprising migration compatibility constraints come from; would we want (say) changing a method from private to public to be a binary-incompatible change? These are the kinds of things that happen when the translation strategy gets too squirrely and ad-hoc. > >> In the case of reflection, I think we can afford to show a consistent view >> for both kinds of inlines, by making all fields and methods appear on >> both projections. > The language does this; while reflection is usually classfile-based, we'd like to treat these two classfiles as being mostly one artifact, so this seems reasonable to consider. > > >> This to say ?the class holding the method? instead of ?the C.ref?, we preserve >> the immediate goal of supporting migration of Optional etc., but we incur >> some migration debt, because it?s harder to move from val-default to ref-default. >> This, I think, is best fixed by adding auto-bridging of some sort later, rather >> than over-rotating towards the migration case right now. >> >> (Did I miss some other reason for putting everything on C.ref?) > > Putting stuff on C.ref means we can lean harder on inheritance/subtyping in the translation scheme; putting stuff on C.val means we're always casting. That might be OK, since there are probably more derefs of the val than of the ref. > > From daniel.smith at oracle.com Sun Feb 9 05:08:42 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Sat, 8 Feb 2020 22:08:42 -0700 Subject: Reference-default style In-Reply-To: References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> Message-ID: <521542A7-804F-4697-B30D-7A5AA4EA573F@oracle.com> > On Feb 7, 2020, at 4:17 PM, John Rose wrote: > > More info, from a JVM perspective: > > In that case, and that case alone, the JVM would validly look up the superclass > chain for a non-abstract method, and link to that instead. This is a very > special case of inheritance where a constructor is inherited and used as-is, rather > than wrapped by a subclass constructor. It?s a valid operation precisely because the > abstract constructor is provably a no-op. This is more complex than I was thinking, and maybe unnecessary? Instead, I was thinking that a class with an abstract constructor must have a superclass that also has an abstract constructor. And the invokespecial is then a no-op (well, a pop, actually). > The Object constructor is the initial > point of this inheritance process, and the end of the upward search. I?m leaning > towards keeping that as non-abstract, both for compatibility, and as a physical > landing place for the upward search past abstract constructors. The cleanest way to handle what I'm envisioning is for Object to have an abstract constructor, too. But if we need to do something special-case for Object, that's possible. > Actually > in HotSpot it does something: For a class with a finalizer it registers something > somewhere. But that?s precisely irrelevant to inlines. Oh, yeah, if we need to make sure that code gets executed (for identity classes), that will affect the design. From john.r.rose at oracle.com Sun Feb 9 06:08:45 2020 From: john.r.rose at oracle.com (John Rose) Date: Sat, 8 Feb 2020 22:08:45 -0800 Subject: Reference-default style In-Reply-To: <521542A7-804F-4697-B30D-7A5AA4EA573F@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> <521542A7-804F-4697-B30D-7A5AA4EA573F@oracle.com> Message-ID: <9C1D0EDD-5658-43F9-9821-BE68A6D6A782@oracle.com> On Feb 8, 2020, at 9:08 PM, Dan Smith wrote: > > Oh, yeah, if we need to make sure that code gets executed (for identity classes), that will affect the design. That?s the root of the stuff you found perhaps unnecessary. It could be done the way you propose also, but adding the ability of the invokespecial to turn into a ?pop?, and dealing with the loss of Object:: as a handy point of reference, makes for a different, less regular set of JVM changes. I could go either way on having Object:: changed to be abstract, but I think it?s safer to leave it exactly as is, and then just say ?inlines never get there?. ? John From brian.goetz at oracle.com Sun Feb 9 15:26:49 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 9 Feb 2020 10:26:49 -0500 Subject: Reference-default style In-Reply-To: <9C1D0EDD-5658-43F9-9821-BE68A6D6A782@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> <521542A7-804F-4697-B30D-7A5AA4EA573F@oracle.com> <9C1D0EDD-5658-43F9-9821-BE68A6D6A782@oracle.com> Message-ID: <30F3A6C3-F3AE-4576-A0F0-D64DBA442526@oracle.com> I think what dan is saying is that you are positing a degree of freedoms that is unnecessary. We want to have abs classes that can be a base for both inline and idents. An abstract ctor can be the indicator of this. But, why bother with allowing such a class to extend one that doesn?t meet the same requirements? They will be useless for in lines anyway. Require that the ctors be ?abstract all the way up.? Sent from my iPad > On Feb 9, 2020, at 1:08 AM, John Rose wrote: > >> On Feb 8, 2020, at 9:08 PM, Dan Smith wrote: >> >> Oh, yeah, if we need to make sure that code gets executed (for identity classes), that will affect the design. > > That?s the root of the stuff you found perhaps unnecessary. > It could be done the way you propose also, but adding the > ability of the invokespecial to turn into a ?pop?, and dealing > with the loss of Object:: as a handy point of reference, > makes for a different, less regular set of JVM changes. > > I could go either way on having Object:: changed to > be abstract, but I think it?s safer to leave it exactly as is, > and then just say ?inlines never get there?. > > ? John From john.r.rose at oracle.com Sun Feb 9 21:08:58 2020 From: john.r.rose at oracle.com (John Rose) Date: Sun, 9 Feb 2020 13:08:58 -0800 Subject: Reference-default style In-Reply-To: <30F3A6C3-F3AE-4576-A0F0-D64DBA442526@oracle.com> References: <30F3A6C3-F3AE-4576-A0F0-D64DBA442526@oracle.com> Message-ID: <121D9D77-2306-4D09-AB3F-EA8426C138B5@oracle.com> Good point. For our purposes the abstract ctor must always resolve to Object. And it must have the empty signature right? A small remaining point: There might be other use cases in the future for other configurations which make logical sense in the same way. If so we can expand the permissions to other constructors besides Object::()V. For now that?s the only one we care about delegating you. Have I got it now? On Feb 9, 2020, at 7:26 AM, Brian Goetz wrote: > > ?I think what dan is saying is that you are positing a degree of freedoms that is unnecessary. We want to have abs classes that can be a base for both inline and idents. An abstract ctor can be the indicator of this. But, why bother with allowing such a class to extend one that doesn?t meet the same requirements? They will be useless for in lines anyway. Require that the ctors be ?abstract all the way up.? > > Sent from my iPad > >> On Feb 9, 2020, at 1:08 AM, John Rose wrote: >> >>> On Feb 8, 2020, at 9:08 PM, Dan Smith wrote: >>> >>> Oh, yeah, if we need to make sure that code gets executed (for identity classes), that will affect the design. >> >> That?s the root of the stuff you found perhaps unnecessary. >> It could be done the way you propose also, but adding the >> ability of the invokespecial to turn into a ?pop?, and dealing >> with the loss of Object:: as a handy point of reference, >> makes for a different, less regular set of JVM changes. >> >> I could go either way on having Object:: changed to >> be abstract, but I think it?s safer to leave it exactly as is, >> and then just say ?inlines never get there?. >> >> ? John From daniel.smith at oracle.com Mon Feb 10 16:04:42 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Mon, 10 Feb 2020 09:04:42 -0700 Subject: Reference-default style In-Reply-To: <121D9D77-2306-4D09-AB3F-EA8426C138B5@oracle.com> References: <30F3A6C3-F3AE-4576-A0F0-D64DBA442526@oracle.com> <121D9D77-2306-4D09-AB3F-EA8426C138B5@oracle.com> Message-ID: > On Feb 9, 2020, at 2:08 PM, John Rose wrote: > > Good point. For our purposes the abstract ctor must always resolve to Object. And it must have the empty signature right? Right. The abstract constructor is an indication that "instances of this class require no initialization", so there's nothing to be done with parameters. (Equivalently, your search will always end up at Object, so it must match the Object descriptor.) > > A small remaining point: There might be other use cases in the future for other configurations which make logical sense in the same way. If so we can expand the permissions to other constructors besides Object::()V. For now that?s the only one we care about delegating you. > > Have I got it now? Yeah, I think so. What I'm most worried about is *yet another* ad hoc method search being performed by invokespecial. Special-case handling of Object is one thing; a full search mechanism is another. Something else we might want to do with this feature is a 'new' that executes no code but produces a fully-formed object. It can't be the current 'new' (verifier can't tell if a class requires instance initialization or not), and maybe this is subsumed by a 'new-and-init' instruction, but something to think about... From daniel.smith at oracle.com Mon Feb 10 18:08:13 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Mon, 10 Feb 2020 11:08:13 -0700 Subject: Reference-default style In-Reply-To: <521542A7-804F-4697-B30D-7A5AA4EA573F@oracle.com> References: <9041A550-BE7D-4D53-9DE5-FF307DD08BD1@oracle.com> <5bc689b8-08a0-9b59-a17e-01b8940e3940@oracle.com> <32ee13ab-0d03-7da9-bd66-32591de965d4@oracle.com> <41508fc8-6ec5-fae6-bcb8-4cd64eb3411e@oracle.com> <521542A7-804F-4697-B30D-7A5AA4EA573F@oracle.com> Message-ID: <22927FBA-1C38-4540-A5FB-4C638564D3C7@oracle.com> > On Feb 8, 2020, at 10:08 PM, Dan Smith wrote: > >> Actually >> in HotSpot it does something: For a class with a finalizer it registers something >> somewhere. But that?s precisely irrelevant to inlines. > > Oh, yeah, if we need to make sure that code gets executed (for identity classes), that will affect the design. In terms of actual code execution, it looks like there's nothing for the Object constructor to do: public java.lang.Object(); descriptor: ()V flags: (0x0001) ACC_PUBLIC Code: stack=0, locals=1, args_size=1 0: return LineNumberTable: line 50: 0 LocalVariableTable: Start Length Slot Name Signature 0 1 0 this Ljava/lang/Object; RuntimeVisibleAnnotations: 0: #65() jdk.internal.HotSpotIntrinsicCandidate If Hotspot wants to do some internal bookkeeping whenever a new instance gets created (either with the 'new' instruction or when, morally, Object. would execute), of course it's free to do so. So I'm tentatively proposing changing this bytecode to use ACC_ABSTRACT: public java.lang.Object(); descriptor: ()V flags: (0x0001) ACC_PUBLIC, ACC_ABSTRACT (How we actually achieve this in src/java.base/share/classes/java/lang/Object.java is TBD.) From daniel.smith at oracle.com Tue Feb 11 00:28:07 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Mon, 10 Feb 2020 17:28:07 -0700 Subject: JVM specs for and methods Message-ID: <3D0C4B52-0D45-4DF2-B1B1-D48DC72FF481@oracle.com> Here are spec change documents for two new JVM instance creation features that have been proposed to support inline classes. They're both fairly small, but involve a handful of different design choices. We can further discuss those design choices in this thread. 1) Abstract instance initialization methods http://cr.openjdk.java.net/~dlsmith/special-methods/special-methods-20200210/specs/abstract-init-jvms.html Classes with abstract methods require no code execution to initialize instances, and so are candidates to be superclasses of inline classes. 2) Unnamed factory methods http://cr.openjdk.java.net/~dlsmith/special-methods/special-methods-20200210/specs/factory-methods-jvms.html The special name '' can be used to declare static factory methods, including in an inline class. (In the previous iteration, we used '' with a non-'void' return. We since discussed avoiding this overloading and associated complexity by choosing a different name.) From daniel.smith at oracle.com Tue Feb 11 23:54:10 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Tue, 11 Feb 2020 16:54:10 -0700 Subject: Superclasses for inline classes In-Reply-To: References: <145A4BBE-4972-4CEB-A149-95C4D6E5C519@oracle.com> Message-ID: <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> So I think the JVM story is fairly settled; I've spun it off into another thread, with a proposed spec. The language story is still uncertain. Here are four alternative designs, all of which are, I think, reasonable approaches to consider. Some discussion about trade-offs is at the end. ----- Alternative 1: No language support The language requires all inline classes to extend Object (or perhaps prohibits the 'extends' clause). Abstract methods are available only as a compiler tool. Some special exceptions are necessary: - Somehow, class files for Object and Number must be generated with abstract methods - Integer, Double, etc., are inline classes but are allowed to extend Number ----- Alternative 2: Abstract constructors in the language abstract class C { public abstract C(); } Like a method, a constructor can be declared 'abstract' and omit a body. (Bikeshed: maybe the 'abstract' keyword is spelled differently, or left off entirely.) These declarations can align closely with the JVM's rules for methods: - The superclass must also have an abstract constructor - No instance initializers or instance field initializers are allowed - It's okay to overload the constructor, but 'abstract' only works on a no-arg constructor - (Perhaps) the class doesn't have to be abstract - It's allowed (as a no-op) to invoke the constructor (new Object() or super()) We'll need one further restriction that isn't checked by the JVM and isn't as principled: no instance fields are allowed, even if they're default-initialized. Otherwise, inline classes will have to search for private fields to decide if extension is legal or not, breaking encapsulation. The abstract-ness of the constructor is part of the API?appears in javadoc, changing it is incompatible. Having an empty or default constructor isn't the same as having an abstract constructor. Inline classes can only extend classes with abstract constructors (and maybe no 'synchronized' instance methods). ----- Alternative 3: Inline-friendly property in the language inlineable abstract class C { } The 'inlineable' property (bikeshed: how to spell this?) enforces some constraints and authorizes children to be inline classes. Specific constraints: - Can't have instance fields - Can't declare a constructor or instance initializer - Must extend an inlineable class - Must be an abstract class or Object (maybe?) - Must not have synchronized methods (probably?) An annotation spelling is an option, too, although that crosses a line?there's a precedent for annotations that prompt compiler errors, but not for annotations that influence bytecode output. An inlineable class's bytecode has an abstract method. The 'inlineable' property is part of the API?appears in javadoc, changing it is incompatible. Inline classes can only extend inlineable classes. ----- Alternative 4: Infer no constructor Typically, for each class that lacks a constructor declaration, a default constructor is provided that does some simple initialization. But in the following circumstances, we claim there is no constructor at all: - Class doesn't declare a constructor or instance initializer - Class doesn't declare fields - Superclass has no constructor - Class is abstract or Object (maybe?) Again, blank private fields may not *need* initialization, but inline subclasses need to know that they exist, and the best way to communicate that is through the constructor. 'new Object()' (and perhaps 'new Foo()' if we drop the abstract class requirement) doesn't reference any constructor at all. In that case, a fresh instance is allocated without any code execution. For compatibility, this also applies to 'super()' (although we can discourage its use in these cases going forward). A class without a constructor has, in bytecode, an abstract method. The lack of a constructor is part of the API?appears in javadoc, changing it is incompatible. An annotation like @FunctionalInterface could help by checking that nothing has changed on recompilation. Inline classes can only extend classes that lack constructors (and maybe 'synchronized' instance methods). ----- Discussion Noise level: We don't want to make a big deal about this feature. People shouldn't think too much about it. (4) wins in this regard?no new syntax, just some hand-waving in the language model about what it actually means when you leave out your constructor. (2) introduces a subtle variation on constructor declarations, which can generally be overlooked. (3) is a neon invitation to treat these classes like a fundamentally new kind of entity. Compatibility: Adding or removing 'abstract' from an '' method is a binary incompatible change, so it's good if that doesn't happen accidentally. But it's hard to increase awareness of that commitment while minimizing noise level, so there are trade-offs. (2) and (3) both force authors to change something. (4) doesn't have that guardrail, and it's quite easy to imagine an innocent refactoring that makes a class inline-hostile. Brian suggested an annotation as an optional guard against this, but I doubt most inline class superclasses will even be aware that they should consider the annotation. And if we succeed in making everyone worry about it, well... adding that overhead as a programming best practice seems counter to (4)'s strengths. What's especially troubling about (4) is we're taking longstanding intuitions about Java programming?an absent constructor is the same as an empty constructor?and saying, just kidding, those are two different things. And not just different implementations, but different APIs with different compatibility promises. In fact, by leaving off the constructor, you've promised to *never* introduce a private instance field to this class. Availability: Ideally, extending a suitable abstract class should be frictionless for inline class authors. In (2) and (3), the authors are blocked until the abstract class author opts in. In (4), the opt in is by default, although there's still a requirement that the abstract class be compiled with an appropriate source version. In practice, if we require an opt in, this will be an obscure, little-used feature; or maybe it will become widespread, but we'll have introduced some new standard boilerplate into the language. If we don't require an opt in, inline classes will be free to extend most abstract classes automatically. For context, Brian passed to me some Google corpus numbers on abstract classes. Abstract classes are fairly rare in the universe of type declarations (4%). Among abstract classes, a large majority seem to be candidate supertypes for inline classes?85% have no fields. And most are public (75%), meaning they're less likely to be aware of all their subclasses. Summary: I'm torn. (4) is really the feature I want, especially on the availability front, but I don't know if we can get away with pulling the rug out from under authors. And stepping back, whether we opt in (as in (4)) or opt out (as in (2), (3)) by default, it's unfortunate that the decision we're asking authors to make is not something they're equipped for. Specifically: "Do I think it's more likely that someday an inline class will want to extend this class, or that someone will want to add a private field to this class?" How can they answer that? Best they can do is play the odds, which I don't imagine change much from class to class, and I'm guessing would say it's much more likely to be extended by an inline class. From brian.goetz at oracle.com Wed Feb 12 17:23:13 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 12 Feb 2020 12:23:13 -0500 Subject: Superclasses for inline classes In-Reply-To: <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> References: <145A4BBE-4972-4CEB-A149-95C4D6E5C519@oracle.com> <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> Message-ID: <51B78554-BB07-4164-8EA4-932910F36AE3@oracle.com> > The language story is still uncertain. Here are four alternative designs, all of which are, I think, reasonable approaches to consider. Some discussion about trade-offs is at the end. Thanks for pulling this together into a nice menu, with chef?s recommendations. Let me add a few notes, which are mostly ?soft? concerns. Overall, having fewer ?feature X can?t work with Y? restrictions is better; alternative 1 (no language support) creates a ?inline classes can?t extend abstract classes? interaction. WE know that there are at least a few abstract classes we want to have play nicely with inline classes: Number and Record immediately come to mind, but AbstractCollection is also a strong candidate. (AbstractList has fields, and therefore is a useful example because these fields are private and serve the implementation, which illustrates the dangers Dan is concerned about: that AbstractList starts out being like AbstractCollection, but then finds itself constrained in its evolution because it has inline subclasses.) I agree that giving this feature too-prominent billing (such as `inline-friendly abstract class`) is a bad trade. I also have no problem with the language inferring whether the superclass is suitable for extension by inline classes; this isn?t much different than requiring an accessible constructor. Some classes are suitable for extension by some other classes under some circumstances; as long as the rules are clear enough, and the compiler error messages are helpful when you cross the line, I don?t need a special opt in to say ?It?s OK for this class to be extended by inline classes.? The problematic cases are all when abstract class C is extended by inline class V, _and C and V are in different maintenance domains_. If the two are in the same maintenance domain (package or module), then refactoring one may have consequences on the other, but the two can be co-refactored to avoid the problem. (This is just like refactoring an enum to be a class; there are pitfalls, but if the two are in the same maintenance domain, they can be more easily navigated.) So, the problematic case is: - A is a public abstract class in maintenance domain M, which is currently suitable for extension by inline classes - V is an inline class that extends A, in a separate maintenance domain N - Later, A wishes to refactor to have, say, private fields, but this would break V. This is surely a possible nasty surprise; the question is how often this is going to happen, and how much it would cost to prevent it from happening. Based on the numbers we got from Google?s code base (thanks!), it looks like ~3% of classes are public abstract classes, and many of those would meet the requirements to be extended by inline classes. But, less-than-3% is still a small number, and cross-maintenance-domain extension is rarer. (This is a mostly closed codebase; if this were a library, I suspect suitable exposed abstract classes would be even rarer, as all the style guidance suggests preferring exposing interfaces to abstract classes where possible.) So I think the chain of events that are required to cause regret (publicly exposed abstract class, cross maintenance-domain extension, and subsequent incompatible evolution) likely constitute a corner-of-a-corner case. This argues for something that minimizes the intrusion into the language. Why do we encourage people to expose interfaces and not abstract classes in APIs? Because interfaces are ?more abstract?, and therefore exposing them is more constraining. So another lever at our disposal is: dial up the style-guide suggestions about not exposing abstract classes across maintenance domains. Finally, there?s some mitigation if the bad case comes to pass. Suppose we have: public abstract class Foo { /* accidentally inline friendly */ } and later you want to add fields to Foo and do: public abstract class Foo { int field; Foo(int f) { ? } } private class Bar extends Foo { ? } but can?t do so because some external inline class has extended it. You still have an option: public abstract class Foo { /* stays inline friendly */ } private abstract class SonOfFoo extends Foo { // all the non-inline-friendliness you want } private class Bar extends SonOfFoo { ? } So, even though the author made a ?mistake? by exposing the abstract class and therefore constraining themselves forever, the workaround isn?t so bad. Of the options, I?m still liking (4), backed up by an annotation like @FunctionalInterface that turns on extra type checking and documentation but serves only to capture design intent. > > ----- > > Alternative 1: No language support > > The language requires all inline classes to extend Object (or perhaps prohibits the 'extends' clause). Abstract methods are available only as a compiler tool. > > Some special exceptions are necessary: > > - Somehow, class files for Object and Number must be generated with abstract methods > > - Integer, Double, etc., are inline classes but are allowed to extend Number > > ----- > > Alternative 2: Abstract constructors in the language > > abstract class C { > public abstract C(); > } > > Like a method, a constructor can be declared 'abstract' and omit a body. (Bikeshed: maybe the 'abstract' keyword is spelled differently, or left off entirely.) > > These declarations can align closely with the JVM's rules for methods: > - The superclass must also have an abstract constructor > - No instance initializers or instance field initializers are allowed > - It's okay to overload the constructor, but 'abstract' only works on a no-arg constructor > - (Perhaps) the class doesn't have to be abstract > - It's allowed (as a no-op) to invoke the constructor (new Object() or super()) > > We'll need one further restriction that isn't checked by the JVM and isn't as principled: no instance fields are allowed, even if they're default-initialized. Otherwise, inline classes will have to search for private fields to decide if extension is legal or not, breaking encapsulation. > > The abstract-ness of the constructor is part of the API?appears in javadoc, changing it is incompatible. Having an empty or default constructor isn't the same as having an abstract constructor. > > Inline classes can only extend classes with abstract constructors (and maybe no 'synchronized' instance methods). > > ----- > > Alternative 3: Inline-friendly property in the language > > inlineable abstract class C { > } > > The 'inlineable' property (bikeshed: how to spell this?) enforces some constraints and authorizes children to be inline classes. > > Specific constraints: > - Can't have instance fields > - Can't declare a constructor or instance initializer > - Must extend an inlineable class > - Must be an abstract class or Object (maybe?) > - Must not have synchronized methods (probably?) > > An annotation spelling is an option, too, although that crosses a line?there's a precedent for annotations that prompt compiler errors, but not for annotations that influence bytecode output. > > An inlineable class's bytecode has an abstract method. > > The 'inlineable' property is part of the API?appears in javadoc, changing it is incompatible. > > Inline classes can only extend inlineable classes. > > ----- > > Alternative 4: Infer no constructor > > Typically, for each class that lacks a constructor declaration, a default constructor is provided that does some simple initialization. But in the following circumstances, we claim there is no constructor at all: > > - Class doesn't declare a constructor or instance initializer > - Class doesn't declare fields > - Superclass has no constructor > - Class is abstract or Object (maybe?) > > Again, blank private fields may not *need* initialization, but inline subclasses need to know that they exist, and the best way to communicate that is through the constructor. > > 'new Object()' (and perhaps 'new Foo()' if we drop the abstract class requirement) doesn't reference any constructor at all. In that case, a fresh instance is allocated without any code execution. > > For compatibility, this also applies to 'super()' (although we can discourage its use in these cases going forward). > > A class without a constructor has, in bytecode, an abstract method. > > The lack of a constructor is part of the API?appears in javadoc, changing it is incompatible. An annotation like @FunctionalInterface could help by checking that nothing has changed on recompilation. > > Inline classes can only extend classes that lack constructors (and maybe 'synchronized' instance methods). > > ----- > > Discussion > > Noise level: We don't want to make a big deal about this feature. People shouldn't think too much about it. (4) wins in this regard?no new syntax, just some hand-waving in the language model about what it actually means when you leave out your constructor. (2) introduces a subtle variation on constructor declarations, which can generally be overlooked. (3) is a neon invitation to treat these classes like a fundamentally new kind of entity. > > Compatibility: Adding or removing 'abstract' from an '' method is a binary incompatible change, so it's good if that doesn't happen accidentally. But it's hard to increase awareness of that commitment while minimizing noise level, so there are trade-offs. (2) and (3) both force authors to change something. (4) doesn't have that guardrail, and it's quite easy to imagine an innocent refactoring that makes a class inline-hostile. > > Brian suggested an annotation as an optional guard against this, but I doubt most inline class superclasses will even be aware that they should consider the annotation. And if we succeed in making everyone worry about it, well... adding that overhead as a programming best practice seems counter to (4)'s strengths. > > What's especially troubling about (4) is we're taking longstanding intuitions about Java programming?an absent constructor is the same as an empty constructor?and saying, just kidding, those are two different things. And not just different implementations, but different APIs with different compatibility promises. In fact, by leaving off the constructor, you've promised to *never* introduce a private instance field to this class. > > Availability: Ideally, extending a suitable abstract class should be frictionless for inline class authors. In (2) and (3), the authors are blocked until the abstract class author opts in. In (4), the opt in is by default, although there's still a requirement that the abstract class be compiled with an appropriate source version. > > In practice, if we require an opt in, this will be an obscure, little-used feature; or maybe it will become widespread, but we'll have introduced some new standard boilerplate into the language. If we don't require an opt in, inline classes will be free to extend most abstract classes automatically. > > For context, Brian passed to me some Google corpus numbers on abstract classes. Abstract classes are fairly rare in the universe of type declarations (4%). Among abstract classes, a large majority seem to be candidate supertypes for inline classes?85% have no fields. And most are public (75%), meaning they're less likely to be aware of all their subclasses. > > Summary: I'm torn. (4) is really the feature I want, especially on the availability front, but I don't know if we can get away with pulling the rug out from under authors. > > And stepping back, whether we opt in (as in (4)) or opt out (as in (2), (3)) by default, it's unfortunate that the decision we're asking authors to make is not something they're equipped for. Specifically: "Do I think it's more likely that someday an inline class will want to extend this class, or that someone will want to add a private field to this class?" How can they answer that? Best they can do is play the odds, which I don't imagine change much from class to class, and I'm guessing would say it's much more likely to be extended by an inline class. > From forax at univ-mlv.fr Wed Feb 12 18:41:32 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 12 Feb 2020 19:41:32 +0100 (CET) Subject: Superclasses for inline classes In-Reply-To: <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> References: <145A4BBE-4972-4CEB-A149-95C4D6E5C519@oracle.com> <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> Message-ID: <1520120957.2115807.1581532892571.JavaMail.zimbra@u-pem.fr> Hi all, sorry, was not available for the meeting today (i'm officially on vacation). I prefer (2) with a restriction like (1). I don't think users should be able to declare this kind of abstract class because you can not evolve them easily. I'm worried that if they become a tool available, people will over-use them (they are like a kind of abstract class with super-power, as a user it's like a shiny cookie), so in the end we will have to introduce inheritance of inline classes. This rule out (3). What (4) try to do is too clever IMO, a garbage class like java.util.Collections (with an 's' at the end) validate all the conditions but should not have an abstract constructor. It's too easy to create such kind of abstract class without knowing it. So for me, it's (2) + (1), at language level, a constructor can be declared abstract but it's restricted to class inside java.lang or java.base (enforced by the compiler and the VM). R?mi ----- Mail original ----- > De: "daniel smith" > ?: "valhalla-spec-experts" > Envoy?: Mercredi 12 F?vrier 2020 00:54:10 > Objet: Re: Superclasses for inline classes > So I think the JVM story is fairly settled; I've spun it off into another > thread, with a proposed spec. > > The language story is still uncertain. Here are four alternative designs, all of > which are, I think, reasonable approaches to consider. Some discussion about > trade-offs is at the end. > > ----- > > Alternative 1: No language support > > The language requires all inline classes to extend Object (or perhaps prohibits > the 'extends' clause). Abstract methods are available only as a compiler > tool. > > Some special exceptions are necessary: > > - Somehow, class files for Object and Number must be generated with abstract > methods > > - Integer, Double, etc., are inline classes but are allowed to extend Number > > ----- > > Alternative 2: Abstract constructors in the language > > abstract class C { > public abstract C(); > } > > Like a method, a constructor can be declared 'abstract' and omit a body. > (Bikeshed: maybe the 'abstract' keyword is spelled differently, or left off > entirely.) > > These declarations can align closely with the JVM's rules for methods: > - The superclass must also have an abstract constructor > - No instance initializers or instance field initializers are allowed > - It's okay to overload the constructor, but 'abstract' only works on a no-arg > constructor > - (Perhaps) the class doesn't have to be abstract > - It's allowed (as a no-op) to invoke the constructor (new Object() or super()) > > We'll need one further restriction that isn't checked by the JVM and isn't as > principled: no instance fields are allowed, even if they're > default-initialized. Otherwise, inline classes will have to search for private > fields to decide if extension is legal or not, breaking encapsulation. > > The abstract-ness of the constructor is part of the API?appears in javadoc, > changing it is incompatible. Having an empty or default constructor isn't the > same as having an abstract constructor. > > Inline classes can only extend classes with abstract constructors (and maybe no > 'synchronized' instance methods). > > ----- > > Alternative 3: Inline-friendly property in the language > > inlineable abstract class C { > } > > The 'inlineable' property (bikeshed: how to spell this?) enforces some > constraints and authorizes children to be inline classes. > > Specific constraints: > - Can't have instance fields > - Can't declare a constructor or instance initializer > - Must extend an inlineable class > - Must be an abstract class or Object (maybe?) > - Must not have synchronized methods (probably?) > > An annotation spelling is an option, too, although that crosses a line?there's a > precedent for annotations that prompt compiler errors, but not for annotations > that influence bytecode output. > > An inlineable class's bytecode has an abstract method. > > The 'inlineable' property is part of the API?appears in javadoc, changing it is > incompatible. > > Inline classes can only extend inlineable classes. > > ----- > > Alternative 4: Infer no constructor > > Typically, for each class that lacks a constructor declaration, a default > constructor is provided that does some simple initialization. But in the > following circumstances, we claim there is no constructor at all: > > - Class doesn't declare a constructor or instance initializer > - Class doesn't declare fields > - Superclass has no constructor > - Class is abstract or Object (maybe?) > > Again, blank private fields may not *need* initialization, but inline subclasses > need to know that they exist, and the best way to communicate that is through > the constructor. > > 'new Object()' (and perhaps 'new Foo()' if we drop the abstract class > requirement) doesn't reference any constructor at all. In that case, a fresh > instance is allocated without any code execution. > > For compatibility, this also applies to 'super()' (although we can discourage > its use in these cases going forward). > > A class without a constructor has, in bytecode, an abstract method. > > The lack of a constructor is part of the API?appears in javadoc, changing it is > incompatible. An annotation like @FunctionalInterface could help by checking > that nothing has changed on recompilation. > > Inline classes can only extend classes that lack constructors (and maybe > 'synchronized' instance methods). > > ----- > > Discussion > > Noise level: We don't want to make a big deal about this feature. People > shouldn't think too much about it. (4) wins in this regard?no new syntax, just > some hand-waving in the language model about what it actually means when you > leave out your constructor. (2) introduces a subtle variation on constructor > declarations, which can generally be overlooked. (3) is a neon invitation to > treat these classes like a fundamentally new kind of entity. > > Compatibility: Adding or removing 'abstract' from an '' method is a binary > incompatible change, so it's good if that doesn't happen accidentally. But it's > hard to increase awareness of that commitment while minimizing noise level, so > there are trade-offs. (2) and (3) both force authors to change something. (4) > doesn't have that guardrail, and it's quite easy to imagine an innocent > refactoring that makes a class inline-hostile. > > Brian suggested an annotation as an optional guard against this, but I doubt > most inline class superclasses will even be aware that they should consider the > annotation. And if we succeed in making everyone worry about it, well... adding > that overhead as a programming best practice seems counter to (4)'s strengths. > > What's especially troubling about (4) is we're taking longstanding intuitions > about Java programming?an absent constructor is the same as an empty > constructor?and saying, just kidding, those are two different things. And not > just different implementations, but different APIs with different compatibility > promises. In fact, by leaving off the constructor, you've promised to *never* > introduce a private instance field to this class. > > Availability: Ideally, extending a suitable abstract class should be > frictionless for inline class authors. In (2) and (3), the authors are blocked > until the abstract class author opts in. In (4), the opt in is by default, > although there's still a requirement that the abstract class be compiled with > an appropriate source version. > > In practice, if we require an opt in, this will be an obscure, little-used > feature; or maybe it will become widespread, but we'll have introduced some new > standard boilerplate into the language. If we don't require an opt in, inline > classes will be free to extend most abstract classes automatically. > > For context, Brian passed to me some Google corpus numbers on abstract classes. > Abstract classes are fairly rare in the universe of type declarations (4%). > Among abstract classes, a large majority seem to be candidate supertypes for > inline classes?85% have no fields. And most are public (75%), meaning they're > less likely to be aware of all their subclasses. > > Summary: I'm torn. (4) is really the feature I want, especially on the > availability front, but I don't know if we can get away with pulling the rug > out from under authors. > > And stepping back, whether we opt in (as in (4)) or opt out (as in (2), (3)) by > default, it's unfortunate that the decision we're asking authors to make is not > something they're equipped for. Specifically: "Do I think it's more likely that > someday an inline class will want to extend this class, or that someone will > want to add a private field to this class?" How can they answer that? Best they > can do is play the odds, which I don't imagine change much from class to class, > and I'm guessing would say it's much more likely to be extended by an inline > class. From daniel.smith at oracle.com Wed Feb 12 23:46:36 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 12 Feb 2020 16:46:36 -0700 Subject: Superclasses for inline classes In-Reply-To: <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> References: <145A4BBE-4972-4CEB-A149-95C4D6E5C519@oracle.com> <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> Message-ID: <07C52215-0702-4277-BF97-01C24F4B496E@oracle.com> > On Feb 11, 2020, at 4:54 PM, Dan Smith wrote: > > Availability: Ideally, extending a suitable abstract class should be frictionless for inline class authors. In (2) and (3), the authors are blocked until the abstract class author opts in. In (4), the opt in is by default, although there's still a requirement that the abstract class be compiled with an appropriate source version. > > In practice, if we require an opt in, this will be an obscure, little-used feature; or maybe it will become widespread, but we'll have introduced some new standard boilerplate into the language. If we don't require an opt in, inline classes will be free to extend most abstract classes automatically. Dan H raised a good point about availability on the call today: some tools like to instrument constructors, including empty/default ones. If we recompile most of the abstract classes in the world to make them instrumentation-hostile (as is the case if they have ACC_ABSTRACT methods), some of those tools are going to break, perhaps with no good workaround for the behavior they want. So there's a trade-off: widespread inline-friendly abstract classes will make inline class authors happy, but instrumentation tools sad. I don't have a sense of how serious this problem is, but it's something we should get a better handle on. From daniel.smith at oracle.com Wed Feb 12 23:49:34 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 12 Feb 2020 16:49:34 -0700 Subject: Superclasses for inline classes In-Reply-To: <1520120957.2115807.1581532892571.JavaMail.zimbra@u-pem.fr> References: <145A4BBE-4972-4CEB-A149-95C4D6E5C519@oracle.com> <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> <1520120957.2115807.1581532892571.JavaMail.zimbra@u-pem.fr> Message-ID: <1DC20F18-ABB0-4633-88E4-F8DEA9860D5D@oracle.com> > On Feb 12, 2020, at 11:41 AM, Remi Forax wrote: > > a garbage class like java.util.Collections (with an 's' at the end) validate all the conditions but should not have an abstract constructor. Why not? If identity classes can extend it, and it has no state/initialization, why not inline classes too? (In reality, this particular class has chosen to actively prevent subclassing and instantiation by declaring a private constructor; it wouldn't qualify): public class Collections { // Suppresses default constructor, ensuring non-instantiability. private Collections() { } From forax at univ-mlv.fr Thu Feb 13 10:57:59 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Thu, 13 Feb 2020 11:57:59 +0100 (CET) Subject: Superclasses for inline classes In-Reply-To: <1DC20F18-ABB0-4633-88E4-F8DEA9860D5D@oracle.com> References: <145A4BBE-4972-4CEB-A149-95C4D6E5C519@oracle.com> <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> <1520120957.2115807.1581532892571.JavaMail.zimbra@u-pem.fr> <1DC20F18-ABB0-4633-88E4-F8DEA9860D5D@oracle.com> Message-ID: <1881739495.251791.1581591479693.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "daniel smith" > ?: "Remi Forax" > Cc: "valhalla-spec-experts" > Envoy?: Jeudi 13 F?vrier 2020 00:49:34 > Objet: Re: Superclasses for inline classes >> On Feb 12, 2020, at 11:41 AM, Remi Forax wrote: >> >> a garbage class like java.util.Collections (with an 's' at the end) validate all >> the conditions but should not have an abstract constructor. > > Why not? If identity classes can extend it, and it has no state/initialization, > why not inline classes too? Sorry to not be clear, because it's not a backward compatible change, the empty constructor becomes abstract. There is a lot of classes like that in the wild, and given that the JLS allows to call static methods on an instance, there are existing code that are using the default constructor even if the author of the class never wanted to allow such usage. R?mi From daniel.smith at oracle.com Thu Feb 13 14:04:41 2020 From: daniel.smith at oracle.com (Dan Smith) Date: Thu, 13 Feb 2020 07:04:41 -0700 Subject: Superclasses for inline classes In-Reply-To: <1881739495.251791.1581591479693.JavaMail.zimbra@u-pem.fr> References: <145A4BBE-4972-4CEB-A149-95C4D6E5C519@oracle.com> <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> <1520120957.2115807.1581532892571.JavaMail.zimbra@u-pem.fr> <1DC20F18-ABB0-4633-88E4-F8DEA9860D5D@oracle.com> <1881739495.251791.1581591479693.JavaMail.zimbra@u-pem.fr> Message-ID: > On Feb 13, 2020, at 3:57 AM, forax at univ-mlv.fr wrote: > > ----- Mail original ----- >> De: "daniel smith" >> ?: "Remi Forax" >> Cc: "valhalla-spec-experts" >> Envoy?: Jeudi 13 F?vrier 2020 00:49:34 >> Objet: Re: Superclasses for inline classes > >>> On Feb 12, 2020, at 11:41 AM, Remi Forax wrote: >>> >>> a garbage class like java.util.Collections (with an 's' at the end) validate all >>> the conditions but should not have an abstract constructor. >> >> Why not? If identity classes can extend it, and it has no state/initialization, >> why not inline classes too? > > Sorry to not be clear, because it's not a backward compatible change, > the empty constructor becomes abstract. The JVM and language features (all 3 variations) are designed to ensure that the default constructor -> abstract constructor change is fully compatible. Can you illustrate what you think would not be compatible? > There is a lot of classes like that in the wild, and given that the JLS allows to call static methods on an instance, there are existing code that are using the default constructor even if the author of the class never wanted to allow such usage. If there is a class that allows unwanted subclasses, it will continue to allow unwanted subclasses. The way to change that is to make it 'final' or declare a 'private' constructor. From forax at univ-mlv.fr Fri Feb 14 19:23:04 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 14 Feb 2020 20:23:04 +0100 (CET) Subject: Superclasses for inline classes In-Reply-To: References: <145A4BBE-4972-4CEB-A149-95C4D6E5C519@oracle.com> <599E0AEA-BC24-42DE-A727-06B00C7057C2@oracle.com> <1520120957.2115807.1581532892571.JavaMail.zimbra@u-pem.fr> <1DC20F18-ABB0-4633-88E4-F8DEA9860D5D@oracle.com> <1881739495.251791.1581591479693.JavaMail.zimbra@u-pem.fr> Message-ID: <827089138.1130434.1581708184161.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "daniel smith" > ?: "Remi Forax" > Cc: "valhalla-spec-experts" > Envoy?: Jeudi 13 F?vrier 2020 15:04:41 > Objet: Re: Superclasses for inline classes >> On Feb 13, 2020, at 3:57 AM, forax at univ-mlv.fr wrote: >> >> ----- Mail original ----- >>> De: "daniel smith" >>> ?: "Remi Forax" >>> Cc: "valhalla-spec-experts" >>> Envoy?: Jeudi 13 F?vrier 2020 00:49:34 >>> Objet: Re: Superclasses for inline classes >> >>>> On Feb 12, 2020, at 11:41 AM, Remi Forax wrote: >>>> >>>> a garbage class like java.util.Collections (with an 's' at the end) validate all >>>> the conditions but should not have an abstract constructor. >>> >>> Why not? If identity classes can extend it, and it has no state/initialization, >>> why not inline classes too? >> >> Sorry to not be clear, because it's not a backward compatible change, >> the empty constructor becomes abstract. > > The JVM and language features (all 3 variations) are designed to ensure that the > default constructor -> abstract constructor change is fully compatible. Can you > illustrate what you think would not be compatible? What am saying is that garbage classes either are declared abstract or have a private constructor to avoid to be instantiated, the former will be eligible for (4) and not the later, so some garbage classes will become treated differently by the VM, will have their javadoc altered, the API seen by reflection changed, etc, even if it was not the intent of their authors. You're right that an abstract garbage class can be sub-classed, yes, using a private constructor that throws a NoSuchMethodError is a better design, but that not the point, the point is why retrofitting an existing class while it's not clear that it was the author intent. For me (4) is too clever with little benefit. We want to introduce a new kind of abstract class with a slightly different semantics that the one existing. Playing the retrofit game here doesn't worth the trouble. R?mi From brian.goetz at oracle.com Tue Feb 18 19:01:50 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 18 Feb 2020 14:01:50 -0500 Subject: Valhalla -- finding the primitives Message-ID: <2e0e145e-940a-df54-511a-581bf2648b3e@oracle.com> I think its worth reflecting on how far we've come in Valhalla, both for the specific designs in the VM and language, and the clarity of the basic concepts. In the early model (Q World), the idea was that we would declare a class as either a value class or a "regular" class, and we would derive various properties based on that: ??? regular classes have identity, value classes do not ??? regular classes are nullable, value classes are not ??? regular classes are reference types, value classes are not This model was derived from the current relationship between `int` and `Integer`.? But, to interoperate with dynamically typed code (such as reflection) and erased generics, we needed reference types, so each value class got a "box" type (denotable as LV at the VM level) which was a reference type.? Any interfaces declared on the value class were superinterfaces of the box.? There was one runtime class, and `getClass()` returned that, regardless of whether invoked on a value or a boxed reference. This worked, but there were many aspects which were either confusing or unsatisfying.? Value classes were neither reference types nor primitives, so we had gone from a type system split cleanly in two (which many people dislike) to one split uncleanly in three (for example, values had the Object methods, but didn't derive from Object, complicating code that was supposed to be generic across values and references alike.) Some chafed at the notion that value types could never be nullable; others didn't like that the all-zero value was always a member of the value set, whether or not it had semantic meaning.? Some reference types had significant identity, and others (boxes) didn't.? And we didn't have a clean story for migration. In the second iteration (L World), we addressed (at the VM level) the need to box in order to access reference-related functionality, by making `QV` a subtype of `LObject`, rationalizing the subtyping relationships between arrays, and replacing the box with a null-adjunction type (LV).? This reduced the pressure on migration substantially, but we still hadn't addressed most of the user model issues, including the tripartite nature of the type system, and we created quite a few problems as a result (such as the relationship between the two class mirrors for QV/LV.) For example, to address initialization safety (where the zero value is outside the domain), we explored the notion of zero-default vs null-default inline classes, which involved treating the all-zero value as a null for some value classes but as a zero for others.? But we kept finding that we were having too many "flavors" of everything, because, in hindsight, the various aspects were not yet cleanly factored down to their primitives.? In the end, it turned out we were conflating a number of distinctions, and kept trying to use one as a proxy for another: ?- nullable vs non-nullable ?- pass-by-reference vs pass-by-value / flattened ?- reference type vs value type ?- identity-ful vs identity-free For example, we wanted to call classes like String "reference types" and classes like Point "value types", but when we got to types like Object and interfaces, they had one foot in each camp.? It turns out, that in the "find the primitive" game, "reference type" wasn't the primitive. Classes.? The user declares _classes_ ("public class Foo { }"); we derive _types_ from class declarations (Foo, Foo[], etc.) The primitive that Valhalla introduces into class declaration is whether the instances of the class _have identity or not_. Traditional classes are now revealed to be "identity classes"; the new kind (identity-free) are called "inline classes".? (This might not be the final word on the subject.) Types and values.? In the type system we have now, some types contain primitive values, and other types contain _references to objects_.? What messed us up for a while is that the type types -- Object and interfaces -- can contain both.? A big AHA of the recent iterations is that it makes sense to talk about both _values of_ inline classes and _references to_ those values. Reference type has (almost) nothing to do with inline vs identity -- it has to do with whether the value set of the type contains values, or references. For an identity class C, we derive one type: C, which consists of references to instances of C.? For an inline class V, we derive two types: `V.ref`, which is a reference type (and therefore nullable), and contains references to the instances of V, and `V.val`, which is not a reference type, and whose values are "raw" instances of V. With this understanding, the nullity problem becomes a simpler one: nullity is a property of _reference types_.? So `V.ref` is nullable, and `V.val` is not; we don't need a way to say "nullable value" or different ways to interpret the default value.? We derive flattening and calling conventions in the same way; for reference types, we always store / pass as-if-by reference, but for "val" types, we store / pass as-if-by value. It is this refined understanding that has brought me back to the ref/val notation _for the types_.? "Inline" is a way of saying "identity free" when declaring classes, but it doesn't say anything (yet) about the semantics of how we represent variables on the heap or pass them on the stack.? For this, we need an additional property of the type, and ref vs val seems to ideally describe what we mean -- that the value set of the type consists of either references or values, and the representation/calling conventions behave as if we are storing/passing references or values.? (Having come to this clarity about the types, we are free to pick a word other than "inline" if we think there is a better way to say "identity-free", though I don't think going back to "value" is necessarily right.) With this distinction in place, some previously nasty problems (such as nullity) become trivial (if you want "nullable values", use references), and some previously impossible problems (such as unifying primitives with values) become tractible. From brian.goetz at oracle.com Wed Feb 19 16:45:38 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 19 Feb 2020 11:45:38 -0500 Subject: Locking on primitive boxes Message-ID: <57a9b8fe-d5ba-3566-3953-110041f62706@oracle.com> One of the main impediments to migrating the primitive wrappers to be the reference projection of an inline class is locking -- there may be code out there that locks on instances of primitive wrappers.? Let's look at how we might wean users off, and what the compatibility constraints really are. First, note that all the wrappers have had static `valueOf()` factories, and the corresponding constructors have been deprecated since Java 9.? It would be reasonable to deprecate these for removal at any time.? (These would not actually be removed, but instead made private.) Having removed the public constructors, now the only way to get your hands on an Integer is through some mechanism like `valueOf()`.? I claim now that: ??? Under such a scheme, any code outside of java.lang that synchronizes on a wrapper is ??? inherently broken That is, it is synchronizing on an object whose locking protocol you are not party to, which is not guaranteed non-aliased, and which may in fact be permanently locked by some other thread. Therefore, it is a forseeable outcome for such a sync attempt to block forever. This was the intent behind https://openjdk.java.net/jeps/169, though it wasn't obvious from the title -- the ability to mark certain objects as permanently locked. So, concrete proposal: ?- Move forward with the deprecation-for-removal of the constructors; ?- Add hortatory notes in the spec for `valueOf()` that locking on objects whose provenance you do not control is a risky business, and may well never complete; ?- Poison instances of primitive wrappers by marking their headers at creation with a bit pattern that will send locking through the slow path, where it can see that these are poisoned objects and cause lock attempts to never complete (or throw) ?- Provide a JDK-scoped option to turn off the above behavior (-Xx:AllowStupidBrokenLockingOnPrimitiveWrappers) I don't see anything stopping us from doing any of these immediately. From brian.goetz at oracle.com Wed Feb 19 19:12:30 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 19 Feb 2020 14:12:30 -0500 Subject: Locking on primitive boxes In-Reply-To: <57a9b8fe-d5ba-3566-3953-110041f62706@oracle.com> References: <57a9b8fe-d5ba-3566-3953-110041f62706@oracle.com> Message-ID: I received the following reply, which is worth reading into the record: > Brian, > > Let me add my 5 cents here. My fear is that actually removing (making > private) constructors of primitive wrappers will break a lot of code > legacy code out there. In fact, even today we use those constructors > to work around limitations in HotSpot's escape analysis and allocation > elimination. E.g. if you conditionally return the result of > `Integer.valueOf(x)` from a method, then the allocation inside of the > valueOf cannot be eliminated by HotSpot (that allocation is > conditional itself and that is too much for HotSpot to see through), > but if, on the other hand, you conditionally return the result of `new > Integer(x)`, then HotSpot consistently eliminates this allocation. > > Sincerely, > Roman Elizarov It is notable that you bring up escape analysis here.? The hiding of the concrete constructors is something we are undertaking _precisely so we can migrate these wrappers to inline classes_; making them inline classes greatly improves the compiler's ability to eliminate allocations.? So I think this concern becomes "your grand plan, which has as a benefit greatly improved escape analysis, will interfere with my workarounds to get better escape analysis." That said, it will take some time for existing uses to migrate away from these constructors -- all the more reason to deprecate them for removal earlier. On 2/19/2020 11:45 AM, Brian Goetz wrote: > One of the main impediments to migrating the primitive wrappers to be > the reference projection of an inline class is locking -- there may be > code out there that locks on instances of primitive wrappers.? Let's > look at how we might wean users off, and what the compatibility > constraints really are. > > First, note that all the wrappers have had static `valueOf()` > factories, and the corresponding constructors have been deprecated > since Java 9.? It would be reasonable to deprecate these for removal > at any time.? (These would not actually be removed, but instead made > private.) > > Having removed the public constructors, now the only way to get your > hands on an Integer is through some mechanism like `valueOf()`.? I > claim now that: > > ??? Under such a scheme, any code outside of java.lang that > synchronizes on a wrapper is > ??? inherently broken > > That is, it is synchronizing on an object whose locking protocol you > are not party to, which is not guaranteed non-aliased, and which may > in fact be permanently locked by some other thread.? Therefore, it is > a forseeable outcome for such a sync attempt to block forever. > > This was the intent behind https://openjdk.java.net/jeps/169, though > it wasn't obvious from the title -- the ability to mark certain > objects as permanently locked. > > So, concrete proposal: > > ?- Move forward with the deprecation-for-removal of the constructors; > > ?- Add hortatory notes in the spec for `valueOf()` that locking on > objects whose provenance you do not control is a risky business, and > may well never complete; > > ?- Poison instances of primitive wrappers by marking their headers at > creation with a bit pattern that will send locking through the slow > path, where it can see that these are poisoned objects and cause lock > attempts to never complete (or throw) > > ?- Provide a JDK-scoped option to turn off the above behavior > (-Xx:AllowStupidBrokenLockingOnPrimitiveWrappers) > > I don't see anything stopping us from doing any of these immediately. > > > From forax at univ-mlv.fr Wed Feb 19 19:29:12 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 19 Feb 2020 20:29:12 +0100 (CET) Subject: Locking on primitive boxes In-Reply-To: <57a9b8fe-d5ba-3566-3953-110041f62706@oracle.com> References: <57a9b8fe-d5ba-3566-3953-110041f62706@oracle.com> Message-ID: <1334803959.1421923.1582140552047.JavaMail.zimbra@u-pem.fr> > De: "Brian Goetz" > ?: "valhalla-spec-experts" > Envoy?: Mercredi 19 F?vrier 2020 17:45:38 > Objet: Locking on primitive boxes > One of the main impediments to migrating the primitive wrappers to be the > reference projection of an inline class is locking -- there may be code out > there that locks on instances of primitive wrappers. Let's look at how we might > wean users off, and what the compatibility constraints really are. > First, note that all the wrappers have had static `valueOf()` factories, and the > corresponding constructors have been deprecated since Java 9. It would be > reasonable to deprecate these for removal at any time. (These would not > actually be removed, but instead made private.) You can make them synthetic instead of private, so there will not be available at compile time but still exist at runtime. > Having removed the public constructors, now the only way to get your hands on an > Integer is through some mechanism like `valueOf()`. I claim now that: > Under such a scheme, any code outside of java.lang that synchronizes on a > wrapper is > inherently broken > That is, it is synchronizing on an object whose locking protocol you are not > party to, which is not guaranteed non-aliased, and which may in fact be > permanently locked by some other thread. Therefore, it is a forseeable outcome > for such a sync attempt to block forever. > This was the intent behind [ https://openjdk.java.net/jeps/169 | > https://openjdk.java.net/jeps/169 ] , though it wasn't obvious from the title > -- the ability to mark certain objects as permanently locked. > So, concrete proposal: > - Move forward with the deprecation-for-removal of the constructors; > - Add hortatory notes in the spec for `valueOf()` that locking on objects whose > provenance you do not control is a risky business, and may well never complete; > - Poison instances of primitive wrappers by marking their headers at creation > with a bit pattern that will send locking through the slow path, where it can > see that these are poisoned objects and cause lock attempts to never complete > (or throw) > - Provide a JDK-scoped option to turn off the above behavior > (-Xx:AllowStupidBrokenLockingOnPrimitiveWrappers) > I don't see anything stopping us from doing any of these immediately. I still think it's a bit early, i would prefer to have reified generics before deciding how to retrofit primitive wrapper, i.e. if we can still have two different behavior, box and value, from the same class. That said, i agree with what is proposed here (with synthetic instead of private), there is no point to continue to allow people to shoot themselves in the foot with synchonizing on wrapper types. R?mi From brian.goetz at oracle.com Wed Feb 19 20:32:59 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 19 Feb 2020 15:32:59 -0500 Subject: Locking on primitive boxes In-Reply-To: <1334803959.1421923.1582140552047.JavaMail.zimbra@u-pem.fr> References: <57a9b8fe-d5ba-3566-3953-110041f62706@oracle.com> <1334803959.1421923.1582140552047.JavaMail.zimbra@u-pem.fr> Message-ID: <11f44ddd-53fd-2d9f-4b4e-86edad144e1e@oracle.com> > > First, note that all the wrappers have had static `valueOf()` > factories, and the corresponding constructors have been deprecated > since Java 9.? It would be reasonable to deprecate these for > removal at any time.? (These would not actually be removed, but > instead made private.) > > > You can make them synthetic instead of private, so there will not be > available at compile time but still exist at runtime. That's a possible intermediate move (thought its not possible to express this in source code currently), to support existing binaries while providing stronger compile-time pushback. > > > I still think it's a bit early, i would prefer to have reified > generics before deciding how to retrofit primitive wrapper, i.e. if we > can still have two different behavior, box and value, from the same class. Waiting for specialized generics seems a bit extreme.? I agree that there is some uncertainty in the migration story for primitives to inlines, and we of course want to nail that down ASAP, but we also don't want to wait too long once we have our story straight, as the leads times here are long. From amalloy at google.com Thu Feb 20 01:16:35 2020 From: amalloy at google.com (Alan Malloy) Date: Wed, 19 Feb 2020 17:16:35 -0800 Subject: Locking on primitive boxes In-Reply-To: <57a9b8fe-d5ba-3566-3953-110041f62706@oracle.com> References: <57a9b8fe-d5ba-3566-3953-110041f62706@oracle.com> Message-ID: I had a look through Google's codebase, to see if we have any outlaws who are using these primitive boxes as locks. Of course, almost all locks (over 99.9% in Google's codebase) have a static type of either Object (typically a new Object() created just for its lock), or Class (typically for class-wide exclusion during a static method/initializer). It's not easy for me to discover how often these Objects are actually Integers, but what I could do was at least find cases where an Integer is explicitly used as a lock, to see if any of those are intentional, important use cases that might be worth changing Brian's proposal for. The TL;DR is: We do have a small number of cases using boxes as locks, but none of them are a good idea, and there are easy "workarounds" available if locking on a box became illegal (and which should really be applied anyway). I don't foresee any serious migration issues that we would incur based on these cases - through again, for Integers whose static type is Object, I have no data at this time, and we would probably appreciate some gradual-migration tools like a warning if the VM notices we're taking an ill-advised lock. For more information about how I came to this conclusion, read on. I searched for all statements of the form: synchronized (expr()) {...}, or a use of wait/notify/notifyAll on an expr(), and asked javac what the static type of expr() is. The only primitive-box types that appear in this list are Boolean, Integer, and Long. The Boolean cases that I looked at all have the same basic structure: someone wants to store a boolean field, but wants to update it in a critical section. Instead of doing the "right" thing and using a separate lock, such as `this` or a dedicated lock object, they just turn their boolean into a Boolean and lock on it while updating it: class Flag { Boolean active = false; void update() { synchronized (active) { active = computeNewState(); } } } This is clearly wrong, and it's easy to fix. There were a slightly more varied set of patterns around Integer locks. I identified 3 basic categories of motivations. One amusing category is "wrong on purpose", as part of some demonstration of a tool that detects locking problems. If this ever becomes illegal, we can just update the example to use an Object instead: final Integer dummy = new Integer(0); synchronized (dummy) { runExample("blocked", new Runnable() { public void run() { synchronized (dummy) { } } }); sleep(2000); } A second, fairly innocuous, case is from authors who seem unaware that you can use Class or Object for locks. Code in this category creates an Integer for the sole purpose of locking on it, and never reads or writes the field otherwise. In one example, the author notes that they wish they could use new Object() instead, but their serialization framework requires all fields to be serializable. Integer must have looked like an easy solution, but we know there were better options available. A simple example of a class in this category: class Counters { private Map values; private final Integer valuesLock = 0; void count(String k, String v) { synchronized(valuesLock) { values.put(k, v); } } } All the above cases share the problem that they introduce unnecessary contention with other objects making the same mistake. The third category I found is worse, because it also has the problem of not ensuring mutual exclusion! The Integer being locked on is reassigned while the lock is held, meaning that other threads could barge in, since they will now be locking on a different object. An example: class Sampler { private Integer count = 0; private final int frequency = 100; void maybeEmit(Object sample) { synchronized(count) { count++; } if (count % frequency == 0) { emit(sample); } } } On Wed, Feb 19, 2020 at 8:46 AM Brian Goetz wrote: > One of the main impediments to migrating the primitive wrappers to be > the reference projection of an inline class is locking -- there may be > code out there that locks on instances of primitive wrappers. Let's > look at how we might wean users off, and what the compatibility > constraints really are. > > First, note that all the wrappers have had static `valueOf()` factories, > and the corresponding constructors have been deprecated since Java 9. > It would be reasonable to deprecate these for removal at any time. > (These would not actually be removed, but instead made private.) > > Having removed the public constructors, now the only way to get your > hands on an Integer is through some mechanism like `valueOf()`. I claim > now that: > > Under such a scheme, any code outside of java.lang that > synchronizes on a wrapper is > inherently broken > > That is, it is synchronizing on an object whose locking protocol you are > not party to, which is not guaranteed non-aliased, and which may in fact > be permanently locked by some other thread. Therefore, it is a > forseeable outcome for such a sync attempt to block forever. > > This was the intent behind https://openjdk.java.net/jeps/169, though it > wasn't obvious from the title -- the ability to mark certain objects as > permanently locked. > > So, concrete proposal: > > - Move forward with the deprecation-for-removal of the constructors; > > - Add hortatory notes in the spec for `valueOf()` that locking on > objects whose provenance you do not control is a risky business, and may > well never complete; > > - Poison instances of primitive wrappers by marking their headers at > creation with a bit pattern that will send locking through the slow > path, where it can see that these are poisoned objects and cause lock > attempts to never complete (or throw) > > - Provide a JDK-scoped option to turn off the above behavior > (-Xx:AllowStupidBrokenLockingOnPrimitiveWrappers) > > I don't see anything stopping us from doing any of these immediately. > > > > From brian.goetz at oracle.com Fri Feb 21 22:16:20 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 21 Feb 2020 17:16:20 -0500 Subject: Updated SOV doc Message-ID: I've updated the Language Model doc at http://cr.openjdk.java.net/~briangoetz/valhalla/sov/02-object-model.html to reflect the discussions surrounding abstract class supertypes and their use as reference projections, as well as the `ref-default` modifier.? Original version available here for those who want to try and diff them: http://cr.openjdk.java.net/~briangoetz/valhalla/sov/02-object-model_01.html