From gavin at hibernate.org Thu Jan 1 03:26:03 2015 From: gavin at hibernate.org (Gavin King) Date: Thu, 1 Jan 2015 04:26:03 +0100 Subject: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: References: <54A4799E.3050300@oracle.com> Message-ID: Inline: On Wed, Dec 31, 2014 at 11:33 PM, Brian Goetz wrote: > *Obviously* it would be more desirable to integrate primitives and values > into generics by leaning on the existing notion of type bound, rather than > introducing all the additional complexity that we're considering. Great! So then if everyone agrees that this is the best and most correct solution, then the only problem that remains is: how do we implement it! Excellent :-) > Where the wheels start to come off the wagon is: how do we represent a > variable of type 'Any" in bytecode (a field, local variable, or method > parameter or return type)? If we can't answer that, we can't allow use of > Any in these places. And solving this problem amounts to only slightly less > than a total redesign of the JVM and bytecode architecture. So this > harmless-seeming question (couched in claims of "simpler" and "more > elegant") amount to "Why not just redesign the VM completely." Well, it seems to me that there's at least two reasonable solutions to this problem, as I'll outline below. > Languages that have attempted to unify primitives and references on the JVM, > with the existing bytecode architecture, while retaining some sort of > compatibility with existing Java idioms, have failed at doing so. (And I am > thankful to have those experiments to inform our work here!) As a concrete > example, I point you to Paul Philips' excellent "Scala War Stories" talk > from JVM Language Summit 2013, which covers the failure of such unifictions, > and more: > http://medianetwork.oracle.com/video/player/2623635250001 Yeah look, I don't want to get hung up on the problems of Scala since Scala's problems are already well known and I was a skeptic of Scala before it was fashionable. Fortunately it's pretty much a strawman in this context since I'm about the last person on earth to advocate Scala as a model for Java. > But you might say "Wait a second, C# managed to pull off this redesign of > the VM to support polymorphism over objects and primitives". And indeed > they did, and overall their solution is quite elegant. And obviously, we > must have known about this example, so why wouldn't we explore this? I'm not an expert on C#, and my suggestion isn't influenced by C#. So let's also put C# aside, and focus on what is best and most correct for _Java_. > We're concerned with it too, as I think we've made quite clear. Great! So let's fix it. > Here's the > position we're in: if we wait until we have a complete, 100% solution before > sharing anything, people throw rocks at us for doing everything behind > closed doors, but if we share our working thoughts in progress, people throw > rocks at us for being half-baked. We've chosen the latter poison, so by all > means, throw your rocks, but don't kid yourself that you've spotted > something that no one else has. (And please, check the attitude at the > door, it's just not helpful.) Look, if this document was merely a thought bubble, then I'll go back to sleep. But AFAICT, and correct me if I'm wrong, but that's actually not the case. This is, if I'm not mistaken, the second public revision of an actual proposal, and it looks like you're actually defending the approach proposed in the document. I mean, the only reason for publishing a proposal in public is to solicit feedback, and so, therefore, I'm giving you my well-intentioned feedback. Whatever you perceive my attitude to be is utterly irrelevant; either I'm right or I'm wrong, and my perceived "attitude", whatever that might be, is not an issue of interest to the Java community. So let's just set that distraction to one side, please. >> What this proposal does is introduce parametric polymorphism over >> primitive types, while leaving it impossible to abstract over >> primitives and reference types with subtype polymorphism. Thus, at the >> intersection of the two systems of abstraction, namely, *variance*, we >> get the broken behavior that a List isn't a List. > > > Indeed, we've already pointed this out, and its not pretty. All > constructive suggestions accepted. But implicitly dropping a key > requirement (like gradual migration compatibility), and then claiming > there's an obvious answer, is not really helpful. I have not proposed to drop any requirement for backward compatibility. >> I therefore suggest a different, simpler, and much more natural >> starting point for this work: stop pretending that there is no type >> Any. > > > This is a particularly funny way to put it, as it is the notion that there > *is* an Any type which requires pretending! There is simply no way (without > boxing) to represent this on the JVM as it currently stands. So what I'm going to do is offer you *two* solutions. One works with the JVM with a fairly small modification, and one requires that the JVM be enhanced to be more useful and more powerful. > But if boxing > were good enough, then we wouldn't need to do anything --we'd just write > ArrayList and be done with it! But obviously boxing isn't good > enough, since we're having this conversation. ... Where > the data hits the heap is where the boxing story (and therefore the Any > story) falls apart. So, the first possibility, as you've intuited, is that assignment of a primitive to Any results in boxing, as it does today when you assign a primitive to Object. *However*, this approach actually does not result in the problem you've described. Consider the possible concrete instantiations of List: - List - boxing, like today with List - List - same as today - List - no boxing, since we can use the same specialization approach used in the paper The only case which involves boxing is a case which you disallow in the current proposal, which means that this is not worse than the current proposal. Now, the downside of this is that it does motivate a relatively small change to the VM where invokeinterface would be applicable to both the reference and primitive specialization of List, since, for an invocation of the instantiation List, we have something that might be an instance of List or might be an instance of List. I don't sounds like an unreasonable thing to contemplate. (Of course, there are also more complicated solutions which don't require a change to the VM here, but they're uglier, and I think the best place to solve the problem is at the VM level.) The second solution is harder to implement but more useful. Since about the 1970's, some programming languages have had the notion of a "tagged" or "discriminated" union type. If you look closely, Any is just a special case of a discriminated union. So if the VM were to add an Any type, implemented as a discriminated union, no boxing would be required when working with Any. > (but please, constructively). Do you a deal: I'll be constructive, as usual, and you'll stop being defensive? OK? > And, feel free to prove us wrong! Try implementing the changes you are > envisioning in the JVM, and show how they can get us to the goal! No, I'm not going to implement this myself. I already have much more than a fulltime job with Ceylon. And, frankly, it's unreasonable for Oracle to demand that any criticism/feedback of its proposals for enhancements to the Java platform be accompanied by implementation. I don't expect my users to submit a patch with every bug report / feature request, and neither should you. -- Gavin King gavin at ceylon-lang.org http://profiles.google.com/gavin.king http://ceylon-lang.org http://hibernate.org http://seamframework.org From gavin at hibernate.org Fri Jan 2 13:46:58 2015 From: gavin at hibernate.org (Gavin King) Date: Fri, 2 Jan 2015 14:46:58 +0100 Subject: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: References: <54A4799E.3050300@oracle.com> Message-ID: On Wed, Dec 31, 2014 at 11:33 PM, Brian Goetz wrote: > For the record, the reason we rejected a unified 'Any' type is: it is a > fiction. So this sentence has been bothering me for a day or so, and after reflection, I think there's three reasons why it's a very bad way to frame the problem. First: if Any is a "fiction" in this sense, then all of Java generics is also a fiction. It's therefore not surprising that, in thinking about how to enhance a "fictional" facility, that other "fictional" constructs would also arise. Second: but in fact it's perfectly normal to have constructs that are "fictional" in this sense. What Any means at the level of the bytecode and virtual machine is something that is completely abstracted by the Java language and almost always irrelevant to the programmer. A Java programmer doesn't write or read bytecode and 98% of Java developers never even look at it. There's only one way in which it could become relevant, and that's if Any entailed an unacceptable loss of performance, which, as I've argued in my previous email, it does not, at least as far as I can see. Third: if it's truly unacceptable for Any to be "fictional" in this sense, which I think is clearly not the case, then there's an obvious solution as both R?mi and I have pointed out: implement an Any type in the JVM as a discriminated union. And indeed, once you start looking closely, it seems to me that this might be the best option of all. With Any in the JVM, then perhaps (of course I've not thought through all the details) all the whole "specialization" machinery - which I think we can agree is somewhat ugly - might be completely unnecessary! Therefore I arrive at two conclusions: - A "fictional" Any is perfectly defensible. It doesn't preclude an implementation based on specialization, AFAICS, and it's not problematic in any way in which Java generics aren't already problematic. - On the other hand, thinking about Any leads us to consider a different possible implementation that doesn't depend on "specialization", which makes the VM more flexible and more powerful, which naturally avoids duplicating .class files, and which is frankly more natural all over. I can imagine this approach not working out for some reason or reasons, since I've of course not totally thought it through, but if it won't work, I'd definitely like to know precisely *why*. It's quite possible, of course, that you've already invested a lot of time into making Any work, and found that it doesn't work because of certain obscure, highly technical points that I have not yet stumbled upon. But if that's the case, then neither the design document for specialization, nor your above email, hints at what these issues might be. At the very least they should be highlighted in the paper so that the Java community understands what they are. Happy new year! -- Gavin King gavin at ceylon-lang.org http://profiles.google.com/gavin.king http://ceylon-lang.org http://hibernate.org http://seamframework.org From brian.goetz at oracle.com Fri Jan 2 16:08:03 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 02 Jan 2015 11:08:03 -0500 Subject: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: References: <54A4799E.3050300@oracle.com> Message-ID: <54A6C263.6090505@oracle.com> >> For the record, the reason we rejected a unified 'Any' type is: it is a >> fiction. > > So this sentence has been bothering me for a day or so Sure, I can unpack this a bit. Not all fictions are harmful. Many features are mere sugar, and sugar is tasty when used judiciously. Enums, constructor chaining, and checked exceptions are good examples; they get sugared away by the compiler. And they are good fictions! There's a perfectly reasonable and performant way to represent them in bytecode, so the language-level abstraction maps cleanly to existing (but different, and -- critically -- simpler) VM abstractions, and the user doesn't notice or care. (Check out my recent Devoxx keynote, where I referred to these positively as "language fictions".) And yes, generics are a fiction too. There's more diverse opinion on whether they are a good fiction or not; erased generics do map *mostly* cleanly to the VM (bridge methods excepted) and they perform well, but people do find erasure to be pretty unnatural, and when they find out they've been "fooled" in this way, sometimes they get pretty upset. On the other hand, an Any relative to to today's JVM would be a harmful fiction, because it *seems* so natural, and yet, our tools for representing it in bytecode would have serious impedence mismatches with user expectations. This is bait-and-switch for users, and a lifetime of pain for implementors. (I had more to say about this topic at my JVMLS keynote from this year; video is online.) > It's quite possible, of course, that you've already invested a lot of > time into making Any work, and found that it doesn't work because of > certain obscure, highly technical points that I have not yet stumbled > upon. Yeah, I think is pretty close to right (except for the assumption that the only reason we'd reject it is some obscure technical point you've not yet thought of -- we reject all sorts of things that are technical feasible, for reasons of compatibility, consistency, maintainability, philosophy, resource availability, priority, security, or just plain "not good enough", and even if the reasons were perfectly transparent, you still might not agree with them.) But, yes, we've explored it in substantial depth, and we still don't see a feasible and acceptable path. (And we probably will continue to think about it; sometimes your first ten ideas are wrong but that doesn't stop you from continuing to try.) > But if that's the case, then neither the design document for > specialization, nor your above email, hints at what these issues might > be. At the very least they should be highlighted in the paper so that > the Java community understands what they are. Sure, in a perfect world, our brains would be available on the internet for direct neural queries, and all curiosities could be satisfied (or maybe not.) In the world we've got, we try our best to balance moving things forward with providing after-the-fact explanations of our thought processes and what we've explored. But this is not always possible in the detail, time-frame, or angle of view that everyone with a question would like. That may suck, but we need to focus our energy primarily on the search for the right answers. From gavin at hibernate.org Fri Jan 2 16:44:54 2015 From: gavin at hibernate.org (Gavin King) Date: Fri, 2 Jan 2015 17:44:54 +0100 Subject: Fwd: Proposal for generics over primitives needs a rethink Message-ID: On Fri, Jan 2, 2015 at 5:08 PM, Brian Goetz wrote: > On the other hand, an Any relative to to today's JVM would be a harmful > fiction, because it *seems* so natural, and yet, our tools for representing > it in bytecode would have serious impedence mismatches with user > expectations. This is bait-and-switch for users, and a lifetime of pain for > implementors. (I had more to say about this topic at my JVMLS keynote from > this year; video is online.) OK, so then please: 1. first, explain what you think the user expectations for Any are, and then 2. give us some code examples where those user expectations would be violated according to the semantics I've suggested for Any. >> It's quite possible, of course, that you've already invested a lot of >> time into making Any work, and found that it doesn't work because of >> certain obscure, highly technical points that I have not yet stumbled >> upon. > > > Yeah, I think is pretty close to right (except for the assumption that the > only reason we'd reject it is some obscure technical point you've not yet > thought of -- we reject all sorts of things that are technical feasible, for > reasons of compatibility, consistency, maintainability, philosophy, resource > availability, priority, security, or just plain "not good enough", and even > if the reasons were perfectly transparent, you still might not agree with > them.) > > But, yes, we've explored it in substantial depth, and we still don't see a > feasible and acceptable path. (And we probably will continue to think about > it; sometimes your first ten ideas are wrong but that doesn't stop you from > continuing to try.) OK, so that's all well and good, but you've essentially dodged the question here: *what*, precisely, is it that makes Any not: - compatible, - consistent, - secure, or - otherwise not "good enough". Because I've asserted that there are at least two ways to implement Any that are, as far as I can tell, compatible, consistent, secure, and "good enough", and you've not responded with any concrete explanation for why they're not. All I'm asking for is a concrete explanation of what is wrong with my suggestion instead of handwaving and complaints about my "attitude". Surely that's not too much too ask? I mean, you've asked me to be "constructive", and I'm making a concrete, constructive, technical suggestion. You say you've explored the idea in "substantial depth" and discovered it to be "unfeasible". So surely, after that in-depth exploration, you must have arrived at some pretty good concrete conclusions that you could have documented in the proposal for "specializations", and that you could have included in your responses to my email. >> But if that's the case, then neither the design document for >> specialization, nor your above email, hints at what these issues might >> be. At the very least they should be highlighted in the paper so that >> the Java community understands what they are. > > > Sure, in a perfect world, our brains would be available on the internet for > direct neural queries, and all curiosities could be satisfied (or maybe > not.) In the world we've got, we try our best to balance moving things > forward with providing after-the-fact explanations of our thought processes > and what we've explored. But this is not always possible in the detail, > time-frame, or angle of view that everyone with a question would like. That > may suck, but we need to focus our energy primarily on the search for the > right answers. I don't understand. Do you or do you not have a concrete explanation for what is wrong with Any? - If you do, then please share it with us, because it's missing from the paper which I and others have taken the time to read and understand and evaluate. - Otherwise, if you don't have any such explanation, then there's something missing here, and it seems to be that Any should be back on the table for consideration. - Or are you saying that such an explanation exists, but you're for some reason unable to share it with me? I've always considered that we, as language designers, have a professional and ethical responsibility to be able to justify the decisions we make. I personally make a point of always having some sort of technical justification for every decision I make, and take time to communicate the reasoning to folks from our user community who ask. -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From brian.goetz at oracle.com Fri Jan 2 17:37:21 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 02 Jan 2015 12:37:21 -0500 Subject: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: References: Message-ID: <54A6D751.2080600@oracle.com> > All I'm asking for is a concrete explanation This all sounds very reasonable, except that it's not actually all that reasonable at all. > - If you do, then please share it with us, because it's missing from > the paper which I and others have taken the time to read and > understand and evaluate. We have to divide our time between exploring, implementing, and communicating. The demand for communication is infinite, as you've so ably demonstrated; if we provided all the communication that was demanded, there'd be no time left for doing, and we'd all lose. So, I'm sorry if my explanation seems inadequate to you. Perhaps you could look at it as half full rather than half empty: you got quite a bit of attention, you got your idea heard, you got confirmation that these ideas had been seriously examined before, and you got some explanation -- all at the time basically of your choosing. That's a lot; sorry if its not enough for you. > - Otherwise, if you don't have any such explanation, then there's > something missing here, and it seems to be that Any should be back on > the table for consideration. The bar for "idea should be on the table" is not "Gavin thinks it is good enough." For that matter, I'm not even sure what it means to be "on the table" (or whether this idea is, or isn't on the table, or even where the table is.) But we've discussed it and considered it to no small degree, and will probably consider it some more. I think that's all the table-hood you can reasonable ask for. > - Or are you saying that such an explanation exists, but you're for > some reason unable to share it with me? We've shared some of our thoughts. Hopefully, enough to convince you and others that this is not just something that never occurred to us, but that it's been examined and some smart people have some serious concerns. If that's not enough for you, I'm sorry. We try very hard to balance the various responsibilities of "doing" vs "communicating". I understand that we can't make everyone happy. From gavin at hibernate.org Fri Jan 2 18:13:50 2015 From: gavin at hibernate.org (Gavin King) Date: Fri, 2 Jan 2015 19:13:50 +0100 Subject: Fwd: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: <54A6D751.2080600@oracle.com> References: <54A6D751.2080600@oracle.com> Message-ID: On Fri, Jan 2, 2015 at 6:37 PM, Brian Goetz wrote: >> All I'm asking for is a concrete explanation > > > This all sounds very reasonable, except that it's not actually all that > reasonable at all. To be clear: you're saying that it's not reasonable to ask for an explanation of why the most obvious and apparently natural way to evolve the type system of Java to support abstraction of generic types over primitives is not feasible? >> - If you do, then please share it with us, because it's missing from >> the paper which I and others have taken the time to read and >> understand and evaluate. > > > We have to divide our time between exploring, implementing, and > communicating. The demand for communication is infinite, as you've so ably > demonstrated; if we provided all the communication that was demanded, > there'd be no time left for doing, and we'd all lose. So, I'm sorry if my > explanation seems inadequate to you. Perhaps you could look at it as half > full rather than half empty: you got quite a bit of attention, you got your > idea heard, you got confirmation that these ideas had been seriously > examined before, and you got some explanation -- all at the time basically > of your choosing. That's a lot; sorry if its not enough for you. Brian, this is one of the most astonishing statements I've seen from a spec lead all the time I've been involved in the JCP. I did not come here seeking attention; I did not come here seeking unverifiable assertions unaccompanied by evidence; I did not come here seeking personal attacks on my "attitude". I came here with what seemed to me a useful and concrete bit of feedback about a document you had published, apparently, at least it seemed to me, with the purpose of soliciting response from the community. I therefore feel entitled to ask again: what is the technical reason that my suggestion is unfeasible. If you can just answer this question for me, then we can move on from this frankly farcical discussion to something much more productive. >> - Otherwise, if you don't have any such explanation, then there's >> something missing here, and it seems to be that Any should be back on >> the table for consideration. > > > The bar for "idea should be on the table" is not "Gavin thinks it is good > enough." For that matter, I'm not even sure what it means to be "on the > table" (or whether this idea is, or isn't on the table, or even where the > table is.) Again, your responses to my technical suggestions and questions are personal criticisms of me. I don't understand where that's coming from or what's motivating you to treat me like this. FTR: we've never interacted before; my messages in this thread have been expressed in a measured tone; I've never once criticized you personally, nor your work on Java SE, not even in private; I'm one of the folks out here in the Java community who often defends Java in public forums; I've made significant contributions to several JSRs; I'm an Oracle Java Champion, and was a JCP star spec lead; I lead a project that develops one of the prominent alternative languages for the JVM; I work for a company which is a Java licensee and which contributes to Open JDK. There should therefore be no question that my motivations here are well-intentioned and constructive. But even if none of the above were true, just like any other member of the Java community, I have a right to a respectful response to my questions. I honestly have not a clue what could be motivating you to try and brush me off like this. So, can we just drop this whole subthread, do a full reset here, and get back to the much more interesting technical issues, *please*? Because I think this is potentially a really great opportunity to make Java better, and I want to be convinced it's being done the Right Way. > But we've discussed it and considered it to no small degree, and > will probably consider it some more. I think that's all the table-hood you > can reasonable ask for. I think I can quite reasonably request that this issue be discussed here, in public. That's what being "open" is about. -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From forax at univ-mlv.fr Sat Jan 3 15:12:08 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 03 Jan 2015 16:12:08 +0100 Subject: Proposal for generics over primitives needs a rethink In-Reply-To: References: <54A4799E.3050300@oracle.com> <54A47F91.4040701@univ-mlv.fr> Message-ID: <54A806C8.7050901@univ-mlv.fr> On 01/01/2015 04:22 AM, Gavin King wrote: > On Wed, Dec 31, 2014 at 11:58 PM, Remi Forax wrote: > >> Just to add that there is another cost with the C# approach, you must have >> exactly the same bytecode for Set and Set so you can not by >> example, specialize Set to use a bitset. > I don't follow what you're saying here. Set is an interface in Java. > There can be multiple implementations of Set. s/Set/HashSet > >> About the introduction of a type Any in the VM, it means that now we have a >> type that can store a primitive or a reference, >> we know how to represent this kind of type using tag bits (like V8 does) or >> Nan boxing (like Mozilla"s *Monkey VM does). >> Basically, representing Any in the VM is like asking for transform a Java VM >> into something very close to a Javascript VM. > Exactly! Precisely the solution I prefer! This is, after all, 70's tech :-) yes :) but it doesn't mix well with late 90's GC algorithms. R?mi From forax at univ-mlv.fr Sat Jan 3 16:10:15 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 03 Jan 2015 17:10:15 +0100 Subject: Fwd: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: References: <54A6D751.2080600@oracle.com> Message-ID: <54A81467.8010207@univ-mlv.fr> On 01/02/2015 07:13 PM, Gavin King wrote: > So, can we just drop this whole subthread, do a full reset here, and > get back to the much more interesting technical issues, *please*? > Because I think this is potentially a really great opportunity to make > Java better, and I want to be convinced it's being done the Right Way. so full reset. The Any problems. The first major problem of Any is that it introduce a new primitive kind, a lot of codes relies on the fact that you can only have boolean, byte, short, char, int, long, double and Object in Java. Introducing a new type will make a lot of code to behave weirdly. The second major problem is how to implement it. It can be implemented as a compiler fiction only like in C# or in Scala, but in that case apart that one can write ArrayList, it will rely on a form of boxing. It can be implemented in the VM. It seems to be the best solution but the engineering cost is insane and in fact Any will not solve the specialization problem. Why the engineering cost is very high ? Currently most of the Java VM implementations rely on the fact that you know upfront (without executing the code) if a field or a local variable is a reference or an object. In term of GC, there is a very clear distinction between the precise GC algorithms used in Java VM and the conservative GC algorithms used by example in C. Technically, with Any, you can implement something in the middle between these two categories of GC, nevertheless, it changes the category of GCs, Java will be able to use. Any will not solve the specialization problem. Let say we now we have introduce Any, so we can create an array of Any. But an array of Any can be at runtime by subtyping either an array of doubles or an array of objects. But by doing this we introduce a subtyping relationship between things that are not structurally equivalent. A possible solution is to align all cells of all arrays to 64 bits which is a very bad property because even if you don't use Any, you pay the cost of that you may use it. Another solution, the one used by V8, is to box all doubles (and long in Java) when you store them in an array*. So an array of Any will not be magically an array of ints, an array of doubles and an array of objects without some trade-offs. The specialization as defined in the 'State of Specialization' http://cr.openjdk.java.net/~briangoetz/valhalla/specialization.html seems in my opinion a good way to go forward. regards, R?mi * that's why JavaScript defined a bunch of typed arrays https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays From twhitmore.nz at gmail.com Sun Jan 4 05:36:53 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Sun, 4 Jan 2015 18:36:53 +1300 Subject: more background to List specializing List Message-ID: Hi Brian, Gavin, people, Thanks for your comments on the idea of List specializing from List. My attention was drawn to the simpler of Gavin's proposals -- 'Any' implements as Object, but can be specialized. (Second 'tagged union' proposal I feel would be too complex.) I know our focus in this current phase is on "Layers", but would like to educate myself further about the general background of issues with Any. To this end I have so far investigated the following videos & resources: - Paul Phillips, Scala War Stories http://medianetwork.oracle.com/video/player/2623635250001 - Brian Goetz - Stewardship: the Sobering Parts https://www.youtube.com/watch?v=2y5Pv4yN0b0 Brian, I did want to research the JVMLS 2014 keynote you mentioned; is that you with John Rose "Evolving the JVM"? - John Rose & Brian Goetz, Evolving the JVM http://medianetwork.oracle.com/video/player/3730888982001 Let me know if this is the video you referred to, and if there were any other good resources? As an aside, I was very interested in the "Stewardship" video, and appreciated the important perspectives Brian conveys & is responsible for... I also got a little bit more insight into Brian's concern with serialization! All the above videos very informative & recommended. Regards, Thomas Whitmore From brian.goetz at oracle.com Sun Jan 4 18:15:24 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 04 Jan 2015 13:15:24 -0500 Subject: more background to List specializing List In-Reply-To: References: Message-ID: <54A9833C.4020109@oracle.com> > Brian, I did want to research the JVMLS 2014 keynote you mentioned; is > that you with John Rose "Evolving the JVM"? > > - John Rose & Brian Goetz, Evolving the JVM > http://medianetwork.oracle.com/video/player/3730888982001 Yes, this is the one I was referring to. While the technical picture has obviously progressed since we gave this talk (yay!), there's a lot of good background there about how we evaluate various options, and what constitutes a "good" solution. From brian.goetz at oracle.com Sun Jan 4 18:40:40 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 04 Jan 2015 13:40:40 -0500 Subject: more background to List specializing List In-Reply-To: References: Message-ID: <54A98928.8030402@oracle.com> > I know our focus in this current phase is on "Layers", but would like to > educate myself further about the general background of issues with Any. Let me correct slightly. We're investigating "layers" or "conditional methods" not because this is the feature we went looking for, but because it is a possible and believed-practical answer to the more general question of "what generic type system can provide us the gradual migration compatibility we are looking for." This is not unlike what happened in 8; we didn't do default methods because we wanted multiple inheritance of behavior; we did it because we needed *interface evolution*, and of the dozen or so possible ways to get there that we evaluated, this one had the best combination of characteristics. So far we've gotten feedback that people don't like the balance of "user model weight" and "general utility" for layers (feedback received loud and clear, no need to reopen this.) More generally, I think there's a lot of tendency to assume here that these writeups constitute *decisions*, which they do not. These writeups are snapshots of where our head is at this week, which we provide so people can get a peek about what we're thinking about. But we're a long way from the end of this road. From twhitmore.nz at gmail.com Sun Jan 4 22:15:00 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Mon, 5 Jan 2015 11:15:00 +1300 Subject: more background to List specializing List In-Reply-To: <54A98928.8030402@oracle.com> References: <54A98928.8030402@oracle.com> Message-ID: Thanks, this is very interesting. And I see that choosing the "functional interface" route for Java 8, has actually worked much better than starting to pull on that "functional types" string, which would have brought _everything_ into the typesystem! At the moment, I'm interested in the questions around top of the type heirarchy -- List, List, List typing. I'm very happy with List, as a practical implementation strategy for List when T is a reftype. But I'm not sure how or whether this can address the inheritance problems Gavin was referring to. I'd like to look into that further. [I'm also interested in Any vs Object since my day jobs are involved in framework & persistence architecture, and I have a general interest in car-crash stories :-)] And on the other side of the coin, if we did have an "abnormal" inheritance situation it might be better to just acknowledge that honestly in syntax, so nobody need be confused. new ArrayList new specialized ArrayList // actually creates a derivative class Tossing up longer syntax, versus explicit acknowledgement of a shiny new performance feature & it's different behavior. Regards, Thomas On Mon, Jan 5, 2015 at 7:40 AM, Brian Goetz wrote: > I know our focus in this current phase is on "Layers", but would like to >> educate myself further about the general background of issues with Any. >> > > Let me correct slightly. We're investigating "layers" or "conditional > methods" not because this is the feature we went looking for, but because > it is a possible and believed-practical answer to the more general question > of "what generic type system can provide us the gradual migration > compatibility we are looking for." > > This is not unlike what happened in 8; we didn't do default methods > because we wanted multiple inheritance of behavior; we did it because we > needed *interface evolution*, and of the dozen or so possible ways to get > there that we evaluated, this one had the best combination of > characteristics. > > So far we've gotten feedback that people don't like the balance of "user > model weight" and "general utility" for layers (feedback received loud and > clear, no need to reopen this.) > > > More generally, I think there's a lot of tendency to assume here that > these writeups constitute *decisions*, which they do not. These writeups > are snapshots of where our head is at this week, which we provide so people > can get a peek about what we're thinking about. But we're a long way from > the end of this road. > > From scolebourne at joda.org Sun Jan 4 23:00:33 2015 From: scolebourne at joda.org (Stephen Colebourne) Date: Sun, 4 Jan 2015 23:00:33 +0000 Subject: more background to List specializing List In-Reply-To: <54A98928.8030402@oracle.com> References: <54A98928.8030402@oracle.com> Message-ID: The specialization document include the phrase "If we were concerned only about primitive instantiations, we could consider some sort of "tagged fixnum" approach. However, this approach essentially reverts to boxing for arbitrary value types, so it does not get us to our goal." Should we take that to indicate that if value types were taken out of the picture, and the only goal was to handle the existing 8 primitive types, that there might be a potentially simpler solution - one where List could be a sub-type of List without boxing? Stephen On 4 January 2015 at 18:40, Brian Goetz wrote: >> I know our focus in this current phase is on "Layers", but would like to >> educate myself further about the general background of issues with Any. > > > Let me correct slightly. We're investigating "layers" or "conditional > methods" not because this is the feature we went looking for, but because it > is a possible and believed-practical answer to the more general question of > "what generic type system can provide us the gradual migration compatibility > we are looking for." > > This is not unlike what happened in 8; we didn't do default methods because > we wanted multiple inheritance of behavior; we did it because we needed > *interface evolution*, and of the dozen or so possible ways to get there > that we evaluated, this one had the best combination of characteristics. > > So far we've gotten feedback that people don't like the balance of "user > model weight" and "general utility" for layers (feedback received loud and > clear, no need to reopen this.) > > > More generally, I think there's a lot of tendency to assume here that these > writeups constitute *decisions*, which they do not. These writeups are > snapshots of where our head is at this week, which we provide so people can > get a peek about what we're thinking about. But we're a long way from the > end of this road. > From brian.goetz at oracle.com Sun Jan 4 23:20:03 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 04 Jan 2015 18:20:03 -0500 Subject: more background to List specializing List In-Reply-To: References: <54A98928.8030402@oracle.com> Message-ID: <54A9CAA3.6070405@oracle.com> More likely: if we took value types out of the picture, we might well not bother with specialization at all; we've limped along without it for a long time, and there have been other solutions (Trove, GS-Collections) that provide 80% of the value of primitive-specialized collections for like 1% of the work. But value types tip the balance from "unpleasant but tolerable with grumbling" to "unacceptable." Zooming back, though, I think what you're really saying is: the part where List is not a subtype of List is really the part you dislike the most, because it lives in this weird middle ground between "all instantiations of List have a common supertype" (Java 5 generics) and "no instantiations of list have a common supertype" (C++). You are probably OK with either all (what you've got now) or nothing (if you hadn't had ten years of Java generics training you on homogeneous translation), but what really freaks you out is this half-here, half-there, that forces the user to be aware that reference instantiations are erased and value instantiations are reified, with all that entails. Is that really your concern here? On 1/4/2015 6:00 PM, Stephen Colebourne wrote: > The specialization document include the phrase "If we were concerned > only about primitive instantiations, we could consider some sort of > "tagged fixnum" approach. However, this approach essentially reverts > to boxing for arbitrary value types, so it does not get us to our > goal." > > Should we take that to indicate that if value types were taken out of > the picture, and the only goal was to handle the existing 8 primitive > types, that there might be a potentially simpler solution - one where > List could be a sub-type of List without boxing? > > Stephen > > > On 4 January 2015 at 18:40, Brian Goetz wrote: >>> I know our focus in this current phase is on "Layers", but would like to >>> educate myself further about the general background of issues with Any. >> >> >> Let me correct slightly. We're investigating "layers" or "conditional >> methods" not because this is the feature we went looking for, but because it >> is a possible and believed-practical answer to the more general question of >> "what generic type system can provide us the gradual migration compatibility >> we are looking for." >> >> This is not unlike what happened in 8; we didn't do default methods because >> we wanted multiple inheritance of behavior; we did it because we needed >> *interface evolution*, and of the dozen or so possible ways to get there >> that we evaluated, this one had the best combination of characteristics. >> >> So far we've gotten feedback that people don't like the balance of "user >> model weight" and "general utility" for layers (feedback received loud and >> clear, no need to reopen this.) >> >> >> More generally, I think there's a lot of tendency to assume here that these >> writeups constitute *decisions*, which they do not. These writeups are >> snapshots of where our head is at this week, which we provide so people can >> get a peek about what we're thinking about. But we're a long way from the >> end of this road. >> From gavin at hibernate.org Mon Jan 5 00:57:46 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 01:57:46 +0100 Subject: Fwd: more background to List specializing List In-Reply-To: References: <54A98928.8030402@oracle.com> Message-ID: Right, so this is the biggest objection to the idea of implementing Any at the VM level: it would let you avoid boxing for the built-in primitives, which is big win, obviously, but not for arbitrary value types, which would still need boxing. How strong of an objection this is depends upon how often you think value objects will occur compared to primitives. That's a question I've been turning over in my head the last couple of days, without being able to really arrive at a firm conclusion by intuition alone. Looking at Java as it exists today, it might be that Optional is the one thing that really tips the balance. But on the other hand, it's hard to say what people are going to do with Java once value types exist; perhaps value types will eventually become the default choice for most user-written classes! The other thing floating around in the back of my mind is that even if a VM-level Any isn't that useful for *Java*, it would still surely be extremely useful for other JVM languages. I can imagine that folks running dynamic langs on the JVM would just *love* to be able to use an array of type Any[], and have that efficiently store numeric values without boxing. The "specialization" approach doesn't seem like it would help them as much. (This is speculation on my part; I've not tried to compile dynamically-typed code for the JVM myself.) On Mon, Jan 5, 2015 at 12:00 AM, Stephen Colebourne wrote: > The specialization document include the phrase "If we were concerned > only about primitive instantiations, we could consider some sort of > "tagged fixnum" approach. However, this approach essentially reverts > to boxing for arbitrary value types, so it does not get us to our > goal." -- Gavin King gavin at ceylon-lang.org http://profiles.google.com/gavin.king http://ceylon-lang.org http://hibernate.org http://seamframework.org -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From gavin at hibernate.org Mon Jan 5 00:58:27 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 01:58:27 +0100 Subject: Fwd: more background to List specializing List In-Reply-To: References: <54A98928.8030402@oracle.com> Message-ID: On Mon, Jan 5, 2015 at 12:00 AM, Stephen Colebourne wrote: > Should we take that to indicate that if value types were taken out of > the picture, and the only goal was to handle the existing 8 primitive > types, that there might be a potentially simpler solution - one where > List could be a sub-type of List without boxing? Stephen, List can't be a List. (Integer is the type int|null, essentially.) You mean List would be a List, right? -- Gavin King gavin at ceylon-lang.org http://profiles.google.com/gavin.king http://ceylon-lang.org http://hibernate.org http://seamframework.org From gavin at hibernate.org Mon Jan 5 00:59:24 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 01:59:24 +0100 Subject: Fwd: more background to List specializing List In-Reply-To: References: <54A98928.8030402@oracle.com> Message-ID: On Mon, Jan 5, 2015 at 1:48 AM, Gavin King wrote: > You mean List would be a List, right? And, by the way, I still don't see why this would not be doable *even with value types in the picture*. The concrete types ArrayList and ArrayList are necessarily disjoint. Therefore, implementation based on specialization is still workable, whether we're talking about primitives or value types. The only issue that comes into play is when you invoke get() on a wildcard instantiations like ArrayList. Then, in this case, you need to make whatever bytecode-level invokexxxxx instruction we have smart enough to so that we dispatch to the right specialized implementation of get(). But that to me sounds totally doable. Doesn't invokedynamic already do that? -- Gavin King gavin at ceylon-lang.org http://profiles.google.com/gavin.king http://ceylon-lang.org http://hibernate.org http://seamframework.org From twhitmore.nz at gmail.com Mon Jan 5 01:08:46 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Mon, 5 Jan 2015 14:08:46 +1300 Subject: Remi's comments on Any Message-ID: Thanks Remi. > The second major problem is how to implement it. > It can be implemented as a compiler fiction only like in C# or in Scala, > but in that case apart that one can write ArrayList, it will > rely on a form of boxing. I am very happy to disregard the "tagged union" approach & not actually implement an Any primitive in the VM. But what if we introduce Any as a 'nominal' type in the type-system, but implemented as a reftype/ or boxed object with equals() and hashCode() methods. That would give us the exact same performance of today for unspecialized code, but a better type-system to describe ArrayList descending from ArrayList. Would that help us? I am _totally_ happy with reftypes & boxing as our base case, only code which has been specialized is expected to perform better, performance on List is not at issue. I understand that, both in regard of API & member types (especially array members), we will actually need to transform or derive a separate version of the class. I don't believe we can get away from that; that's why it's called "specialization". ArrayList.class != ArrayListclass Regards, Thomas Whitmore From scolebourne at joda.org Mon Jan 5 01:10:26 2015 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 5 Jan 2015 01:10:26 +0000 Subject: more background to List specializing List In-Reply-To: <54A9CAA3.6070405@oracle.com> References: <54A98928.8030402@oracle.com> <54A9CAA3.6070405@oracle.com> Message-ID: On 4 January 2015 at 23:20, Brian Goetz wrote: > Zooming back, though, I think what you're really saying is: the part where > List is not a subtype of List is really the part you dislike the most, > because it lives in this weird middle ground between "all instantiations of > List have a common supertype" (Java 5 generics) and "no instantiations of > list have a common supertype" (C++). You are probably OK with either all > (what you've got now) or nothing (if you hadn't had ten years of Java > generics training you on homogeneous translation), but what really freaks > you out is this half-here, half-there, that forces the user to be aware that > reference instantiations are erased and value instantiations are reified, > with all that entails. > > Is that really your concern here? My concerns are, *at this point*, somewhat larger than that. Right now, my concern is that the combination of changes may well be making Java as a whole worse, not better. Right now, I'm in the camp that I'd rather reject adding value types because the negatives appear to outweigh the positives. I do, of course expect this judgement to change as things develop, but it is a big concern I've been mulling on over the holiday period. The parts I like include - JVM optimisations for types that don't require a synchronization lock - greater locality of data, within the parent object - the potential for arrays of value types - maybe ClassDynamic might be useful, but its very hard to tell at this point - distinguishing types that don't have identity and optimising based on that The parts I don't like include - making old code incompatible with new code (see below) - that List is not a subtype of List, List or List - that Object changes its meaning (see below) - that List changes its meaning (see below) - that instanceof changes its meaning - that Class changes its meaning - that there is more "weirdness" not less - that the thrust is to create more primitive types, not unify those we've got - that there is too much concern on memory/performance and not enough on developer productivity - Java isn't C Thinking about the negatives, a lot relate to primitive types always causing problems in code today (typically framework code). With 8 special cases today, its managable. With an entire world of special cases, its not. Of course the plan is to make them not special cases, but what I'm seeing so far is making the bifurcation bigger not smaller (worse not better). I have a particular problem with the proposal where IMO it makes old code incompatible with new code. Consider an OSS library released for JDK8 that has a method process(Object). The intent is that the method accepts anything, we don't care what the type is. When used with JDK10/11 it still accepts anything, except that instead of a small, known number of types being boxed (for which the library might have a process(int) overload, there are now *lots* of boxing operations. Technically, thats compatible, but in practical terms the boxing makes it incompatible (performance/gc). Of course the situation with process(List) is worse again. Here I want to accept any list, I don't care what type. In JDK 10/11 it now won't even work. Callers won't be able to pass in a List. (As opposed to IntList which can be passed in providing it extends List, and which could be handled specially internally within process(List) using instanceof if necessary). Note that while the method process(List) could be fixed to work by using , OSS libraries don't work like that - they can't just have a new release as they have to still support developers who can't upgrade. ie. I fear that value types will badly bifurcate the ecosystem. TLDR, Object is *already* the Any type, and List is *already* the List. Breaking that is a big problem for me. Given how it looks that value types would actually work in the JVM, thats why I'd rather have no value types than the ones currently on offer. On your original question about List specifically, I've coded one piece of source code so I have one class. 1:1. The type is unimportant most of the time - its a compiler fiction. A List is a List is a List. Thus its not the half-there half-not aspect that bothers, but losing 1:1. Some other thoughts and vague ideas as I've been mulling on this. Separating synchronization from Object seems like a Good Thing. Possible approach: add interface Lockable with locking methods from Object. Remove methods from Object using JDK 9 as a stepping stone via deprecation. Use classloading/module linking tricks to add Lockable interface to all classes that were compiled in JDK 8 or earlier. Fixing up the List.remove(int) vs List.remove(Object) problem seems like a Good Thing. But its a more general problem: Why not just rename the method? Use JDK 9 module linker step to bind the old method name to the new method name. (Being able to rename methods and be backwards compatible would be a major gain for all large Java systems). Same renaming strategy applies to many of the problem cases under discussion. Special casing the 8 primitive types in generics, with List extending List. (Hence my original question). Instead of value types, some form of "packed objects" seems like an alternative that might have less issues. By packed objects I mean where the state of the child object is embedded in the parent object. (Child objects are not seen by gc, have no identity and are copied where necessary, Null handled with bit flag where necessary. Variables extended to be able to point to a child within a parent or an item in an array. And yes, this is super-vague, and could be worse than value types). All of the above focussed on providing similar results to developers without the seemingly evil value types. I'm very aware of where you are in this whole process, so there is no need to respond to each of my vague thoughts (and I'd encourage others not to jump in to avoid wasting too much time here). I do trust that things will improve. But I'm not currently optimistic that they will improve enough. Stephen From scolebourne at joda.org Mon Jan 5 01:18:13 2015 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 5 Jan 2015 01:18:13 +0000 Subject: more background to List specializing List In-Reply-To: References: <54A98928.8030402@oracle.com> Message-ID: On 5 January 2015 at 00:44, Gavin King wrote: > How strong of an objection this is depends upon how often you think > value objects will occur compared to primitives. That's a question > I've been turning over in my head the last couple of days, without > being able to really arrive at a firm conclusion by intuition alone. Given that Java has no convenient bean/property support but value types will likely provide an equals/hashCode/toString, I believe that they will be widely used as domain objects, even for objects way bigger than the size sweet-spot of value types. This opinion make me more concerned about how they interoperate with libraries pre-JDK 10/11. Stephen PS. On List vs List I'm sure you're right... From gavin at hibernate.org Mon Jan 5 01:20:43 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 02:20:43 +0100 Subject: Fwd: Remi's comments on Any In-Reply-To: References: Message-ID: On Mon, Jan 5, 2015 at 2:08 AM, Thomas W wrote: > I am very happy to disregard the "tagged union" approach & not actually > implement an Any primitive in the VM. So while it's certainly not *necessary* in order to get Any in Java's type system, it still seems to me like it's something that would make the JVM much more useful. I want to hear from someone compiling *dynamic* langs like JavaScript to the JVM what they think about it. > But what if we introduce Any as a 'nominal' type in the type-system, but > implemented as a reftype/ or boxed object with equals() and hashCode() > methods. That would give us the exact same performance of today for > unspecialized code, but a better type-system to describe ArrayList > descending from ArrayList. Right, that was the first of my implementation suggestions above. To recap: use exactly the implementation ("specialization") proposed today for concrete instantiations like ArrayList and ArrayList. But use boxing for the much less common instantiation ArrayList. > I am _totally_ happy with reftypes & boxing as our base case, only code > which has been specialized is expected to perform better, performance on > List is not at issue. Well the really interesting thing is that in principle you don't even necessarily need to allow a concrete instantiation ArrayList! You could prevent people from writing "new ArrayList()" if you prefer, and I for one would still be happy enough! If you did that, instantiations like ArrayList would only be *use-site* instantiations. This would still be enough fix the bothersome properties of the type system that I'm objecting to. I'm not advocating that, however. FTR, I don't see anything wrong with ArrayList, whether implemented using boxing or using a tagged union. -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From gavin at hibernate.org Mon Jan 5 01:27:37 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 02:27:37 +0100 Subject: more background to List specializing List In-Reply-To: References: <54A98928.8030402@oracle.com> Message-ID: Stephen you might be right but note that "beans" as they are used today actually are usually mutable and have identity. In UI binding frameworks, in ORMs, etc. As much as I appreciate little immutable objects, it's not at all clear to me that the Java ecosystem is really set up to turn on a dime and start using them everywhere. If we want a way to get a cheap equals()/hashCode(), we should add an annotation that generates equals()/hashCode(). Tying auto-generation of equals()/hashCode() to value semantics is to my mind making auto-generation much less useful. On Mon, Jan 5, 2015 at 2:18 AM, Stephen Colebourne wrote: > On 5 January 2015 at 00:44, Gavin King wrote: >> How strong of an objection this is depends upon how often you think >> value objects will occur compared to primitives. That's a question >> I've been turning over in my head the last couple of days, without >> being able to really arrive at a firm conclusion by intuition alone. > > Given that Java has no convenient bean/property support but value > types will likely provide an equals/hashCode/toString, I believe that > they will be widely used as domain objects, even for objects way > bigger than the size sweet-spot of value types. This opinion make me > more concerned about how they interoperate with libraries pre-JDK > 10/11. > > Stephen > PS. On List vs List I'm sure you're right... -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From brian.goetz at oracle.com Mon Jan 5 01:29:17 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 04 Jan 2015 20:29:17 -0500 Subject: more background to List specializing List In-Reply-To: References: <54A98928.8030402@oracle.com> <54A9CAA3.6070405@oracle.com> Message-ID: <54A9E8ED.3070403@oracle.com> > Thinking about the negatives, a lot relate to primitive types always > causing problems in code today (typically framework code). With 8 > special cases today, its managable. With an entire world of special > cases, its not. Of course the plan is to make them not special cases, > but what I'm seeing so far is making the bifurcation bigger not > smaller (worse not better). It is making it bigger, but also *more regular*. The problem today is not simply that primitives are different from references, but that you get exactly eight, no more, no less. Boxing is ad-hoc. No methods. No interfaces. No generics. I think the better way to look at it is not going from "eight special cases to infinitely many", but going from "one general case and eight special cases to two general cases." From brian.goetz at oracle.com Mon Jan 5 01:36:17 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 04 Jan 2015 20:36:17 -0500 Subject: Fwd: more background to List specializing List In-Reply-To: References: <54A98928.8030402@oracle.com> Message-ID: <54A9EA91.1010300@oracle.com> BTW, if anyone wants a hard problem to think about that we've not yet tackled, I encourage someone to explore how we're going to migrate existing j.u.Optional to be a value type -- without breaking existing source or binaries. (Hint: exploiting the L/Q duality gets you partway there.) On 1/4/2015 7:57 PM, Gavin King wrote: > Right, so this is the biggest objection to the idea of implementing > Any at the VM level: it would let you avoid boxing for the built-in > primitives, which is big win, obviously, but not for arbitrary value > types, which would still need boxing. > > How strong of an objection this is depends upon how often you think > value objects will occur compared to primitives. That's a question > I've been turning over in my head the last couple of days, without > being able to really arrive at a firm conclusion by intuition alone. > Looking at Java as it exists today, it might be that Optional is the > one thing that really tips the balance. But on the other hand, it's > hard to say what people are going to do with Java once value types > exist; perhaps value types will eventually become the default choice > for most user-written classes! > > The other thing floating around in the back of my mind is that even if > a VM-level Any isn't that useful for *Java*, it would still surely be > extremely useful for other JVM languages. I can imagine that folks > running dynamic langs on the JVM would just *love* to be able to use > an array of type Any[], and have that efficiently store numeric values > without boxing. The "specialization" approach doesn't seem like it > would help them as much. (This is speculation on my part; I've not > tried to compile dynamically-typed code for the JVM myself.) > > > On Mon, Jan 5, 2015 at 12:00 AM, Stephen Colebourne > wrote: >> The specialization document include the phrase "If we were concerned >> only about primitive instantiations, we could consider some sort of >> "tagged fixnum" approach. However, this approach essentially reverts >> to boxing for arbitrary value types, so it does not get us to our >> goal." > > -- > Gavin King > gavin at ceylon-lang.org > http://profiles.google.com/gavin.king > http://ceylon-lang.org > http://hibernate.org > http://seamframework.org > From gavin at hibernate.org Mon Jan 5 01:54:00 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 02:54:00 +0100 Subject: Fwd: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: <54A81467.8010207@univ-mlv.fr> References: <54A6D751.2080600@oracle.com> <54A81467.8010207@univ-mlv.fr> Message-ID: Sorry, Remi, I missed this email, it went straight to my spam folder or something for some reason. On Sat, Jan 3, 2015 at 5:10 PM, Remi Forax wrote: > On 01/02/2015 07:13 PM, Gavin King wrote: > The first major problem of Any is that it introduce a new primitive kind, > a lot of codes relies on the fact that you can only have boolean, byte, > short, > char, int, long, double and Object in Java. Introducing a new type will > make a lot of code to behave weirdly. You mean code in Java programs? I'm not seeing this. By nature it's impossible to switch over the primitive types with instanceof or whatever, so I'm having a really hard time visualizing what kind of Java code would break. Examples? > The second major problem is how to implement it. > It can be implemented as a compiler fiction only like in C# or in Scala, > but in that case apart that one can write ArrayList, it will > rely on a form of boxing. Well, that's not quite right. That the concrete instantiation ArrayList works using boxing doesn't say much about whether ArrayList does. You can still use specialization, AFAICT. > It can be implemented in the VM. It seems to be the best solution > but the engineering cost is insane and in fact Any will not solve > the specialization problem. > > Why the engineering cost is very high ? > Currently most of the Java VM implementations rely on the fact that > you know upfront (without executing the code) if a field or a local variable > is a reference or an object. In term of GC, there is a very clear > distinction > between the precise GC algorithms used in Java VM and the conservative > GC algorithms used by example in C. Technically, with Any, you can > implement something in the middle between these two categories of GC, > nevertheless, it changes the category of GCs, Java will be able to use. But you're introducing a new kind of field/variable here. Pre-existing code should keep behaving the same. It's only for new usages of this new thing that the new behavior is required. I understand that the VM would need to change to accommodate this. But I think it makes the VM more useful, and I strongly speculate that all the dynamic language folks would thank you for it. > Any will not solve the specialization problem. > Let say we now we have introduce Any, so we can create an array of Any. > But an array of Any can be at runtime by subtyping either > an array of doubles or an array of objects. > But by doing this we introduce a subtyping relationship between things that > are not structurally equivalent. A possible solution is to align all cells > of all arrays > to 64 bits which is a very bad property because even if you don't use Any, > you pay the cost of that you may use it. Another solution, the one used by > V8, > is to box all doubles (and long in Java) when you store them in an array*. > So an array of Any will not be magically an array of ints, an array of > doubles > and an array of objects without some trade-offs. Hrm, you're right, I had not considered that byte[] is an Any[], due to the totally broken "covariance" of arrays in Java. That sucks, and it's worse that it sucks because of the type system being broken :-( Couldn't we just say that a byte[] actually isn't an Any[]? I mean, that's actually *correct*. Why double down on stupid? -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From twhitmore.nz at gmail.com Mon Jan 5 02:41:16 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Mon, 5 Jan 2015 15:41:16 +1300 Subject: Any as a simple reftype base to the type-system Message-ID: Hi Gavin, people, Traditionally Object has effectively been "Any" in the Java type system -- with builtin boxing/ unboxing to use it for primitives. > Well the really interesting thing is that in principle you don't even > necessarily need to allow a concrete instantiation ArrayList! You > could prevent people from writing "new ArrayList()" if you > prefer, and I for one would still be happy enough! Well, to avoid member collisions & actually achieve specialization, ArrayList.class != ArrayListclass. But if 'Any' is effectively treated as a non-lockable reftype super to Object.. could we just redefine collections to be parameterized on and be done with it? - Any would implement (essentially same as) Object, without the locks. - Any[] would implement same as Object[]. - boxing/unboxing from primitives would be same. - equals(), hashCode() and toString() would be available. If we can do the above, I don't necessarily see problems instantiating ArrayList. There is a need for wide ecosystem compatibility, but the above principles seem to cover quite much. (These are also similar to the issues Stephen Colebourne raises on separating Lockable from Object, but taken from a different angle.) > If you did that, instantiations like ArrayList would > only be *use-site* instantiations. This would still be enough fix the > bothersome properties of the type system that I'm objecting to. Yes, I think the type system is very worthwhile to consider & this is one of the reasons I am interested in this kind of approach. Regards, Thomas Whitmore From brian.goetz at oracle.com Mon Jan 5 03:00:20 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 04 Jan 2015 22:00:20 -0500 Subject: more background to List specializing List In-Reply-To: References: <54A98928.8030402@oracle.com> <54A9CAA3.6070405@oracle.com> Message-ID: <54A9FE44.5070806@oracle.com> > - that there is too much concern on memory/performance and not enough > on developer productivity - Java isn't C We care a lot about developer productivity -- remember Java 8? But we also have to pay attention to performance too. You might not care (and that's great for you if you can avoid caring), but not all users have that luxury. And developers spent ridiculous energy working around boxing / objects. Value types are not an end unto themselves; they power features that offer productivity and performance both -- alternate numerics, tuples, faster iterators, etc. (It's too bad they weren't around for JSR-310, as you went to great effort to ensure that date-times behaved like values, but you still had to pay the boxing cost anyway.) Value types are also a huge boon for non-Java languages, who have to represent other-language primitives in Java, such as numerics that don't quite line up with Java's eight primitives. So, I think what you're saying is there isn't as much for *you* in this release as there was in other releases. And that's OK. We have a diverse base of users, and we have to serve them all. Value types are a foundational tool that will make a huge difference for a significant category of users. From twhitmore.nz at gmail.com Mon Jan 5 03:05:59 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Mon, 5 Jan 2015 16:05:59 +1300 Subject: Stephen's comments on Value Types Message-ID: Hi Stephen, Interested by the things you say. Very many of them are valid, but I also have a different angle on some of them? Consider an OSS library released for > JDK8 that has a method process(Object). The intent is that the method > accepts anything, we don't care what the type is. When used with > JDK10/11 it still accepts anything, except that instead of a small, > known number of types being boxed (for which the library might have a > process(int) overload, there are now *lots* of boxing operations. > Since "Values" are currently objects & must be processed as reftypes, we can say this is similar to the current situation. So I do not feel this is too bad. I guess one thing to look at, is that framework code should ideally not be repeatedly creating the boxes at many levels -- in such a situation, it might be slower. For most cases, I expect performance would likely be similar. Of course the situation with process(List) is worse again. Here I > want to accept any list, I don't care what type. In JDK 10/11 it now > won't even work. Callers won't be able to pass in a List. > (As opposed to IntList which can be passed in providing it extends > List, and which could be handled specially internally within > process(List) using instanceof if necessary). This is why Gavin and myself are considering List, with List to be a supertype of both List and List. If it could be achieved, this would give commonality. > Note that while the method process(List) could be fixed to work by > using , OSS > libraries don't work like that - they can't just have a new release > as they have to still support developers who can't upgrade. > The open wildcard is currently defined to mean '? extends Object', and could therefore be redefined to mean '? extends Any' without any source-code change. Compatibility would obviously still be an important issue, but some of the difficulties might be reduced. One question is whether 'Any' is even different (in the VM) from Object -- if it's fully assignable without casting, ArrayList can implicitly encompass Any and 'Object item' can still be valid. > TLDR, Object is *already* the Any type, and List is *already* the > List. Breaking that is a big problem for me. > I'm also interested in Any as a "compiler fiction" translating to Object in the VM, but specializable. I see the (very close) relationship between Object and Any as something that works in our favor! Some other thoughts and vague ideas as I've been mulling on this. > > Separating synchronization from Object seems like a Good Thing. > Possible approach: add interface Lockable with locking methods from > Object. Remove methods from Object using JDK 9 as a stepping stone via > deprecation. Use classloading/module linking tricks to add Lockable > interface to all classes that were compiled in JDK 8 or earlier. > I'm suggesting the same thing -- but by abstracting 'Any' without these, above. Any can then be ancestor to primitives/ value-types and is easily marked to specialize. 'Any' has to be present in bytecode I think, but in the VM could potentially be flattened to just implement as Object. This would avoid need to for check-casts casting from Any -> Object. > Fixing up the List.remove(int) vs List.remove(Object) problem seems > like a Good Thing. But its a more general problem: Why not just rename > the method? Use JDK 9 module linker step to bind the old method name > to the new method name. (Being able to rename methods and be backwards > compatible would be a major gain for all large Java systems). Same > renaming strategy applies to many of the problem cases under > discussion. > Interesting idea, not qualified to comment though. > Instead of value types, some form of "packed objects" seems like an > alternative that might have less issues. By packed objects I mean > where the state of the child object is embedded in the parent object. > (Child objects are not seen by gc, have no identity and are copied > where necessary, Null handled with bit flag where necessary. Variables > extended to be able to point to a child within a parent or an item in > an array. And yes, this is super-vague, and could be worse than value > types). > Just sounds like Value Types to me, but more hacky. I actually feel value types are good at this point, I'm not really pessimistic about performance, we just desire a good solution to Collections/ specialization and everything will work out fine. Regards, Thomas Whitmore From scolebourne at joda.org Mon Jan 5 08:23:58 2015 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 5 Jan 2015 08:23:58 +0000 Subject: more background to List specializing List In-Reply-To: <54A9FE44.5070806@oracle.com> References: <54A98928.8030402@oracle.com> <54A9CAA3.6070405@oracle.com> <54A9FE44.5070806@oracle.com> Message-ID: On 5 January 2015 at 03:00, Brian Goetz wrote: >> - that there is too much concern on memory/performance and not enough >> on developer productivity - Java isn't C > We care a lot about developer productivity -- remember Java 8? But we also > have to pay attention to performance too. You might not care (and that's > great for you if you can avoid caring), but not all users have that luxury. > And developers spent ridiculous energy working around boxing / objects. > > Value types are not an end unto themselves; they power features that offer > productivity and performance both -- alternate numerics, tuples, faster > iterators, etc. (It's too bad they weren't around for JSR-310, as you went > to great effort to ensure that date-times behaved like values, but you still > had to pay the boxing cost anyway.) > > Value types are also a huge boon for non-Java languages, who have to > represent other-language primitives in Java, such as numerics that don't > quite line up with Java's eight primitives. > > So, I think what you're saying is there isn't as much for *you* in this > release as there was in other releases. And that's OK. We have a diverse > base of users, and we have to serve them all. Value types are a > foundational tool that will make a huge difference for a significant > category of users. I actually like the potential benefits of value types. And I definitely care about performance to a degree. But if performance is key to a system it probably shouldn't be in Java, and value types aren't going to give anywhere near the amount of control necessary to perform the range of optimisations at the CPU level that true memory control can achieve. (That LocalDate is an object not a value is pain that will definitely cause me issues this year, so I do "get" the potential upsides here) But for every upside of value types there is also a downside. Its a trade off, and I'm not comfortable with the balance right now. Function types (in JDK 8) were dropped to make lambdas palatable, IMO something key needs to change in order for value types to be palatable. > I think the better way to look at it is not going from "eight special cases > to infinitely many", but going from "one general case and eight special > cases to two general cases." Right now we have a world 99% of objects. A single unified approach to coding. Sure there are 8 special cases, but that is very managable and all the tricks and issues around them are very well known. In most cases they cause no issues - complaints about them are more grumbles than real pain. In that world Object is the top type - sure it might not be in technical terms, but it is in practical terms. With value types, that Object is not the top type becomes in-your-face visible, something that IMO effectively changes the meaning of Object (in practical terms), and means code written today won't have the same semantic meaning wrt top type in JDK 10/11. Thus, in the world being outlined the entire coding world is bifurcated. Two general cases look to be worse than one general case plus 8 special cases. No piece of code can ignore value types because value types will be everywhere. As I've said before, what I want is "faster objects", not "user-defined primitives". Stephen From forax at univ-mlv.fr Mon Jan 5 11:06:08 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 05 Jan 2015 12:06:08 +0100 Subject: Fwd: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: References: <54A6D751.2080600@oracle.com> <54A81467.8010207@univ-mlv.fr> Message-ID: <54AA7020.6070401@univ-mlv.fr> On 01/05/2015 02:54 AM, Gavin King wrote: > Sorry, Remi, I missed this email, it went straight to my spam folder > or something for some reason. > > On Sat, Jan 3, 2015 at 5:10 PM, Remi Forax wrote: >> On 01/02/2015 07:13 PM, Gavin King wrote: >> The first major problem of Any is that it introduce a new primitive kind, >> a lot of codes relies on the fact that you can only have boolean, byte, >> short, >> char, int, long, double and Object in Java. Introducing a new type will >> make a lot of code to behave weirdly. > You mean code in Java programs? > > I'm not seeing this. By nature it's impossible to switch over the > primitive types with instanceof or whatever, so I'm having a really > hard time visualizing what kind of Java code would break. > > Examples? switch(type.getName()) { ... > >> The second major problem is how to implement it. >> It can be implemented as a compiler fiction only like in C# or in Scala, >> but in that case apart that one can write ArrayList, it will >> rely on a form of boxing. > Well, that's not quite right. That the concrete instantiation > ArrayList works using boxing doesn't say much about whether > ArrayList does. You can still use specialization, AFAICT. yes, but you don't need Any to do primitive type specialization, you just need Any as a bound of a type parameter that's the whole point of the discussion now ? > >> It can be implemented in the VM. It seems to be the best solution >> but the engineering cost is insane and in fact Any will not solve >> the specialization problem. >> >> Why the engineering cost is very high ? >> Currently most of the Java VM implementations rely on the fact that >> you know upfront (without executing the code) if a field or a local variable >> is a reference or an object. In term of GC, there is a very clear >> distinction >> between the precise GC algorithms used in Java VM and the conservative >> GC algorithms used by example in C. Technically, with Any, you can >> implement something in the middle between these two categories of GC, >> nevertheless, it changes the category of GCs, Java will be able to use. > But you're introducing a new kind of field/variable here. Pre-existing > code should keep behaving the same. It's only for new usages of this > new thing that the new behavior is required. > > I understand that the VM would need to change to accommodate this. But > I think it makes the VM more useful, Yes, it makes the VM more powerful, I think we can all agree with that. The problem is that its a pervasive change that as far as I know nobody has even tried. I will be very happy to see Redhat to found a project like this, taking the source of hotspot and trying to add Any. About the separation of behaviors, it's exactly like saying, if I use a JVM but only with primitive types, no objects, I will not pay the price of having to deal with objects. That's not true, the VM will still prepare internal data structures to be able to crawle the stack during a GC even if a GC will never occur in that case. > and I strongly speculate that all the dynamic language folks would thank you for it. :) Having all values stored as Any require the VM to check the runtime type of all operands for each operation, this is what CPython does, but not what more advanced dynamic language runtimes do. They do optimistic speculative type analysis, i.e. generate a code by examples for ints and de-optimise if a type is a String and then re-generate a new code. So sure Any will help, but not as much as you think. > >> Any will not solve the specialization problem. >> Let say we now we have introduce Any, so we can create an array of Any. >> But an array of Any can be at runtime by subtyping either >> an array of doubles or an array of objects. >> But by doing this we introduce a subtyping relationship between things that >> are not structurally equivalent. A possible solution is to align all cells >> of all arrays >> to 64 bits which is a very bad property because even if you don't use Any, >> you pay the cost of that you may use it. Another solution, the one used by >> V8, >> is to box all doubles (and long in Java) when you store them in an array*. >> So an array of Any will not be magically an array of ints, an array of >> doubles >> and an array of objects without some trade-offs. > Hrm, you're right, I had not considered that byte[] is an Any[], due > to the totally broken "covariance" of arrays in Java. > > That sucks, and it's worse that it sucks because of the type system > being broken :-( > > Couldn't we just say that a byte[] actually isn't an Any[]? I mean, > that's actually *correct*. Why double down on stupid? how to implement ArrayList in that case ? class ArrayList { T[] array; // <-- this is an array of Any ! } cheers, R?mi From forax at univ-mlv.fr Mon Jan 5 11:27:43 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 05 Jan 2015 12:27:43 +0100 Subject: Fwd: Remi's comments on Any In-Reply-To: References: Message-ID: <54AA752F.90001@univ-mlv.fr> On 01/05/2015 02:20 AM, Gavin King wrote: [...] >> I am _totally_ happy with reftypes & boxing as our base case, only code >> which has been specialized is expected to perform better, performance on >> List is not at issue. > Well the really interesting thing is that in principle you don't even > necessarily need to allow a concrete instantiation ArrayList! You > could prevent people from writing "new ArrayList()" if you > prefer, and I for one would still be happy enough! > > If you did that, instantiations like ArrayList would > only be *use-site* instantiations. This would still be enough fix the > bothersome properties of the type system that I'm objecting to. and the best way to avoid concrete instantiation of ArrayList, is not to use a *use-site* annotation like a wildcard but a *declaration site* like class ArrayList { ... } join us to the dark side ... > > I'm not advocating that, however. FTR, I don't see anything wrong with > ArrayList, whether implemented using boxing or using a tagged > union. > boxing make ArrayList is not too different from ArrayList, and tagged union will only happen in an alternate universe (never say never). R?mi From gavin at hibernate.org Mon Jan 5 11:35:03 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 12:35:03 +0100 Subject: Fwd: Remi's comments on Any In-Reply-To: <54AA752F.90001@univ-mlv.fr> References: <54AA752F.90001@univ-mlv.fr> Message-ID: On Mon, Jan 5, 2015 at 12:27 PM, Remi Forax wrote: > and the best way to avoid concrete instantiation of ArrayList, > is not to use a *use-site* annotation like a wildcard but a *declaration > site* > like > class ArrayList { ... } I don't see how that's right. Then how would I write down the instantiation ArrayList? When I'm saying we don't necessarily need to let you write: ArrayList list = new ArrayList(); I'm not saying we don't need to let you write: ArrayList list = new ArrayList(); I don't understand how you're proposing to write this second example. > boxing make ArrayList is not too different from ArrayList, > and tagged union will only happen in an alternate universe (never say > never). I still don't understand the resistance to the tagged union Any. It's surely way, way, way easier to implement than the value types proposal, and could be implemented at the same time. -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From forax at univ-mlv.fr Mon Jan 5 11:41:55 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 05 Jan 2015 12:41:55 +0100 Subject: more background to List specializing List In-Reply-To: References: <54A98928.8030402@oracle.com> <54A9CAA3.6070405@oracle.com> Message-ID: <54AA7883.90502@univ-mlv.fr> About List not being a subtype of List, You can note that in Trove, the specialized classes doesn't inherits from the interface of Collection API. You have to use decorators for that. Using the same idea (delegation is sometimes more powerful than direct implementation), the interface List can have a method boxed() that returns a view of the list as a List. At least the performance model (who do boxing, who don't) will be clear. R?mi On 01/05/2015 02:10 AM, Stephen Colebourne wrote: > On 4 January 2015 at 23:20, Brian Goetz wrote: >> Zooming back, though, I think what you're really saying is: the part where >> List is not a subtype of List is really the part you dislike the most, >> because it lives in this weird middle ground between "all instantiations of >> List have a common supertype" (Java 5 generics) and "no instantiations of >> list have a common supertype" (C++). You are probably OK with either all >> (what you've got now) or nothing (if you hadn't had ten years of Java >> generics training you on homogeneous translation), but what really freaks >> you out is this half-here, half-there, that forces the user to be aware that >> reference instantiations are erased and value instantiations are reified, >> with all that entails. >> >> Is that really your concern here? > My concerns are, *at this point*, somewhat larger than that. Right > now, my concern is that the combination of changes may well be making > Java as a whole worse, not better. Right now, I'm in the camp that I'd > rather reject adding value types because the negatives appear to > outweigh the positives. I do, of course expect this judgement to > change as things develop, but it is a big concern I've been mulling on > over the holiday period. > > The parts I like include > - JVM optimisations for types that don't require a synchronization lock > - greater locality of data, within the parent object > - the potential for arrays of value types > - maybe ClassDynamic might be useful, but its very hard to tell at this point > - distinguishing types that don't have identity and optimising based on that > > The parts I don't like include > - making old code incompatible with new code (see below) > - that List is not a subtype of List, List or List > - that Object changes its meaning (see below) > - that List changes its meaning (see below) > - that instanceof changes its meaning > - that Class changes its meaning > - that there is more "weirdness" not less > - that the thrust is to create more primitive types, not unify those we've got > - that there is too much concern on memory/performance and not enough > on developer productivity - Java isn't C > > Thinking about the negatives, a lot relate to primitive types always > causing problems in code today (typically framework code). With 8 > special cases today, its managable. With an entire world of special > cases, its not. Of course the plan is to make them not special cases, > but what I'm seeing so far is making the bifurcation bigger not > smaller (worse not better). > > I have a particular problem with the proposal where IMO it makes old > code incompatible with new code. Consider an OSS library released for > JDK8 that has a method process(Object). The intent is that the method > accepts anything, we don't care what the type is. When used with > JDK10/11 it still accepts anything, except that instead of a small, > known number of types being boxed (for which the library might have a > process(int) overload, there are now *lots* of boxing operations. > Technically, thats compatible, but in practical terms the boxing makes > it incompatible (performance/gc). > > Of course the situation with process(List) is worse again. Here I > want to accept any list, I don't care what type. In JDK 10/11 it now > won't even work. Callers won't be able to pass in a List. > (As opposed to IntList which can be passed in providing it extends > List, and which could be handled specially internally within > process(List) using instanceof if necessary). Note that while the > method process(List) could be fixed to work by using , OSS > libraries don't work like that - they can't just have a new release as > they have to still support developers who can't upgrade. ie. I fear > that value types will badly bifurcate the ecosystem. > > TLDR, Object is *already* the Any type, and List is *already* the > List. Breaking that is a big problem for me. > > Given how it looks that value types would actually work in the JVM, > thats why I'd rather have no value types than the ones currently on > offer. > > On your original question about List specifically, I've coded one > piece of source code so I have one class. 1:1. The type is unimportant > most of the time - its a compiler fiction. A List is a List is a List. > Thus its not the half-there half-not aspect that bothers, but losing > 1:1. > > > Some other thoughts and vague ideas as I've been mulling on this. > > Separating synchronization from Object seems like a Good Thing. > Possible approach: add interface Lockable with locking methods from > Object. Remove methods from Object using JDK 9 as a stepping stone via > deprecation. Use classloading/module linking tricks to add Lockable > interface to all classes that were compiled in JDK 8 or earlier. > > Fixing up the List.remove(int) vs List.remove(Object) problem seems > like a Good Thing. But its a more general problem: Why not just rename > the method? Use JDK 9 module linker step to bind the old method name > to the new method name. (Being able to rename methods and be backwards > compatible would be a major gain for all large Java systems). Same > renaming strategy applies to many of the problem cases under > discussion. > > Special casing the 8 primitive types in generics, with List > extending List. (Hence my original question). > > Instead of value types, some form of "packed objects" seems like an > alternative that might have less issues. By packed objects I mean > where the state of the child object is embedded in the parent object. > (Child objects are not seen by gc, have no identity and are copied > where necessary, Null handled with bit flag where necessary. Variables > extended to be able to point to a child within a parent or an item in > an array. And yes, this is super-vague, and could be worse than value > types). > > All of the above focussed on providing similar results to developers > without the seemingly evil value types. > > > I'm very aware of where you are in this whole process, so there is no > need to respond to each of my vague thoughts (and I'd encourage others > not to jump in to avoid wasting too much time here). I do trust that > things will improve. But I'm not currently optimistic that they will > improve enough. > > Stephen From gavin at hibernate.org Mon Jan 5 12:04:41 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 13:04:41 +0100 Subject: more background to List specializing List In-Reply-To: <54AA7883.90502@univ-mlv.fr> References: <54A98928.8030402@oracle.com> <54A9CAA3.6070405@oracle.com> <54AA7883.90502@univ-mlv.fr> Message-ID: On Mon, Jan 5, 2015 at 12:41 PM, Remi Forax wrote: > At least the performance model (who do boxing, who > don't) will be clear. OK, but the type system will still be all messed up :-) -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From ron at paralleluniverse.co Mon Jan 5 12:10:18 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Mon, 5 Jan 2015 14:10:18 +0200 Subject: Simplifying reified generics with partial specialization Message-ID: Hi. I've been following the discussion here, and I think there's a simple solution that will satisfy everyone: let us enjoy the performance gains of specialization, maintain full backwards compatibility, keep the generic class hierarchy simple and intuitive, and remove the need for layers. The cost would be relying on JVM optimizations for full performance (which is, I think, what Java has always done). The idea is simple: a generic instantiation over a value type would specialize fields (and array fields, obviously) and local variables, but not arguments and return values of non-private methods. These would use the value type's (or primitive's) boxed counterpart and will be erased (or not -- depending on how reference erasure will be done under Valhalla). This would mean that ArrayList will implement List (and will therefore extend List and List) and Object will essentially be the Any type (and List, if allowed, would just be an alias for List). It would also mean that we automatically get the performance benefits associated with the flattened arrays used to store the ArrayList's data. However, for full performance, it would rely on JVM employing escape analysis (or even a simpler optimization) to elide boxing. More advanced forms of specialization (such as those made possible by layers) would be possible, but not by using direct Java syntax, but by employing a library like ASM, but this is a topic for a separate discussion (Brian implied that classdynamic is intended to be trigger custom generation anyway). Ron From forax at univ-mlv.fr Mon Jan 5 13:50:06 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 05 Jan 2015 14:50:06 +0100 Subject: Fwd: Remi's comments on Any In-Reply-To: References: <54AA752F.90001@univ-mlv.fr> Message-ID: <54AA968E.6080100@univ-mlv.fr> On 01/05/2015 12:35 PM, Gavin King wrote: > On Mon, Jan 5, 2015 at 12:27 PM, Remi Forax wrote: > >> and the best way to avoid concrete instantiation of ArrayList, >> is not to use a *use-site* annotation like a wildcard but a *declaration >> site* >> like >> class ArrayList { ... } > I don't see how that's right. Then how would I write down the > instantiation ArrayList? > > When I'm saying we don't necessarily need to let you write: > > ArrayList list = new ArrayList(); > > I'm not saying we don't need to let you write: > > ArrayList list = new ArrayList(); > > I don't understand how you're proposing to write this second example. ArrayList list = new ArrayList().boxed(); > >> boxing make ArrayList is not too different from ArrayList, >> and tagged union will only happen in an alternate universe (never say >> never). > I still don't understand the resistance to the tagged union Any. It's > surely way, way, way easier to implement than the value types > proposal, and could be implemented at the same time. no, it's the opposite, value type can be implemented as a JIT only optimisation (if you forget data structure specialization) by example, the change is not as pervasive as Any. R?mi From gavin at hibernate.org Mon Jan 5 14:21:25 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 15:21:25 +0100 Subject: Fwd: Remi's comments on Any In-Reply-To: <54AA968E.6080100@univ-mlv.fr> References: <54AA752F.90001@univ-mlv.fr> <54AA968E.6080100@univ-mlv.fr> Message-ID: >> On Mon, Jan 5, 2015 at 12:27 PM, Remi Forax wrote: >> When I'm saying we don't necessarily need to let you write: >> >> ArrayList list = new ArrayList(); >> >> I'm not saying we don't need to let you write: >> >> ArrayList list = new ArrayList(); >> >> I don't understand how you're proposing to write this second example. > > > ArrayList list = new ArrayList().boxed(); Right, which means we're still making the generic type system of Java more weird instead of more regular. Generics are already much too complicated without more extra weird stuff. I understand that the folks who have to implement this stuff on the VM are naturally looking for the thing that is simplest to implement. But if that ease of implementation surfaces itself as extra complexity in the type system, I don't think it does anyone a favor. > no, it's the opposite, value type can be implemented as a JIT only > optimisation > (if you forget data structure specialization) by example, the change is not > as pervasive as Any. Well, OK, I'll have to take your word for it, but I still think it might be worth doing. All this specialization stuff in principle lets statically typed languages avoid boxing on the JVM. It doesn't help dynamically typed languages, AFAICS. -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From simon at ochsenreither.de Mon Jan 5 15:07:07 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Mon, 05 Jan 2015 16:07:07 +0100 Subject: Simplifying reified generics with partial specialization In-Reply-To: References: Message-ID: <54AAA89B.903@ochsenreither.de> > This would mean that ArrayList will implement List (and will > therefore extend List and List) and Object will essentially be the Any > type (and List, if allowed, would just be an alias for List). The issue is that you can do things with ArrayList that you can't do with ArrayList. What will happen when you do call "list.add(null)"? Will it take the array of ints, allocate a new array of Integers, box all ints to Integers and copy them over to the new reference array, then add null as the last element? (I think I already mentioned an example where such issues get even more pronounced in one of my earlier emails. E. g. what's supposed to happen if you add a String to your specialized covariant immutable data-structure of ints? If you suggest that if a generic parameter is instantiated with a value type the generic bounds are collapsed, then that's what Brian was saying in the beginning.) From ron at paralleluniverse.co Mon Jan 5 15:23:50 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Mon, 5 Jan 2015 17:23:50 +0200 Subject: Simplifying reified generics with partial specialization In-Reply-To: <54AAA89B.903@ochsenreither.de> References: <54AAA89B.903@ochsenreither.de> Message-ID: > The issue is that you can do things with ArrayList that you can't do with ArrayList. > What will happen when you do call "list.add(null)"? Why, you'd get an NPE, of course! While ArrayList would be a List, it will *not* be an ArrayList. And a List is certainly allowed (it even says so in the spec) to throw an NPE when a null element is added, or a ClassCastException when a wrong type is added. This would be the exact same behavior as trying to auto-unbox a null Integer into an int, or insert the wrong type into an array. Ron Pressler paralleluniverse.co @puniverseco on Twitter On Mon, Jan 5, 2015 at 5:07 PM, Simon Ochsenreither wrote: > This would mean that ArrayList will implement List (and will >> therefore extend List and List) and Object will essentially be the Any >> type (and List, if allowed, would just be an alias for >> List). >> > The issue is that you can do things with ArrayList that you can't > do with ArrayList. > What will happen when you do call "list.add(null)"? Will it take the array > of ints, allocate a new array of Integers, box all ints to Integers and > copy them over to the new reference array, then add null as the last > element? > > (I think I already mentioned an example where such issues get even more > pronounced in one of my earlier emails. E. g. what's supposed to happen if > you add a String to your specialized covariant immutable data-structure of > ints? If you suggest that if a generic parameter is instantiated with a > value type the generic bounds are collapsed, then that's what Brian was > saying in the beginning.) > From tomas.mikula at gmail.com Mon Jan 5 15:28:54 2015 From: tomas.mikula at gmail.com (Tomas Mikula) Date: Mon, 5 Jan 2015 16:28:54 +0100 Subject: Simplifying reified generics with partial specialization In-Reply-To: References: Message-ID: On Mon, Jan 5, 2015 at 1:10 PM, Ron Pressler wrote: > Hi. I've been following the discussion here, and I think there's a simple > solution that will satisfy everyone: [...] > > The idea is simple: a generic instantiation over a value type would > specialize fields (and array fields, obviously) and local variables, but > not arguments and return values of non-private methods. This does not let us enjoy a performance gain in higher-order operations, such as summing a Stream using Stream#reduce: Stream stream; stream.reduce(0, (a, b) -> a + b); The second argument to the reduce method has the type BinaryOperator, whose method T apply(T a, T b) is actualy Integer apply(Integer a, Integer b); as argument types and return types are not specialized as per your proposal. Thus, all ints in the specialized Stream still have to be boxed to be passed to the apply method, then unboxed inside the apply method for the + operator, whose result is again boxed as a return value of apply. Regards, Tomas From simon at ochsenreither.de Mon Jan 5 15:41:21 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Mon, 05 Jan 2015 16:41:21 +0100 Subject: Simplifying reified generics with partial specialization In-Reply-To: References: <54AAA89B.903@ochsenreither.de> Message-ID: <54AAB0A1.5030801@ochsenreither.de> > Why, you'd get an NPE, of course! While ArrayList would be a List, it will *not* be an ArrayList. That sounds incredibly asymmetric. What's the rule behind "it implements X, but not Y"? What about the second, more interesting case I mentioned? From brian.goetz at oracle.com Mon Jan 5 16:04:09 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 11:04:09 -0500 Subject: Fwd: Remi's comments on Any In-Reply-To: References: <54AA752F.90001@univ-mlv.fr> <54AA968E.6080100@univ-mlv.fr> Message-ID: <54AAB5F9.2080504@oracle.com> > I understand that the folks who have to implement this stuff on the VM > are naturally looking for the thing that is simplest to implement. But > if that ease of implementation surfaces itself as extra complexity in > the type system, I don't think it does anyone a favor. Welcome to the real world :( Working within the architecture of the JVM is not mere laziness. If the language-level abstractions we give people for modeling programs don't match up appropriately with the VM-level abstractions for representing those programs, this is not doing anyone a favor, because we are selling (harmful) fictions. As a simple example, suppose than in Java 5, we pushed autoboxing deeper, and let you write: List and really that was just sugar for List. This gives the illusion of a prettier type system, but comes at the cost of some nasty fall-off-a-cliff performance that is even more confusing that what we've got now. If all we cared about was simplicity of implementation, we'd probably already have done something (dumb) like this. I get that you've convinced yourself that its possible to just magically remove the complexity (just introduce Any!) and then you get the best of both worlds. But given that the JVM implementation experts have not responded with "gee, brilliant idea, thanks!", perhaps you should consider that this approach is not as straightforward as you think? From stef at epardaud.fr Mon Jan 5 16:32:12 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Mon, 05 Jan 2015 17:32:12 +0100 Subject: Fwd: Remi's comments on Any In-Reply-To: <54AAB5F9.2080504@oracle.com> References: <54AA752F.90001@univ-mlv.fr> <54AA968E.6080100@univ-mlv.fr> <54AAB5F9.2080504@oracle.com> Message-ID: <54AABC8C.10205@epardaud.fr> Hi, Can I ask where the current ideas/proposal for value types is located? I'm curious how it integrates with the object hierarchy and how you can differentiate one value type from another at the VM level. I think this is relevant to the current discussion, depending on how/if they're tagged and/or boxed. Thanks, cheers. From brian.goetz at oracle.com Mon Jan 5 16:50:08 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 11:50:08 -0500 Subject: Fwd: Remi's comments on Any In-Reply-To: <54AABC8C.10205@epardaud.fr> References: <54AA752F.90001@univ-mlv.fr> <54AA968E.6080100@univ-mlv.fr> <54AAB5F9.2080504@oracle.com> <54AABC8C.10205@epardaud.fr> Message-ID: <54AAC0C0.8090406@oracle.com> Current ideas are linked from the Valhalla project page. (Be aware that these will not capture all of the things going on in our heads at any time, so the writeups will be out of date / incomplete at times.) On 1/5/2015 11:32 AM, St?phane ?pardaud wrote: > Hi, > > Can I ask where the current ideas/proposal for value types is located? > I'm curious how it integrates with the object hierarchy and how you can > differentiate one value type from another at the VM level. I think this > is relevant to the current discussion, depending on how/if they're > tagged and/or boxed. > > Thanks, cheers. From brian.goetz at oracle.com Mon Jan 5 17:30:44 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 12:30:44 -0500 Subject: Simplifying reified generics with partial specialization In-Reply-To: References: Message-ID: <54AACA44.3040808@oracle.com> A reasonable idea (and, probably not surprising, we've investigated big parts of this too). Some people have speculated "why can't values just be 'fast objects'", to which the answer is that they can be for purposes of passing data between methods, but cannot for purposes of storing data in the heap. Your proposal, if I read it correctly, splits these issues, using elidable boxing where we can (a la Albert Noll's work on heisenboxing, which allows us to put boxed types in APIs and optimize away the boxing cost for the data-passing part of the problem (but not the heap-storage part of the problem, see [1] and related thread) to handle the data movement part, and calls for some upgraded VM treatment of fields to support some notion of "field overriding" (no small request, but not as insane as some other suggestions we've received.) Note that the first part is really just an implementation choice; what it does is reduces the invasiveness of specialization, by limiting how much code has to be specialized, pushing the work to the VM to optimize away box/unbox conversions; it's mostly just turning one of the knobs. [1] http://mail.openjdk.java.net/pipermail/valhalla-dev/2014-November/000380.html On 1/5/2015 7:10 AM, Ron Pressler wrote: > Hi. I've been following the discussion here, and I think there's a simple > solution that will satisfy everyone: let us enjoy the performance gains of > specialization, maintain full backwards compatibility, keep the generic > class hierarchy simple and intuitive, and remove the need for layers. The > cost would be relying on JVM optimizations for full performance (which is, > I think, what Java has always done). > > The idea is simple: a generic instantiation over a value type would > specialize fields (and array fields, obviously) and local variables, but > not arguments and return values of non-private methods. These would use the > value type's (or primitive's) boxed counterpart and will be erased (or not > -- depending on how reference erasure will be done under Valhalla). > > This would mean that ArrayList will implement List (and will > therefore extend List and List) and Object will essentially be the Any > type (and List, if allowed, would just be an alias for List). > It would also mean that we automatically get the performance benefits > associated with the flattened arrays used to store the ArrayList's data. > However, for full performance, it would rely on JVM employing escape > analysis (or even a simpler optimization) to elide boxing. > > More advanced forms of specialization (such as those made possible by > layers) would be possible, but not by using direct Java syntax, but by > employing a library like ASM, but this is a topic for a separate discussion > (Brian implied that classdynamic is intended to be trigger custom > generation anyway). > > Ron > From ron at paralleluniverse.co Mon Jan 5 17:34:10 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Mon, 5 Jan 2015 19:34:10 +0200 Subject: Simplifying reified generics with partial specialization In-Reply-To: <54AACA44.3040808@oracle.com> References: <54AACA44.3040808@oracle.com> Message-ID: Exactly. The storage will still be specialized, but the API will be boxed (hence, partial specialization) and the boxing elided where possible. I'm writing a response on how to deal with high-order functions which might help the JVM with the possibly difficult (is it?) task of eliding boxing in a deep "boxed API" stack. Ron Pressler paralleluniverse.co @puniverseco on Twitter On Mon, Jan 5, 2015 at 7:30 PM, Brian Goetz wrote: > A reasonable idea (and, probably not surprising, we've investigated big > parts of this too). Some people have speculated "why can't values just be > 'fast objects'", to which the answer is that they can be for purposes of > passing data between methods, but cannot for purposes of storing data in > the heap. > > Your proposal, if I read it correctly, splits these issues, using elidable > boxing where we can (a la Albert Noll's work on heisenboxing, which allows > us to put boxed types in APIs and optimize away the boxing cost for the > data-passing part of the problem (but not the heap-storage part of the > problem, see [1] and related thread) to handle the data movement part, and > calls for some upgraded VM treatment of fields to support some notion of > "field overriding" (no small request, but not as insane as some other > suggestions we've received.) > > Note that the first part is really just an implementation choice; what it > does is reduces the invasiveness of specialization, by limiting how much > code has to be specialized, pushing the work to the VM to optimize away > box/unbox conversions; it's mostly just turning one of the knobs. > > [1] http://mail.openjdk.java.net/pipermail/valhalla-dev/2014- > November/000380.html > > > On 1/5/2015 7:10 AM, Ron Pressler wrote: > >> Hi. I've been following the discussion here, and I think there's a simple >> solution that will satisfy everyone: let us enjoy the performance gains of >> specialization, maintain full backwards compatibility, keep the generic >> class hierarchy simple and intuitive, and remove the need for layers. The >> cost would be relying on JVM optimizations for full performance (which is, >> I think, what Java has always done). >> >> The idea is simple: a generic instantiation over a value type would >> specialize fields (and array fields, obviously) and local variables, but >> not arguments and return values of non-private methods. These would use >> the >> value type's (or primitive's) boxed counterpart and will be erased (or not >> -- depending on how reference erasure will be done under Valhalla). >> >> This would mean that ArrayList will implement List (and will >> therefore extend List and List) and Object will essentially be the Any >> type (and List, if allowed, would just be an alias for >> List). >> It would also mean that we automatically get the performance benefits >> associated with the flattened arrays used to store the ArrayList's data. >> However, for full performance, it would rely on JVM employing escape >> analysis (or even a simpler optimization) to elide boxing. >> >> More advanced forms of specialization (such as those made possible by >> layers) would be possible, but not by using direct Java syntax, but by >> employing a library like ASM, but this is a topic for a separate >> discussion >> (Brian implied that classdynamic is intended to be trigger custom >> generation anyway). >> >> Ron >> >> From ron at paralleluniverse.co Mon Jan 5 17:37:06 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Mon, 5 Jan 2015 19:37:06 +0200 Subject: Simplifying reified generics with partial specialization In-Reply-To: <54AAB0A1.5030801@ochsenreither.de> References: <54AAA89B.903@ochsenreither.de> <54AAB0A1.5030801@ochsenreither.de> Message-ID: A specialized class does not extend its "template" (i.e. ArrayList doesn't extend ArrayList) but still implement its interfaces. As to your other example, you'd get a ClassCastException/ArrayStoreException because the storage is still specialized. This behavior is specifically allowed by the List interface contract. Ron Pressler paralleluniverse.co @puniverseco on Twitter On Mon, Jan 5, 2015 at 5:41 PM, Simon Ochsenreither wrote: > > Why, you'd get an NPE, of course! While ArrayList would be a > List, it will *not* be an ArrayList. > > That sounds incredibly asymmetric. What's the rule behind "it implements > X, but not Y"? > > > What about the second, more interesting case I mentioned? > From ron at paralleluniverse.co Mon Jan 5 17:45:35 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Mon, 5 Jan 2015 19:45:35 +0200 Subject: Simplifying reified generics with partial specialization In-Reply-To: References: Message-ID: To handle high-order functions, you can employ one of two solutions: either rely on JVM optimization to elide boxing, or, if that's too hard for the JVM, you can make use of the imperfect but currently acceptable solution of bridge methods. In the latter case, IntBinaryOperator would extend BinaryOperator and the lambda expression would generate a bridge method for the boxed case. The specialized ArrayList (and its resulting stream) would call the applyAsInt method thus eliding the boxing at compile time rather than relying on the JVM to optimize it at runtime. All of this specialization is hidden in the specialized implementation without changing the public API. I realize I'm probably glossing over some details, but this generation of bridge methods and specialized call to the specialized method would still be easier -- both to implement and to use -- than layers. Of course, there might be some missing pieces here which would make this approach unworkable, but it requires more careful consideration. Ron Pressler paralleluniverse.co @puniverseco on Twitter On Mon, Jan 5, 2015 at 5:28 PM, Tomas Mikula wrote: > On Mon, Jan 5, 2015 at 1:10 PM, Ron Pressler > wrote: > > Hi. I've been following the discussion here, and I think there's a simple > > solution that will satisfy everyone: [...] > > > > The idea is simple: a generic instantiation over a value type would > > specialize fields (and array fields, obviously) and local variables, but > > not arguments and return values of non-private methods. > > This does not let us enjoy a performance gain in higher-order > operations, such as summing a Stream using Stream#reduce: > > Stream stream; > stream.reduce(0, (a, b) -> a + b); > > The second argument to the reduce method has the type > BinaryOperator, whose method > > T apply(T a, T b) > > is actualy > > Integer apply(Integer a, Integer b); > > as argument types and return types are not specialized as per your > proposal. Thus, all ints in the specialized Stream still have to be > boxed to be passed to the apply method, then unboxed inside the apply > method for the + operator, whose result is again boxed as a return > value of apply. > > Regards, > Tomas > From simon at ochsenreither.de Mon Jan 5 17:47:32 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Mon, 05 Jan 2015 18:47:32 +0100 Subject: Simplifying reified generics with partial specialization In-Reply-To: References: <54AAA89B.903@ochsenreither.de> <54AAB0A1.5030801@ochsenreither.de> Message-ID: <54AACE34.3050703@ochsenreither.de> > As to your other example, you'd get a > ClassCastException/ArrayStoreException because the storage is still > specialized. This behavior is specifically allowed by the List > interface contract. a) That contract comes from pre-Generics ages. b) One of the goals of Generics was to move errors from runtime to compile time. This would reverse the success of it (success in the sense of "I haven't seen a ClassCastException coming from the collection API for a long time"). I think it's fair to say that pretty much no codebase in the wild today would be prepared to handle such CCEs. c) What about all the APIs out there, which are defined more strictly? From martin.odersky at epfl.ch Mon Jan 5 17:48:53 2015 From: martin.odersky at epfl.ch (martin odersky) Date: Mon, 5 Jan 2015 18:48:53 +0100 Subject: Fwd: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: <54AA7020.6070401@univ-mlv.fr> References: <54A6D751.2080600@oracle.com> <54A81467.8010207@univ-mlv.fr> <54AA7020.6070401@univ-mlv.fr> Message-ID: On Mon, Jan 5, 2015 at 12:06 PM, Remi Forax wrote: > > On 01/05/2015 02:54 AM, Gavin King wrote: >> >> Sorry, Remi, I missed this email, it went straight to my spam folder >> or something for some reason. >> >> On Sat, Jan 3, 2015 at 5:10 PM, Remi Forax wrote: >>> >>> On 01/02/2015 07:13 PM, Gavin King wrote: >>> The first major problem of Any is that it introduce a new primitive kind, >>> a lot of codes relies on the fact that you can only have boolean, byte, >>> short, >>> char, int, long, double and Object in Java. Introducing a new type will >>> make a lot of code to behave weirdly. >> >> You mean code in Java programs? >> >> I'm not seeing this. By nature it's impossible to switch over the >> primitive types with instanceof or whatever, so I'm having a really >> hard time visualizing what kind of Java code would break. >> >> Examples? > > > switch(type.getName()) { ... > >> >>> The second major problem is how to implement it. >>> It can be implemented as a compiler fiction only like in C# or in Scala, >>> but in that case apart that one can write ArrayList, it will >>> rely on a form of boxing. >> >> Well, that's not quite right. That the concrete instantiation >> ArrayList works using boxing doesn't say much about whether >> ArrayList does. You can still use specialization, AFAICT. > > > yes, but you don't need Any to do primitive type specialization, > you just need Any as a bound of a type parameter that's > the whole point of the discussion now ? > >> >>> It can be implemented in the VM. It seems to be the best solution >>> but the engineering cost is insane and in fact Any will not solve >>> the specialization problem. >>> >>> Why the engineering cost is very high ? >>> Currently most of the Java VM implementations rely on the fact that >>> you know upfront (without executing the code) if a field or a local >>> variable >>> is a reference or an object. In term of GC, there is a very clear >>> distinction >>> between the precise GC algorithms used in Java VM and the conservative >>> GC algorithms used by example in C. Technically, with Any, you can >>> implement something in the middle between these two categories of GC, >>> nevertheless, it changes the category of GCs, Java will be able to use. >> >> But you're introducing a new kind of field/variable here. Pre-existing >> code should keep behaving the same. It's only for new usages of this >> new thing that the new behavior is required. >> >> I understand that the VM would need to change to accommodate this. But >> I think it makes the VM more useful, > > > Yes, it makes the VM more powerful, I think we can all agree with that. > The problem is that its a pervasive change that as far as I know > nobody has even tried. > > I will be very happy to see Redhat to found a project like this, > taking the source of hotspot and trying to add Any. > > About the separation of behaviors, it's exactly like saying, > if I use a JVM but only with primitive types, no objects, > I will not pay the price of having to deal with objects. > That's not true, the VM will still prepare internal data structures > to be able to crawle the stack during a GC even if a GC > will never occur in that case. > >> and I strongly speculate that all the dynamic language folks would thank >> you for it. > > > :) > > Having all values stored as Any require the VM to check the runtime type of > all operands > for each operation, this is what CPython does, but not what more advanced > dynamic language runtimes do. > They do optimistic speculative type analysis, i.e. generate a code by > examples for ints > and de-optimise if a type is a String and then re-generate a new code. > > So sure Any will help, but not as much as you think. > >> >>> Any will not solve the specialization problem. >>> Let say we now we have introduce Any, so we can create an array of Any. >>> But an array of Any can be at runtime by subtyping either >>> an array of doubles or an array of objects. >>> But by doing this we introduce a subtyping relationship between things >>> that >>> are not structurally equivalent. A possible solution is to align all >>> cells >>> of all arrays >>> to 64 bits which is a very bad property because even if you don't use >>> Any, >>> you pay the cost of that you may use it. Another solution, the one used >>> by >>> V8, >>> is to box all doubles (and long in Java) when you store them in an >>> array*. >>> So an array of Any will not be magically an array of ints, an array of >>> doubles >>> and an array of objects without some trade-offs. >> >> Hrm, you're right, I had not considered that byte[] is an Any[], due >> to the totally broken "covariance" of arrays in Java. >> >> That sucks, and it's worse that it sucks because of the type system >> being broken :-( >> I think Any[] needs needs to erase either to Object, or to a new type such as "AnyArray". You need to use reflection to access the elements of of an Any[] by looking at the array element type tag. In Scala, we do not have exactly the same problem as Array is invariant, but an analogous problem arises for []T, where T is an unbounded type variable. We erase []T to Object. It does not run super-fast now (performance hit of about a factor of 10-20 compared to native array access), but it looks easy to optimize if the JVM is aware of it. Cheers - Martin >> Couldn't we just say that a byte[] actually isn't an Any[]? I mean, >> that's actually *correct*. Why double down on stupid? > > > how to implement ArrayList in that case ? > > class ArrayList { > T[] array; // <-- this is an array of Any ! > } > > cheers, > R?mi > > -- Martin Odersky EPFL From ron at paralleluniverse.co Mon Jan 5 18:08:36 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Mon, 5 Jan 2015 20:08:36 +0200 Subject: Simplifying reified generics with partial specialization In-Reply-To: <54AACE34.3050703@ochsenreither.de> References: <54AAA89B.903@ochsenreither.de> <54AAB0A1.5030801@ochsenreither.de> <54AACE34.3050703@ochsenreither.de> Message-ID: You would still get a compile-time error if you try to add a String to an ArrayList, just as you'd get one for ArrayList today. As for runtime behavior -- there's nothing surprising here. ArrayList wouldn't change behavior because ArrayList doesn't extend ArrayList, and List might well throw a CCE at runtime even today -- after all, that's what you'd get when using Collections.checkedList today. Any code that takes a List today might get a checked list and get a CCE if it tries to do something crazy with it. Code that for whatever reason takes ArrayList simply won't be able to take an ArrayList (just as it can't take a specialized Trove class today), but that's what you'd also get with Brian's proposal. Ron Pressler paralleluniverse.co @puniverseco on Twitter On Mon, Jan 5, 2015 at 7:47 PM, Simon Ochsenreither wrote: > As to your other example, you'd get a ClassCastException/ArrayStoreException >> because the storage is still specialized. This behavior is specifically >> allowed by the List interface contract. >> > > a) That contract comes from pre-Generics ages. > b) One of the goals of Generics was to move errors from runtime to compile > time. This would reverse the success of it (success in the sense of "I > haven't seen a ClassCastException coming from the collection API for a long > time"). I think it's fair to say that pretty much no codebase in the wild > today would be prepared to handle such CCEs. > c) What about all the APIs out there, which are defined more strictly? > From ron at paralleluniverse.co Mon Jan 5 18:11:18 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Mon, 5 Jan 2015 20:11:18 +0200 Subject: Simplifying reified generics with partial specialization In-Reply-To: <54AACA44.3040808@oracle.com> References: <54AACA44.3040808@oracle.com> Message-ID: Sorry, I missed a part of your post: > and calls for some upgraded VM treatment of fields to support some notion of "field overriding" Any hidden elements -- fields and local variables -- would simply be specialized, just as in your proposal. In short: hidden behavior is specialized, public API is boxed (and hopefully optimized). Ron Pressler paralleluniverse.co @puniverseco on Twitter On Mon, Jan 5, 2015 at 7:30 PM, Brian Goetz wrote: > A reasonable idea (and, probably not surprising, we've investigated big > parts of this too). Some people have speculated "why can't values just be > 'fast objects'", to which the answer is that they can be for purposes of > passing data between methods, but cannot for purposes of storing data in > the heap. > > Your proposal, if I read it correctly, splits these issues, using elidable > boxing where we can (a la Albert Noll's work on heisenboxing, which allows > us to put boxed types in APIs and optimize away the boxing cost for the > data-passing part of the problem (but not the heap-storage part of the > problem, see [1] and related thread) to handle the data movement part, and > calls for some upgraded VM treatment of fields to support some notion of > "field overriding" (no small request, but not as insane as some other > suggestions we've received.) > > Note that the first part is really just an implementation choice; what it > does is reduces the invasiveness of specialization, by limiting how much > code has to be specialized, pushing the work to the VM to optimize away > box/unbox conversions; it's mostly just turning one of the knobs. > > [1] http://mail.openjdk.java.net/pipermail/valhalla-dev/2014- > November/000380.html > > > On 1/5/2015 7:10 AM, Ron Pressler wrote: > >> Hi. I've been following the discussion here, and I think there's a simple >> solution that will satisfy everyone: let us enjoy the performance gains of >> specialization, maintain full backwards compatibility, keep the generic >> class hierarchy simple and intuitive, and remove the need for layers. The >> cost would be relying on JVM optimizations for full performance (which is, >> I think, what Java has always done). >> >> The idea is simple: a generic instantiation over a value type would >> specialize fields (and array fields, obviously) and local variables, but >> not arguments and return values of non-private methods. These would use >> the >> value type's (or primitive's) boxed counterpart and will be erased (or not >> -- depending on how reference erasure will be done under Valhalla). >> >> This would mean that ArrayList will implement List (and will >> therefore extend List and List) and Object will essentially be the Any >> type (and List, if allowed, would just be an alias for >> List). >> It would also mean that we automatically get the performance benefits >> associated with the flattened arrays used to store the ArrayList's data. >> However, for full performance, it would rely on JVM employing escape >> analysis (or even a simpler optimization) to elide boxing. >> >> More advanced forms of specialization (such as those made possible by >> layers) would be possible, but not by using direct Java syntax, but by >> employing a library like ASM, but this is a topic for a separate >> discussion >> (Brian implied that classdynamic is intended to be trigger custom >> generation anyway). >> >> Ron >> >> From brian.goetz at oracle.com Mon Jan 5 19:35:01 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 14:35:01 -0500 Subject: Can we also get some feedback on specialization, please? Message-ID: <54AAE765.6040402@oracle.com> There's been a lot of passionate discussion here about the directions we should or should not take Java. And this is all understandable, and perhaps inevitable, but ... it seems to be crowding out what is actually the primary purpose of this (-dev) list. We had hoped, in taking the effort to write up State of the Specialization, to get some comments on ... specialization :) So far, nearly all the comments have been on the speculative parts of the writeup (layers/conditional methods), as well as the usual (supersized) portion of "I think you should do X/Y/Z instead". Which is all OK, but we were actually hoping to get some feedback on the part that is written up in some detail -- and mostly implemented! I know it's fun to discuss the big sexy stuff, but we can't let this crowd out the main goal. So my question is: has anyone tried to write a program with specialized generics with the valhalla implementation? Well, we'd really appreciate it. I know people want to contribute. At least right now, here's what contribution looks like from our perspective: 1.) Please try it out! Write some code! 2.) Tell us what works and what doesn't! 3.) Nice comments are always a bonus! (And are welcome even if you didn't do (1) and (2)). 4.) If you can't do (1) and (2), and still have something important to say in the big-picture department, please do it a) nicely, b) constructively, c) succinctly. Thanks, -Brian From gavin at hibernate.org Mon Jan 5 19:46:39 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 20:46:39 +0100 Subject: Fwd: Remi's comments on Any In-Reply-To: <54AAB5F9.2080504@oracle.com> References: <54AA752F.90001@univ-mlv.fr> <54AA968E.6080100@univ-mlv.fr> <54AAB5F9.2080504@oracle.com> Message-ID: On Mon, Jan 5, 2015 at 5:04 PM, Brian Goetz wrote: > As a simple example, suppose than in Java 5, we pushed autoboxing deeper, > and let you write: > > List > > and really that was just sugar for List. This gives the illusion > of a prettier type system, Well I guess I don't favor saying that List means List, since a List can contain null elements. Yes I know that Java's type system doesn't provide typesafety around null, but I still think it would be unfortunate if "int" sometimes meant "can be null". OTOH, it might be reasonable to make List be a subtype of List, interpreting Integer to mean, essentially, int|null. > I get that you've convinced yourself that its possible to just magically > remove the complexity (just introduce Any!) and then you get the best of > both worlds. No, I certainly have not convinced myself of that, as you can see from what I've written in my emails above. It's one potential path, and if it doesn't work, then I'd like to understand why it doesn't work. > But given that the JVM implementation experts have not > responded with "gee, brilliant idea, thanks!", perhaps you should consider > that this approach is not as straightforward as you think? One of the interesting things about our field is that we don't have to just take the assertions of experts, and believe them to be true, without looking for the reasoning behind the assertions. Knowledge in computing isn't of some esoteric nature, inaccessible to mere ordinary men. This is just like in other STEM fields: if an expert mathematician asserts a conjecture, without providing a proof of that conjecture, we don't call it a theorem. Likewise, if a great scientist claims to have experimental results that support a theory, we aren't obliged to accept that theory unless the scientist provides the information we need to reproduce an experiment and replicate its results. So when an expert makes an assertion, and isn't prepared to explain the reasoning behind it, it appears as if the expert isn't very confident in the soundness of their reasoning, and that immediately leads one suspect that the reasoning might in fact not be sound. It's not a waste of time, in such cases, to attempt to understand and reproduce the reasoning, because even experts make mistakes from time to time. Therefore "just trust us, we know what we're doing" is pretty much an invitation to *distrust* the assertion. -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From gavin at hibernate.org Mon Jan 5 19:52:34 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 20:52:34 +0100 Subject: Simplifying reified generics with partial specialization In-Reply-To: References: Message-ID: On Mon, Jan 5, 2015 at 1:10 PM, Ron Pressler wrote: > This would mean that ArrayList will implement List (and will > therefore extend List and List) Ron I think this would be a bad idea. Yes, I know that Java unfortunately doesn't provide a typesafe null value, but I still think that it would be a bad idea to have occurrences of "int" which actually mean int|null. I think there's a really strong precedent in the language that ints can't be null. OTOH, perhaps it would be viable to make ArrayList a subtype of ArrayList. From martijnverburg at gmail.com Mon Jan 5 19:58:52 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 5 Jan 2015 19:58:52 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54AAE765.6040402@oracle.com> References: <54AAE765.6040402@oracle.com> Message-ID: Hi Brian/Valhalla members, Thanks for this timely post! I've been lurking on this list for some time wondering how best we (Adoption Group / Java User Groups) can assist. The best course so far has been to do nothing as our membership typically don't have the expertise to contribute at this early stage, except for cheer leading. On that note, the community at large is very excited (the volume of well meaning blog posts and conf talks etc attest to this) about seeing Value Type/Packed Objects/Whatever the end result will be - so thanks to everyone here for tackling this beast, we kinda thought at one stage it would never get looked at! Sounds like there might be a couple of things we could do now? So here's my suggestions (please push back if this is not helpful / too early): 1.) We have an OpenJDK hack day on the 17th of Jan. I could write up a quick "How to build Valhalla and write some code against it" guide and send it here for review/upload to the project site. We could have some of the more advanced members of the LJC in give specialisation a go (assuming they understand it, but IIRC some of them lurk on this list and are quite capable). Typically for speed on the day we have a small GitHub based project/tutorial to collate code samples that people try (as we did for Lambdas and Date & Time), maybe that could serve as a point for other more advanced members in the community to try their hands at. 2.) Provide more resources on the Valhalla project page and a guide on how to contribute at levels X, Y and Z which would roughly correspond to (X) Full time - JVM internal expert/implementers, (Y) Advanced Java/JVM users (e.g. Popular framework/lib authors and such like) and (Z) Joe/Jane Java Application Devs who are excited to try things as early as is deemed feasible. Let me know if either of those two are helpful and if the Valhalla project wants us to do anything else (e.g. We can also work on promotion and dissemination of the hopefully *correct* information). Cheers, Martijn On 5 January 2015 at 19:35, Brian Goetz wrote: > There's been a lot of passionate discussion here about the directions we > should or should not take Java. And this is all understandable, and perhaps > inevitable, but ... it seems to be crowding out what is actually the > primary purpose of this (-dev) list. We had hoped, in taking the effort to > write up State of the Specialization, to get some comments on ... > specialization :) > > So far, nearly all the comments have been on the speculative parts of the > writeup (layers/conditional methods), as well as the usual (supersized) > portion of "I think you should do X/Y/Z instead". Which is all OK, but we > were actually hoping to get some feedback on the part that is written up in > some detail -- and mostly implemented! I know it's fun to discuss the big > sexy stuff, but we can't let this crowd out the main goal. > > So my question is: has anyone tried to write a program with specialized > generics with the valhalla implementation? Well, we'd really appreciate > it. > > I know people want to contribute. At least right now, here's what > contribution looks like from our perspective: > > 1.) Please try it out! Write some code! > 2.) Tell us what works and what doesn't! > 3.) Nice comments are always a bonus! (And are welcome even if you > didn't do (1) and (2)). > 4.) If you can't do (1) and (2), and still have something important to > say in the big-picture department, please do it a) nicely, b) > constructively, c) succinctly. > > Thanks, > -Brian > > > From ron at paralleluniverse.co Mon Jan 5 20:00:41 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Mon, 05 Jan 2015 12:00:41 -0800 (PST) Subject: Simplifying reified generics with partial specialization In-Reply-To: References: Message-ID: > it would be a bad idea to have occurrences of ?int" which actually mean int|null. They don?t. ArrayList is specialized and cannot hold nulls, and while it would implement List, it throws an NPE when trying to store a null, just like when trying to auto-unbox a null Integer. A List is perfectly allowed to behave this way. But I suggest that, per Brian?s request, we continue this discussion elsewhere. This isn?t the place for protracted discussions. > On Jan 5, 2015, at 9:52 PM, Gavin King wrote: > > > On Mon, Jan 5, 2015 at 1:10 PM, Ron Pressler wrote: > > >> This would mean that ArrayList will implement List (and will >> therefore extend List and List) >> >> >> > > > Ron I think this would be a bad idea. Yes, I know that Java > unfortunately doesn't provide a typesafe null value, but I still think > that it would be a bad idea to have occurrences of "int" which > actually mean int|null. I think there's a really strong precedent in > the language that ints can't be null. > > > OTOH, perhaps it would be viable to make ArrayList a subtype of > ArrayList. > > From gavin at hibernate.org Mon Jan 5 20:01:14 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 21:01:14 +0100 Subject: Fwd: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: <54A81467.8010207@univ-mlv.fr> References: <54A6D751.2080600@oracle.com> <54A81467.8010207@univ-mlv.fr> Message-ID: On Sat, Jan 3, 2015 at 5:10 PM, Remi Forax wrote: > Why the engineering cost is very high ? > Currently most of the Java VM implementations rely on the fact that > you know upfront (without executing the code) if a field or a local variable > is a reference or an object. In term of GC, there is a very clear > distinction > between the precise GC algorithms used in Java VM and the conservative > GC algorithms used by example in C. Technically, with Any, you can > implement something in the middle between these two categories of GC, > nevertheless, it changes the category of GCs, Java will be able to use. Remi the more I think about this objection the less sense it makes to me. Currently, a field of type java.lang.Object may or may not reference an instance (it might be null). This is exactly the same as with an Any field: it may or may not reference an instance, depending upon whether it is null or primitive. So from the point of view of GC, precisely what is the difference? Any seems to me the same as java.lang.Object for this purpose. -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From martijnverburg at gmail.com Mon Jan 5 20:09:19 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 5 Jan 2015 20:09:19 +0000 Subject: Simplifying reified generics with partial specialization In-Reply-To: References: Message-ID: On this side note - there is/was a great mailing list for these sorts of discussions that I'd enjoyed lurking on ( https://groups.google.com/forum/#!forum/jvm-languages) - perhaps we could resurrect that list somewhat and have some of those discussions there? Cheers, Martijn On 5 January 2015 at 20:00, Ron Pressler wrote: > > it would be a bad idea to have occurrences of ?int" which actually > mean int|null. > > > They don?t. ArrayList is specialized and cannot hold nulls, and while > it would implement List, it throws an NPE when trying to store a > null, just like when trying to auto-unbox a null Integer. A List > is perfectly allowed to behave this way. > > But I suggest that, per Brian?s request, we continue this discussion > elsewhere. This isn?t the place for protracted discussions. > > > > > > > > > On Jan 5, 2015, at 9:52 PM, Gavin King wrote: > > > > > > On Mon, Jan 5, 2015 at 1:10 PM, Ron Pressler > wrote: > > > > > >> This would mean that ArrayList will implement List (and > will > >> therefore extend List and List) > >> > >> > >> > > > > > > Ron I think this would be a bad idea. Yes, I know that Java > > unfortunately doesn't provide a typesafe null value, but I still think > > that it would be a bad idea to have occurrences of "int" which > > actually mean int|null. I think there's a really strong precedent in > > the language that ints can't be null. > > > > > > OTOH, perhaps it would be viable to make ArrayList a subtype of > > ArrayList. > > > > > > From gavin at hibernate.org Mon Jan 5 20:21:49 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 21:21:49 +0100 Subject: Fwd: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: <54AA7020.6070401@univ-mlv.fr> References: <54A6D751.2080600@oracle.com> <54A81467.8010207@univ-mlv.fr> <54AA7020.6070401@univ-mlv.fr> Message-ID: On Mon, Jan 5, 2015 at 12:06 PM, Remi Forax wrote: > > On 01/05/2015 02:54 AM, Gavin King wrote: >> I'm not seeing this. By nature it's impossible to switch over the >> primitive types with instanceof or whatever, so I'm having a really >> hard time visualizing what kind of Java code would break. >> >> Examples? > > > switch(type.getName()) { ... So you mean the introduction of Any could break code that uses reflection? I mean, sure, I suppose, in principle, but I guess I just don't see this as really a big deal. What would happen in practice is that attempting to use a newly-written class *that uses Any* with an older reflection-based library that hasn't been updated yet would cause an error from the library. But except in very extreme cases, programs that used to work will keep working, since they don't use Any, right? > yes, but you don't need Any to do primitive type specialization, > you just need Any as a bound of a type parameter that's > the whole point of the discussion now ? What I'm saying is that: whatever the implementation, specialization, or something else. There should be a type List that is a supertype of both List and List. *How* that is achieved (via boxing, or via a tagged union) is of much less importance to me. > Yes, it makes the VM more powerful, I think we can all agree with that. > The problem is that its a pervasive change that as far as I know > nobody has even tried. So we know it's really really hard even though nobody has actually tried to do it? Hrm, trying to keep my skepticism in check here ;-) > I will be very happy to see Redhat to found a project like this, > taking the source of hotspot and trying to add Any. Stef and I will chat to the guys working on Open JDK and ask them what they think. > About the separation of behaviors, it's exactly like saying, > if I use a JVM but only with primitive types, no objects, > I will not pay the price of having to deal with objects. > That's not true, the VM will still prepare internal data structures > to be able to crawle the stack during a GC even if a GC > will never occur in that case. Sure, but I don't think that's where most of the cost of having objects comes from. If I write a program that never actually allocates any objects, it will run much faster than one which allocates millions of objects ;-) > Having all values stored as Any require the VM to check the runtime type of > all operands > for each operation, this is what CPython does, but not what more advanced > dynamic language runtimes do. > They do optimistic speculative type analysis, i.e. generate a code by > examples for ints > and de-optimise if a type is a String and then re-generate a new code. > > So sure Any will help, but not as much as you think. OK, that's interesting. >> Couldn't we just say that a byte[] actually isn't an Any[]? I mean, >> that's actually *correct*. Why double down on stupid? > > > how to implement ArrayList in that case ? > > class ArrayList { > T[] array; // <-- this is an array of Any ! > } Ah, yes, so indeed this motivates the idea that ArrayList is not a concrete instantiation. And that's precisely why you prefer ArrayList. OK, that's very interesting. I had not picked up on this one. I'll reflect on this some more, but indeed I see how this could be the killer argument against the "tagged union" approach to Any. Thanks. -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From gavin at hibernate.org Mon Jan 5 20:27:26 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 21:27:26 +0100 Subject: Fwd: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: References: <54A6D751.2080600@oracle.com> <54A81467.8010207@univ-mlv.fr> <54AA7020.6070401@univ-mlv.fr> Message-ID: On Mon, Jan 5, 2015 at 6:48 PM, martin odersky wrote: > I think Any[] needs needs to erase either to Object, or to a new type > such as "AnyArray". You need to use reflection to access the elements > of of an Any[] by looking at the array element type tag. Ahyes, that sounds like it would work. It also sounds like with this approach you might want an additional specialization. You would have: - ArrayList (for ArrayList, ArrayList, etc) - ArrayList$$Any (for ArrayList, uses reflection) - ArrayList$$int and friends Thanks Martin. -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From gavin at hibernate.org Mon Jan 5 20:28:53 2015 From: gavin at hibernate.org (Gavin King) Date: Mon, 5 Jan 2015 21:28:53 +0100 Subject: Simplifying reified generics with partial specialization In-Reply-To: References: Message-ID: On Mon, Jan 5, 2015 at 9:00 PM, Ron Pressler wrote: > But I suggest that, per Brian?s request, we continue this discussion > elsewhere. This isn?t the place for protracted discussions. Dude?! You asked me to post here! -- Gavin King gavin.king at gmail.com http://in.relation.to/Bloggers/Gavin http://hibernate.org http://seamframework.org From maurizio.cimadamore at oracle.com Mon Jan 5 22:16:21 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 05 Jan 2015 22:16:21 +0000 Subject: Fwd: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: References: <54A6D751.2080600@oracle.com> <54A81467.8010207@univ-mlv.fr> Message-ID: <54AB0D35.5050009@oracle.com> On 05/01/15 20:01, Gavin King wrote: > This is exactly the same as with an Any field: it may or may not > reference an instance, depending upon whether it is null or primitive. Exact GC algorithms assume you have a way of knowing where your references are. In JVMs (at least the ones I know) this is typically done by associating a GC bitmask where, for each field of a given class C, a 0 means primitive and 1 means reference; the GC will only 'follow' the fields whose bit is set to 1. Now, you have the problem of 'nulls' but, if you think about it, 'null' is a very very special value - it's a reference with all bits set to 0. So it is very easy for the GC to call that one out. Now think about a field whose type is Any - the mask should be set conservatively to 1, as _in principle_ this might hold a reference to another object, right? But then, how does the VM/GC distinguish between an instance of a value class (i.e. value) and a reference? I.e. how can the GC be instructed not to follow that particular 64-bit value that looks 'like' a reference? Now, I can think of several ways around this (and this is not the point of this email) - but let's not pretend that all this is straightforward. Maurizio From brian.goetz at oracle.com Mon Jan 5 22:29:24 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 17:29:24 -0500 Subject: Fwd: Fwd: Proposal for generics over primitives needs a rethink In-Reply-To: <54AB0D35.5050009@oracle.com> References: <54A6D751.2080600@oracle.com> <54A81467.8010207@univ-mlv.fr> <54AB0D35.5050009@oracle.com> Message-ID: <54AB1044.30103@oracle.com> One note on this: Our VM calls this bitmap the "oopmap", where for each each class it staticall computes the reference-primitive footprint for instances. Value types will require some (small) evolution to this, for collecting arrays of values. Since values may have reference components, you need to overlay the component type oopmap over each element of a packed value array. Not complicated but it is one thing to add to the to-do list. On 1/5/2015 5:16 PM, Maurizio Cimadamore wrote: > > On 05/01/15 20:01, Gavin King wrote: >> This is exactly the same as with an Any field: it may or may not >> reference an instance, depending upon whether it is null or primitive. > Exact GC algorithms assume you have a way of knowing where your > references are. In JVMs (at least the ones I know) this is typically > done by associating a GC bitmask where, for each field of a given class > C, a 0 means primitive and 1 means reference; the GC will only 'follow' > the fields whose bit is set to 1. Now, you have the problem of 'nulls' > but, if you think about it, 'null' is a very very special value - it's a > reference with all bits set to 0. So it is very easy for the GC to call > that one out. > > Now think about a field whose type is Any - the mask should be set > conservatively to 1, as _in principle_ this might hold a reference to > another object, right? But then, how does the VM/GC distinguish between > an instance of a value class (i.e. value) and a reference? I.e. how can > the GC be instructed not to follow that particular 64-bit value that > looks 'like' a reference? > > Now, I can think of several ways around this (and this is not the point > of this email) - but let's not pretend that all this is straightforward. > > Maurizio From twhitmore.nz at gmail.com Mon Jan 5 22:36:16 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Tue, 6 Jan 2015 11:36:16 +1300 Subject: Martin's comments on "Any[] array" performance Message-ID: Hi Martin, people, Just as an aside, I was interested by your comments on "Any arrays" -- Any[] or []T in Scala. This is a bit of a vexed performance area, I'm just thinking it might be worth a look at how native libraries, bytecode, or VM can support this access in a more performant manner. > problem arises for []T, where T is an unbounded type variable. We erase []T to Object. It does > not run super-fast now (performance hit of about a factor of 10-20 compared to > native array access), but it looks easy to optimize if the JVM is aware of it. Not sure if it will be relevant to our specialization efforts directly, but we are talking about this area. Could it be worth a look? Regards, Thomas Whitmore From twhitmore.nz at gmail.com Mon Jan 5 23:16:33 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Tue, 6 Jan 2015 12:16:33 +1300 Subject: further to Stephen's thoughts on Value Objects Message-ID: Hi Stephen, people, Just one further thought about Value Objects -- Stephen's objection was in regard to repeated boxing in/out of API. > Consider an OSS library released for > JDK8 that has a method process(Object). The intent is that the method > accepts anything, we don't care what the type is. When used with > JDK10/11 it still accepts anything, except that instead of a small, > known number of types being boxed (for which the library might have a > process(int) overload, there are now *lots* of boxing operations. I'm not a VM expert, but I understand that locality is the main issue in terms of memory performance. Would it be possible for the 'Value Object boxes' to just hold a pointer to the original Value Object struct, to retain locality & avoid copying? This would still incur cost of creating a box, but keep the benefit of locality. I did look at Albert Noll's report on heisenboxing, and it sounds good -- I'd support formally sidelining or eliminating identity for primitive wrappers -- but didn't discuss the implementation details of boxing, only unboxing. Quite possibly this has already been considered. Regards, Thomas Whitmore From richard.warburton at gmail.com Mon Jan 5 23:43:19 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Mon, 5 Jan 2015 23:43:19 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54AAE765.6040402@oracle.com> References: <54AAE765.6040402@oracle.com> Message-ID: Hi Brian, On Mon, Jan 5, 2015 at 7:35 PM, Brian Goetz wrote: > There's been a lot of passionate discussion here about the directions we > should or should not take Java. And this is all understandable, and perhaps > inevitable, but ... it seems to be crowding out what is actually the > primary purpose of this (-dev) list. We had hoped, in taking the effort to > write up State of the Specialization, to get some comments on ... > specialization :) > > So far, nearly all the comments have been on the speculative parts of the > writeup (layers/conditional methods), as well as the usual (supersized) > portion of "I think you should do X/Y/Z instead". Which is all OK, but we > were actually hoping to get some feedback on the part that is written up in > some detail -- and mostly implemented! I know it's fun to discuss the big > sexy stuff, but we can't let this crowd out the main goal. > > So my question is: has anyone tried to write a program with specialized > generics with the valhalla implementation? Well, we'd really appreciate > it. > Apologies if this is a dumb question or has been already answered but what in the current implementation is expected to work? Is it everything in the latest state of specialization? Not expecting too much detail - just a vague outline ;) regards, Richard Warburton http://insightfullogic.com @RichardWarburto From brian.goetz at oracle.com Tue Jan 6 00:01:39 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 19:01:39 -0500 Subject: Can we also get some feedback on specialization, please? In-Reply-To: References: <54AAE765.6040402@oracle.com> Message-ID: <54AB25E3.1060308@oracle.com> > Apologies if this is a dumb question or has been already answered but > what in the current implementation is expected to work? Is it everything > in the latest state of specialization? Not expecting too much detail - > just a vague outline ;) It should cover all the conversions outlined in the latest SotS. There are a few known things that don't work quite right yet, things like: - Nonstatic inner classes and local classes (we're close on this, though) - super-calls inside generic methods But, it should be enough to write an ArrayList-like class that is generic in any-T. To run it, just clone and build the valhalla repo (make images), and use that as your JDK. From richard.warburton at gmail.com Tue Jan 6 00:19:58 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Tue, 6 Jan 2015 00:19:58 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54AB25E3.1060308@oracle.com> References: <54AAE765.6040402@oracle.com> <54AB25E3.1060308@oracle.com> Message-ID: Hi Brian, It should cover all the conversions outlined in the latest SotS. There are > a few known things that don't work quite right yet, things like: > > - Nonstatic inner classes and local classes (we're close on this, though) > - super-calls inside generic methods > > But, it should be enough to write an ArrayList-like class that is generic > in any-T. > Thanks. To run it, just clone and build the valhalla repo (make images), and use > that as your JDK. > Unless I really need sleep (it is after midnight here) there are no links from the project home page (http://openjdk.java.net/projects/valhalla/) to the source repositories (http://hg.openjdk.java.net/valhalla). I imagine this might cause confusion for casual users who don't know the mapping between project names and repo urls and it would be nice to add it. Or even link to http://hg.openjdk.java.net/valhalla/valhalla - since that's what they'll need to hg clone. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From richard.warburton at gmail.com Tue Jan 6 01:02:31 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Tue, 6 Jan 2015 01:02:31 +0000 Subject: Clarification on migration to value types and boxed vs unboxed representations Message-ID: Hi, Obviously I may have missed some explanation in which case apologies but I'd just like to clarify the situation with respect to migrating existing value-like classes to value types. So let's suppose I've got a class (let's take java.time.LocalDate as an example) which morally should be a value type but isn't because the class predates the language feature. Its final, it isn't serializable or defines a serialization delegates, all its fields are final, the class is immutable. Looks like a great candidate to be converted into a value type and the preconditions seem to apply. Unfortunately some party-pooper somewhere has written code like: LocalDate date = ...; synchronized(date) { // Poop on party } By my reading of the current value types spec one wouldn't be allowed to convert LocalDate to a value type because there's no way to use intrinsic locks on a value type when it has no object header to stash information. It seems like there are other migration headaches as well because currently I can assign a LocalDate value to an Object variable. If value types don't subtype Object (and I'm presuming they won't and not claiming that they should) then any code which does this be in a bad place as well. I guess I got to this point in the email and then realised that you could just box the darn things and this would solve both problems. Does that make sense or am I completely misunderstanding things here? Its probably worth also noting at this point that about 18 months ago the LJC ran a hackday on the IBM Packed Objects proposal which has some overlap with value types. One of the things that people found most confusing was that packed objects had a transparent way of returning what was called a "proxy object". So that's like a boxed version of the packed object. It was not at all obvious when looking at the code that you wrote as to whether you were referring to the packed or the proxy object. I suppose part of that was not being able to easily refer to a spec - but part of it is just the general opaqueness of implicit conversions in general. Having a similar situation with value types would be a real shame. Would it be possible to clarify when value types become converted to their boxed equivalents? regards, Richard Warburton http://insightfullogic.com @RichardWarburto From vitalyd at gmail.com Tue Jan 6 01:25:37 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 20:25:37 -0500 Subject: further to Stephen's thoughts on Value Objects In-Reply-To: References: Message-ID: The problem here would be the added GC pressure from the constant boxjng, unless fully eliminated by the compiler. How would you store a pointer to the value object if that thing never had a heap address? If it's a stack allocated variable then it only lives in stack memory and/or registers. Without JVM gymnastics, you'd have to copy data into a heap box; locality is fine then but you get GC pressure and copying cost. Also, how reliable is it to rely on JVM to always remove the boxing? If this is predicated on escape analysis, I'd be nervous since it leaves quite a bit to be desired. Sent from my phone On Jan 5, 2015 6:17 PM, "Thomas W" wrote: > Hi Stephen, people, > > Just one further thought about Value Objects -- Stephen's objection was in > regard to repeated boxing in/out of API. > > > Consider an OSS library released for > > JDK8 that has a method process(Object). The intent is that the method > > accepts anything, we don't care what the type is. When used with > > JDK10/11 it still accepts anything, except that instead of a small, > > known number of types being boxed (for which the library might have a > > process(int) overload, there are now *lots* of boxing operations. > > I'm not a VM expert, but I understand that locality is the main issue in > terms of memory performance. Would it be possible for the 'Value Object > boxes' to just hold a pointer to the original Value Object struct, to > retain locality & avoid copying? > > This would still incur cost of creating a box, but keep the benefit of > locality. > > I did look at Albert Noll's report on heisenboxing, and it sounds good -- > I'd support formally sidelining or eliminating identity for primitive > wrappers -- but didn't discuss the implementation details of boxing, only > unboxing. Quite possibly this has already been considered. > > > Regards, > Thomas Whitmore > From vitalyd at gmail.com Tue Jan 6 01:28:25 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 20:28:25 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: Message-ID: Boxing it would just create subtle bugs as they'd be synchronizing on a new local box on each invocation. Not sure how to avoid that though if there's existing code lime that. Sent from my phone On Jan 5, 2015 8:02 PM, "Richard Warburton" wrote: > Hi, > > Obviously I may have missed some explanation in which case apologies but > I'd just like to clarify the situation with respect to migrating existing > value-like classes to value types. So let's suppose I've got a class (let's > take java.time.LocalDate as an example) which morally should be a value > type but isn't because the class predates the language feature. Its final, > it isn't serializable or defines a serialization delegates, all its fields > are final, the class is immutable. Looks like a great candidate to be > converted into a value type and the preconditions seem to apply. > > Unfortunately some party-pooper somewhere has written code like: > > LocalDate date = ...; > synchronized(date) { > // Poop on party > } > > By my reading of the current value types spec one wouldn't be allowed to > convert LocalDate to a value type because there's no way to use intrinsic > locks on a value type when it has no object header to stash information. It > seems like there are other migration headaches as well because currently I > can assign a LocalDate value to an Object variable. If value types don't > subtype Object (and I'm presuming they won't and not claiming that they > should) then any code which does this be in a bad place as well. > > I guess I got to this point in the email and then realised that you could > just box the darn things and this would solve both problems. Does that make > sense or am I completely misunderstanding things here? > > Its probably worth also noting at this point that about 18 months ago the > LJC ran a hackday on the IBM Packed Objects proposal which has some overlap > with value types. One of the things that people found most confusing was > that packed objects had a transparent way of returning what was called a > "proxy object". So that's like a boxed version of the packed object. It was > not at all obvious when looking at the code that you wrote as to whether > you were referring to the packed or the proxy object. I suppose part of > that was not being able to easily refer to a spec - but part of it is just > the general opaqueness of implicit conversions in general. > > Having a similar situation with value types would be a real shame. Would it > be possible to clarify when value types become converted to their boxed > equivalents? > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > From john.r.rose at oracle.com Tue Jan 6 01:40:28 2015 From: john.r.rose at oracle.com (John Rose) Date: Mon, 5 Jan 2015 17:40:28 -0800 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: Message-ID: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> On Jan 5, 2015, at 5:02 PM, Richard Warburton wrote: > > Hi, > > Obviously I may have missed some explanation in which case apologies but > I'd just like to clarify the situation with respect to migrating existing > value-like classes to value types. So let's suppose I've got a class (let's > take java.time.LocalDate as an example) which morally should be a value > type but isn't because the class predates the language feature. Its final, > it isn't serializable or defines a serialization delegates, all its fields > are final, the class is immutable. Looks like a great candidate to be > converted into a value type and the preconditions seem to apply. > > Unfortunately some party-pooper somewhere has written code like: > > LocalDate date = ...; > synchronized(date) { > // Poop on party > } The simple answer: It's party foul; show them the door. There are also tales of people synchronizing on interned strings and even interned integers. Code that does that is broken and needs to be identified. Value types are designed to refuse synchronization even if they are boxed into object wrappers. If the javac compiler doesn't see the problem, the JVM will throw something like IllegalMonitorStateException. This is new behavior for objects, but it is far better than a conversion (explicit or implicit) which creates a temporary object that will silently accept a lock attempt. Refusing locks is also cheap and easy to implement. (Unlike special processing for "==", which has its own kind of sadness.) > By my reading of the current value types spec one wouldn't be allowed to > convert LocalDate to a value type because there's no way to use intrinsic > locks on a value type when it has no object header to stash information. It > seems like there are other migration headaches as well because currently I > can assign a LocalDate value to an Object variable. If value types don't > subtype Object (and I'm presuming they won't and not claiming that they > should) then any code which does this be in a bad place as well. > > I guess I got to this point in the email and then realised that you could > just box the darn things and this would solve both problems. Does that make > sense or am I completely misunderstanding things here? Boxing them doesn't solve anything; it just moves the identity bug into an obscure place. > Its probably worth also noting at this point that about 18 months ago the > LJC ran a hackday on the IBM Packed Objects proposal which has some overlap > with value types. One of the things that people found most confusing was > that packed objects had a transparent way of returning what was called a > "proxy object". So that's like a boxed version of the packed object. It was > not at all obvious when looking at the code that you wrote as to whether > you were referring to the packed or the proxy object. I suppose part of > that was not being able to easily refer to a spec - but part of it is just > the general opaqueness of implicit conversions in general. Key question: How did users experience the difference between the packed and the proxy version? In that design it might have been because of side effects viewed through a surprising alias (since packed values are manipulated by-reference). Value types as proposed do not have that set of surprises, because they are immutable. Mutability plus unexpected aliasing causes race conditions and other bugs, which we want to avoid. (BTW, mutable structs plus by-value copying also causes lost writes, which is corner case in C# that we want to avoid. Mutable = bad, sorry.) > Having a similar situation with value types would be a real shame. Would it > be possible to clarify when value types become converted to their boxed > equivalents? If you cast a value to Object you get a box. Just like casting an int to an Object. ("Codes like a class, works like an int.") The syntax for naming the box-type for a value V is TBD; there are not many use cases for it. (Bad bikeshed color of the moment: V&Object.) ? John From vitalyd at gmail.com Tue Jan 6 01:58:46 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 20:58:46 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> Message-ID: Mutable structs aren't used all that often in c# but there are times when they're very useful. C# allows passing them by ref, which mitigates this problem. This also allows one to have large structs but avoid the copying cost. What's the plan to avoid such copying costs with proposed value types in java? Sent from my phone On Jan 5, 2015 8:40 PM, "John Rose" wrote: > On Jan 5, 2015, at 5:02 PM, Richard Warburton > wrote: > > > > Hi, > > > > Obviously I may have missed some explanation in which case apologies but > > I'd just like to clarify the situation with respect to migrating existing > > value-like classes to value types. So let's suppose I've got a class > (let's > > take java.time.LocalDate as an example) which morally should be a value > > type but isn't because the class predates the language feature. Its > final, > > it isn't serializable or defines a serialization delegates, all its > fields > > are final, the class is immutable. Looks like a great candidate to be > > converted into a value type and the preconditions seem to apply. > > > > Unfortunately some party-pooper somewhere has written code like: > > > > LocalDate date = ...; > > synchronized(date) { > > // Poop on party > > } > > The simple answer: It's party foul; show them the door. > > There are also tales of people synchronizing on interned strings and even > interned integers. > > Code that does that is broken and needs to be identified. > > Value types are designed to refuse synchronization even if they are boxed > into object wrappers. If the javac compiler doesn't see the problem, the > JVM will throw something like IllegalMonitorStateException. > > This is new behavior for objects, but it is far better than a conversion > (explicit or implicit) which creates a temporary object that will silently > accept a lock attempt. > > Refusing locks is also cheap and easy to implement. (Unlike special > processing for "==", which has its own kind of sadness.) > > > By my reading of the current value types spec one wouldn't be allowed to > > convert LocalDate to a value type because there's no way to use intrinsic > > locks on a value type when it has no object header to stash information. > It > > seems like there are other migration headaches as well because currently > I > > can assign a LocalDate value to an Object variable. If value types don't > > subtype Object (and I'm presuming they won't and not claiming that they > > should) then any code which does this be in a bad place as well. > > > > I guess I got to this point in the email and then realised that you could > > just box the darn things and this would solve both problems. Does that > make > > sense or am I completely misunderstanding things here? > > Boxing them doesn't solve anything; it just moves the identity bug into an > obscure place. > > > Its probably worth also noting at this point that about 18 months ago the > > LJC ran a hackday on the IBM Packed Objects proposal which has some > overlap > > with value types. One of the things that people found most confusing was > > that packed objects had a transparent way of returning what was called a > > "proxy object". So that's like a boxed version of the packed object. It > was > > not at all obvious when looking at the code that you wrote as to whether > > you were referring to the packed or the proxy object. I suppose part of > > that was not being able to easily refer to a spec - but part of it is > just > > the general opaqueness of implicit conversions in general. > > Key question: How did users experience the difference between the packed > and the proxy version? > > In that design it might have been because of side effects viewed through a > surprising alias (since packed values are manipulated by-reference). Value > types as proposed do not have that set of surprises, because they are > immutable. > > Mutability plus unexpected aliasing causes race conditions and other bugs, > which we want to avoid. > > (BTW, mutable structs plus by-value copying also causes lost writes, which > is corner case in C# that we want to avoid. Mutable = bad, sorry.) > > > Having a similar situation with value types would be a real shame. Would > it > > be possible to clarify when value types become converted to their boxed > > equivalents? > > If you cast a value to Object you get a box. Just like casting an int to > an Object. ("Codes like a class, works like an int.") The syntax for > naming the box-type for a value V is TBD; there are not many use cases for > it. (Bad bikeshed color of the moment: V&Object.) > > ? John From simon at ochsenreither.de Tue Jan 6 02:00:46 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Tue, 06 Jan 2015 03:00:46 +0100 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> Message-ID: <54AB41CE.8030600@ochsenreither.de> > Mutable structs aren't used all that often in c# but there are times when > they're very useful. C# allows passing them by ref, which mitigates this > problem. This also allows one to have large structs but avoid the copying > cost. > > What's the plan to avoid such copying costs with proposed value types in > java? You could pass them in an array. From vitalyd at gmail.com Tue Jan 6 02:02:34 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 21:02:34 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> Message-ID: Also, you can just as well get lost writes with immutable objects if you forget to reassign the variable with the result of a mutating operation. Sent from my phone On Jan 5, 2015 8:40 PM, "John Rose" wrote: > On Jan 5, 2015, at 5:02 PM, Richard Warburton > wrote: > > > > Hi, > > > > Obviously I may have missed some explanation in which case apologies but > > I'd just like to clarify the situation with respect to migrating existing > > value-like classes to value types. So let's suppose I've got a class > (let's > > take java.time.LocalDate as an example) which morally should be a value > > type but isn't because the class predates the language feature. Its > final, > > it isn't serializable or defines a serialization delegates, all its > fields > > are final, the class is immutable. Looks like a great candidate to be > > converted into a value type and the preconditions seem to apply. > > > > Unfortunately some party-pooper somewhere has written code like: > > > > LocalDate date = ...; > > synchronized(date) { > > // Poop on party > > } > > The simple answer: It's party foul; show them the door. > > There are also tales of people synchronizing on interned strings and even > interned integers. > > Code that does that is broken and needs to be identified. > > Value types are designed to refuse synchronization even if they are boxed > into object wrappers. If the javac compiler doesn't see the problem, the > JVM will throw something like IllegalMonitorStateException. > > This is new behavior for objects, but it is far better than a conversion > (explicit or implicit) which creates a temporary object that will silently > accept a lock attempt. > > Refusing locks is also cheap and easy to implement. (Unlike special > processing for "==", which has its own kind of sadness.) > > > By my reading of the current value types spec one wouldn't be allowed to > > convert LocalDate to a value type because there's no way to use intrinsic > > locks on a value type when it has no object header to stash information. > It > > seems like there are other migration headaches as well because currently > I > > can assign a LocalDate value to an Object variable. If value types don't > > subtype Object (and I'm presuming they won't and not claiming that they > > should) then any code which does this be in a bad place as well. > > > > I guess I got to this point in the email and then realised that you could > > just box the darn things and this would solve both problems. Does that > make > > sense or am I completely misunderstanding things here? > > Boxing them doesn't solve anything; it just moves the identity bug into an > obscure place. > > > Its probably worth also noting at this point that about 18 months ago the > > LJC ran a hackday on the IBM Packed Objects proposal which has some > overlap > > with value types. One of the things that people found most confusing was > > that packed objects had a transparent way of returning what was called a > > "proxy object". So that's like a boxed version of the packed object. It > was > > not at all obvious when looking at the code that you wrote as to whether > > you were referring to the packed or the proxy object. I suppose part of > > that was not being able to easily refer to a spec - but part of it is > just > > the general opaqueness of implicit conversions in general. > > Key question: How did users experience the difference between the packed > and the proxy version? > > In that design it might have been because of side effects viewed through a > surprising alias (since packed values are manipulated by-reference). Value > types as proposed do not have that set of surprises, because they are > immutable. > > Mutability plus unexpected aliasing causes race conditions and other bugs, > which we want to avoid. > > (BTW, mutable structs plus by-value copying also causes lost writes, which > is corner case in C# that we want to avoid. Mutable = bad, sorry.) > > > Having a similar situation with value types would be a real shame. Would > it > > be possible to clarify when value types become converted to their boxed > > equivalents? > > If you cast a value to Object you get a box. Just like casting an int to > an Object. ("Codes like a class, works like an int.") The syntax for > naming the box-type for a value V is TBD; there are not many use cases for > it. (Bad bikeshed color of the moment: V&Object.) > > ? John From vitalyd at gmail.com Tue Jan 6 02:05:15 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 21:05:15 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: <54AB41CE.8030600@ochsenreither.de> References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> <54AB41CE.8030600@ochsenreither.de> Message-ID: How does that help? You need a heap object (array) and if you then modify one of the structs in there, you need to copy it and emplace it back into the array slot. Sent from my phone On Jan 5, 2015 9:01 PM, "Simon Ochsenreither" wrote: > Mutable structs aren't used all that often in c# but there are times when >> they're very useful. C# allows passing them by ref, which mitigates this >> problem. This also allows one to have large structs but avoid the copying >> cost. >> >> What's the plan to avoid such copying costs with proposed value types in >> java? >> > You could pass them in an array. > From mikeb01 at gmail.com Tue Jan 6 02:14:43 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Tue, 6 Jan 2015 15:14:43 +1300 Subject: Question on layer/peeling Message-ID: Hi, The SotS talks about the use of 'layer' to create an alternative implementation of methods when the type of an is known to be a reference type. However, the examples only show the use of the layer keyword on an interface definition, where as I've encountered at least one case where the internal implementation needs to differentiate between a reference-type and value-type based collection. The example I'm thinking about is the null-ing out of array elements in a collection (which is obviously a no-op with a value type, but necessity with reference types). Is an interface required in order to define a 'layer' or could it be done within a concrete class? E.g. is the following or something similar possible? If not, how would it be achieved with current spec? class ArrayList { T[] values; int position; void removeLast() { if (position <= 0) { return; } --position; clear(position); } private void clear(int index) { } layer { private void clear(int index) { values[index] = null; } } } Mike. From brian.goetz at oracle.com Tue Jan 6 02:15:17 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 21:15:17 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: Message-ID: <54AB4535.1040500@oracle.com> > Obviously I may have missed some explanation in which case apologies but > I'd just like to clarify the situation with respect to migrating existing > value-like classes to value types. So let's suppose I've got a class (let's > take java.time.LocalDate as an example) which morally should be a value > type but isn't because the class predates the language feature. Its final, > it isn't serializable or defines a serialization delegates, all its fields > are final, the class is immutable. Looks like a great candidate to be > converted into a value type and the preconditions seem to apply. This what we call a /value-based/ class. The question is, I believe, will it possible to (compatibly) migrate a class that was a value-based reference class into a value type. We certainly hope so, but we don't yet have a story for this, as I mentioned the other day. As John pointed out, we're pretty comfortable with the consequences for synchronization (both the 310 classes an Optional refer to the value-based disclaimer, which says "do this and you'll be sorry.") The hard part about this migration compatibility is: nullability. (This is where someone repeats the usual commercial for null-safe types and/or elvii operators.) While null truly makes no sense for Optional, and I could imagine being willing to hose code that relies on Optionals being null, this is probably the exception among known value-based classes (e.g., LocalDate). So while we may have a migration story for libraries that dispense and receive formerly-value-based value types, we don't have a story for migrating client code that might expect nullability for those types. (As to the timing of boxing and unboxing, it would be exactly as primitives today.) From vitalyd at gmail.com Tue Jan 6 02:19:45 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 21:19:45 -0500 Subject: Question on layer/peeling In-Reply-To: References: Message-ID: C# has a default (T) keyword to allow generic code to obtain the "zero" value for a type param. Something like that for java would be nice. Sent from my phone On Jan 5, 2015 9:15 PM, "Michael Barker" wrote: > Hi, > > The SotS talks about the use of 'layer' to create an alternative > implementation of methods when the type of an is known to be a > reference type. However, the examples only show the use of the layer > keyword on an interface definition, where as I've encountered at least one > case where the internal implementation needs to differentiate between a > reference-type and value-type based collection. The example I'm thinking > about is the null-ing out of array elements in a collection (which is > obviously a no-op with a value type, but necessity with reference > types). Is an interface required in order to define a 'layer' or could it > be done within a concrete class? > > E.g. is the following or something similar possible? If not, how would it > be achieved with current spec? > > class ArrayList { > T[] values; > int position; > > void removeLast() { > if (position <= 0) { > return; > } > > --position; > clear(position); > } > > private void clear(int index) { > } > > layer { > private void clear(int index) { > values[index] = null; > } > } > } > > Mike. > From brian.goetz at oracle.com Tue Jan 6 02:21:56 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 21:21:56 -0500 Subject: Question on layer/peeling In-Reply-To: References: Message-ID: <54AB46C4.3040408@oracle.com> (Public service announcement: Any talk about "spec" is woefully premature; what we've got is an idea on a white board.) The case you describe is what we call "implementation by parts", where there is a method who is a member of Foo but which might have different implementation depending on whether T is a value or an (erased) reference. This is one of the use cases described in SotS, and such things come up in a number of places inside the implementation of Collections. While we care little about syntax right now, what you wrote is indeed in the spirit of the idea, and how you would address an "implementation by parts" in that situation. Implementation-wise, this is pretty straightforward with the existing specialization story. The compiler emits two methods with some "applicability" attributes, and the specializer selects which to propagate into the specialized classfile. On 1/5/2015 9:14 PM, Michael Barker wrote: > Hi, > > The SotS talks about the use of 'layer' to create an alternative > implementation of methods when the type of an is known to be a > reference type. However, the examples only show the use of the layer > keyword on an interface definition, where as I've encountered at least one > case where the internal implementation needs to differentiate between a > reference-type and value-type based collection. The example I'm thinking > about is the null-ing out of array elements in a collection (which is > obviously a no-op with a value type, but necessity with reference > types). Is an interface required in order to define a 'layer' or could it > be done within a concrete class? > > E.g. is the following or something similar possible? If not, how would it > be achieved with current spec? > > class ArrayList { > T[] values; > int position; > > void removeLast() { > if (position <= 0) { > return; > } > > --position; > clear(position); > } > > private void clear(int index) { > } > > layer { > private void clear(int index) { > values[index] = null; > } > } > } > > Mike. > From brian.goetz at oracle.com Tue Jan 6 02:23:46 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 21:23:46 -0500 Subject: Question on layer/peeling In-Reply-To: References: Message-ID: <54AB4732.6080207@oracle.com> Yes, this is pretty straightforward. In the bucket of "things that are easy and small, so we'll ignore them until we solve the ones that are big and difficult.") The hardest part is picking a syntax (please, no suggestions!) On 1/5/2015 9:19 PM, Vitaly Davidovich wrote: > C# has a default (T) keyword to allow generic code to obtain the "zero" > value for a type param. Something like that for java would be nice. > > Sent from my phone > On Jan 5, 2015 9:15 PM, "Michael Barker" wrote: > >> Hi, >> >> The SotS talks about the use of 'layer' to create an alternative >> implementation of methods when the type of an is known to be a >> reference type. However, the examples only show the use of the layer >> keyword on an interface definition, where as I've encountered at least one >> case where the internal implementation needs to differentiate between a >> reference-type and value-type based collection. The example I'm thinking >> about is the null-ing out of array elements in a collection (which is >> obviously a no-op with a value type, but necessity with reference >> types). Is an interface required in order to define a 'layer' or could it >> be done within a concrete class? >> >> E.g. is the following or something similar possible? If not, how would it >> be achieved with current spec? >> >> class ArrayList { >> T[] values; >> int position; >> >> void removeLast() { >> if (position <= 0) { >> return; >> } >> >> --position; >> clear(position); >> } >> >> private void clear(int index) { >> } >> >> layer { >> private void clear(int index) { >> values[index] = null; >> } >> } >> } >> >> Mike. >> From vitalyd at gmail.com Tue Jan 6 02:26:43 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 21:26:43 -0500 Subject: Question on layer/peeling In-Reply-To: <54AB4732.6080207@oracle.com> References: <54AB4732.6080207@oracle.com> Message-ID: Sure, I don't care about syntax at this point so long as this (common) use case will be straightforward to support. Sent from my phone On Jan 5, 2015 9:24 PM, "Brian Goetz" wrote: > Yes, this is pretty straightforward. In the bucket of "things that are > easy and small, so we'll ignore them until we solve the ones that are big > and difficult.") > > The hardest part is picking a syntax (please, no suggestions!) > > > On 1/5/2015 9:19 PM, Vitaly Davidovich wrote: > >> C# has a default (T) keyword to allow generic code to obtain the "zero" >> value for a type param. Something like that for java would be nice. >> >> Sent from my phone >> On Jan 5, 2015 9:15 PM, "Michael Barker" wrote: >> >> Hi, >>> >>> The SotS talks about the use of 'layer' to create an alternative >>> implementation of methods when the type of an is known to be a >>> reference type. However, the examples only show the use of the layer >>> keyword on an interface definition, where as I've encountered at least >>> one >>> case where the internal implementation needs to differentiate between a >>> reference-type and value-type based collection. The example I'm thinking >>> about is the null-ing out of array elements in a collection (which is >>> obviously a no-op with a value type, but necessity with reference >>> types). Is an interface required in order to define a 'layer' or could >>> it >>> be done within a concrete class? >>> >>> E.g. is the following or something similar possible? If not, how would >>> it >>> be achieved with current spec? >>> >>> class ArrayList { >>> T[] values; >>> int position; >>> >>> void removeLast() { >>> if (position <= 0) { >>> return; >>> } >>> >>> --position; >>> clear(position); >>> } >>> >>> private void clear(int index) { >>> } >>> >>> layer { >>> private void clear(int index) { >>> values[index] = null; >>> } >>> } >>> } >>> >>> Mike. >>> >>> From richard.warburton at gmail.com Tue Jan 6 02:36:16 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Tue, 6 Jan 2015 02:36:16 +0000 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> Message-ID: Hi, Thanks for your reply John - that clarified a lot. > Its probably worth also noting at this point that about 18 months ago the > > LJC ran a hackday on the IBM Packed Objects proposal which has some > overlap > > with value types. One of the things that people found most confusing was > > that packed objects had a transparent way of returning what was called a > > "proxy object". So that's like a boxed version of the packed object. It > was > > not at all obvious when looking at the code that you wrote as to whether > > you were referring to the packed or the proxy object. I suppose part of > > that was not being able to easily refer to a spec - but part of it is > just > > the general opaqueness of implicit conversions in general. > > Key question: How did users experience the difference between the packed > and the proxy version? > > In that design it might have been because of side effects viewed through a > surprising alias (since packed values are manipulated by-reference). Value > types as proposed do not have that set of surprises, because they are > immutable. > No - I think the confusion was that you couldn't look at the code and obviously see what was a reference and what was packed when thinking about the object graph. This applied to reading from values as much as writing to them. I appreciate that mutability adds its own set of confusions but IMHO this wasn't one of them. > Having a similar situation with value types would be a real shame. Would > it > > be possible to clarify when value types become converted to their boxed > > equivalents? > > If you cast a value to Object you get a box. Just like casting an int to > an Object. ("Codes like a class, works like an int.") The syntax for > naming the box-type for a value V is TBD; there are not many use cases for > it. (Bad bikeshed color of the moment: V&Object.) That's re-assuring to know. I guess of more concern to me is that re-reading once I assign the value to an interface there will be boxing (right?), and this seems a lot trickier to spot than the explicit casting case. I'll try to keep an eye out for this situation. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From mikeb01 at gmail.com Tue Jan 6 02:39:54 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Tue, 6 Jan 2015 15:39:54 +1300 Subject: Question on layer/peeling In-Reply-To: References: Message-ID: > > C# has a default (T) keyword to allow generic code to obtain the "zero" > value for a type param. Something like that for java would be nice. > When I first wrote the example, I was thinking that a simple no-op in the value type case would be sufficient. However, now that I think about it something like a "zero" value might be necessary. If the value type used within the collection happened to have a reference as one of its components then there is the ability to leak memory. For this case, it wouldn't need to be user-defined as long as "zero" meant that all references are null. Mike. From vitalyd at gmail.com Tue Jan 6 02:42:42 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 21:42:42 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> Message-ID: The re-reading a value type into interface causing boxing is typically solved (again, using .net as an example) by having constrained generic methods that are constrained to receiving an interface type (or subtype), I.e. using the "where" clause in c#. In those cases, there's no boxing. With specialization, I imagine that will work as well although need to see if/how constraints will be surfaced. Sent from my phone On Jan 5, 2015 9:36 PM, "Richard Warburton" wrote: > Hi, > > Thanks for your reply John - that clarified a lot. > > > Its probably worth also noting at this point that about 18 months ago the > > > LJC ran a hackday on the IBM Packed Objects proposal which has some > > overlap > > > with value types. One of the things that people found most confusing > was > > > that packed objects had a transparent way of returning what was called > a > > > "proxy object". So that's like a boxed version of the packed object. It > > was > > > not at all obvious when looking at the code that you wrote as to > whether > > > you were referring to the packed or the proxy object. I suppose part of > > > that was not being able to easily refer to a spec - but part of it is > > just > > > the general opaqueness of implicit conversions in general. > > > > Key question: How did users experience the difference between the packed > > and the proxy version? > > > > In that design it might have been because of side effects viewed through > a > > surprising alias (since packed values are manipulated by-reference). > Value > > types as proposed do not have that set of surprises, because they are > > immutable. > > > > No - I think the confusion was that you couldn't look at the code and > obviously see what was a reference and what was packed when thinking about > the object graph. This applied to reading from values as much as writing to > them. I appreciate that mutability adds its own set of confusions but IMHO > this wasn't one of them. > > > Having a similar situation with value types would be a real shame. Would > > it > > > be possible to clarify when value types become converted to their boxed > > > equivalents? > > > > If you cast a value to Object you get a box. Just like casting an int to > > an Object. ("Codes like a class, works like an int.") The syntax for > > naming the box-type for a value V is TBD; there are not many use cases > for > > it. (Bad bikeshed color of the moment: V&Object.) > > > That's re-assuring to know. I guess of more concern to me is that > re-reading once I assign the value to an interface there will be boxing > (right?), and this seems a lot trickier to spot than the explicit casting > case. I'll try to keep an eye out for this situation. > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > From mikeb01 at gmail.com Tue Jan 6 02:43:47 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Tue, 6 Jan 2015 15:43:47 +1300 Subject: Question on layer/peeling In-Reply-To: <54AB46C4.3040408@oracle.com> References: <54AB46C4.3040408@oracle.com> Message-ID: > > (Public service announcement: Any talk about "spec" is woefully premature; > what we've got is an idea on a white board.) > Ooops, s/spec/draft proposal/g > The case you describe is what we call "implementation by parts", where > there is a method who is a member of Foo but which might have > different implementation depending on whether T is a value or an (erased) > reference. This is one of the use cases described in SotS, and such things > come up in a number of places inside the implementation of Collections. > That was my expectation. I toying with a specialised HashMap to a feel for how specialisation will work. I'm assuming that the layer parts aren't implemented yet - I couldn't find anything in a cursory browse of the commits. Is there something I could do in the meantime as a hack (e.g. instanceof check and downcast to Object[])? Mike. From brian.goetz at oracle.com Tue Jan 6 02:48:02 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 21:48:02 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> Message-ID: <54AB4CE2.8050208@oracle.com> > That was my expectation. I toying with a specialised HashMap to a feel > for how specialisation will work. I'm assuming that the layer parts > aren't implemented yet - I couldn't find anything in a cursory browse of > the commits. Is there something I could do in the meantime as a hack > (e.g. instanceof check and downcast to Object[])? Thanks for this. This is the kind of useful experiment that we are happy to see! The layer part is indeed not yet implemented. But even a catalogue of where the code can't be made value-clean would be a useful contribution. We know that the Collections implementation is full of stuff that will be hard to any-fy without help from layering or other tools, but don't yet have a complete catalog. From richard.warburton at gmail.com Tue Jan 6 02:59:19 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Tue, 6 Jan 2015 02:59:19 +0000 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: <54AB4535.1040500@oracle.com> References: <54AB4535.1040500@oracle.com> Message-ID: Hi Brian, Obviously I may have missed some explanation in which case apologies but >> I'd just like to clarify the situation with respect to migrating existing >> value-like classes to value types. So let's suppose I've got a class >> (let's >> take java.time.LocalDate as an example) which morally should be a value >> type but isn't because the class predates the language feature. Its final, >> it isn't serializable or defines a serialization delegates, all its fields >> are final, the class is immutable. Looks like a great candidate to be >> converted into a value type and the preconditions seem to apply. >> > > This what we call a /value-based/ class. > > The question is, I believe, will it possible to (compatibly) migrate a > class that was a value-based reference class into a value type. We > certainly hope so, but we don't yet have a story for this, as I mentioned > the other day. > Sorry - still catching up on my reading backlog on this list. As John pointed out, we're pretty comfortable with the consequences for > synchronization (both the 310 classes an Optional refer to the value-based > disclaimer, which says "do this and you'll be sorry.") Developers are incredibly unlikely to have read this javadoc line. I imagine you already know this, but for posterity's sake it seemed worth noting it. I suspect that 3rd party library authors are in a more pickley situation having not planned this through as clearly. Upside: they are probably willing to do backwards incompatible releases where its a very corner-case situation. I'm trying to think if I believe this to be worse than variables called enum or implementations of List that had `List sort(Comparator)`. The hard part about this migration compatibility is: nullability. (This is > where someone repeats the usual commercial for null-safe types and/or elvii > operators.) While null truly makes no sense for Optional, and I could > imagine being willing to hose code that relies on Optionals being null, > this is probably the exception among known value-based classes (e.g., > LocalDate). So while we may have a migration story for libraries that > dispense and receive formerly-value-based value types, we don't have a > story for migrating client code that might expect nullability for those > types. > Yup - definitely reasonable with LocalDate. I think the lock situation is probably problematic for people who haven't explicitly caveated it in their documentation - I can't see any sensible reason why you would do that but someone, somewhere will have done so. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From brian.goetz at oracle.com Tue Jan 6 03:10:28 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 22:10:28 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: <54AB4535.1040500@oracle.com> Message-ID: <54AB5224.2090405@oracle.com> > I suspect that 3rd party library authors are in a more pickley situation > having not planned this through as clearly. Upside: they are probably > willing to do backwards incompatible releases where its a very > corner-case situation. I'm trying to think if I believe this to be worse > than variables called enum or implementations of List that had `List > sort(Comparator)`. Age makes a difference too. Optional is new, and the disclaimers arrived on day 1. Integer, on the other hand, is probably hopelessly polluted, and I am sure that it would break gobs of important code if Integer ceased to be lockable (despite what we may think of such a practice.) It may be that Optional remains forever polluted by identity, preventing us from migrating it to a value, which would be Really Sad, but not out of the realm of possible. I'm still optimistic (mostly probably because I've not thought really hard about it yet.) From john.r.rose at oracle.com Tue Jan 6 03:43:28 2015 From: john.r.rose at oracle.com (John Rose) Date: Mon, 5 Jan 2015 19:43:28 -0800 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> Message-ID: <9401B971-06D9-42E6-95C7-88AC395B160E@oracle.com> On Jan 5, 2015, at 6:02 PM, Vitaly Davidovich wrote: > Also, you can just as well get lost writes with immutable objects if you forget to reassign the variable with the result of a mutating operation. The risks are in some sense complementary. But IMO this one, unlike the aliasing problem, is easy to check for statically: class ImmutableComplex { ... @Preserve ImmutableComplex add(ImmutableComplex that) { return new ImmutableComplex(this.re + that.re, this.im + that.im); } } class String { ... @Preserve String concat(String str) { ... } } void dufus(String s) { s.concat(" more!"); // javac says Warning: discarding a preserved value return s; } ? John From mikeb01 at gmail.com Tue Jan 6 03:52:57 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Tue, 6 Jan 2015 16:52:57 +1300 Subject: Question on layer/peeling In-Reply-To: <54AB4CE2.8050208@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> Message-ID: > > The layer part is indeed not yet implemented. But even a catalogue of > where the code can't be made value-clean would be a useful contribution. > We know that the Collections implementation is full of stuff that will be > hard to any-fy without help from layering or other tools, but don't yet > have a complete catalog. > I think I've managed to hit the wall with the current implementation of Valhalla. I think the "implementation by parts" work or something similar needs to make progress before moderately complex collections like HashMap can be implemented. So, sticking points thus far. 1) Nulls - checking against null inputs. Ideally no-op and always true for values. 2) Nulls - nulling out reference array elements. Need some notion of "zero" for value types, so that nested value types containing references don't leak memory. 3) Methods from Object, e.g. hashCode and equals. At the moment an doesn't have any methods, there's no obvious way to get the hash code of, or equate any-type variables. #1 & #2 can be worked around by simply not doing them and suffering the negative effects (NPEs and memory leaks respectively). I can't find an easy way to work-around #3. It was mentioned that value types will need to define hashCode/equals/toString. If this is the case, should the existing primitives support them too, such all possible types of an any-typed variable supports said methods? Obviously this impacts a HashMap implementation heavily as it is all about the hashCode and equals, but would probably impact an array ArrayList implementation too as it requires equals to support contains. Mike. From john.r.rose at oracle.com Tue Jan 6 03:55:01 2015 From: john.r.rose at oracle.com (John Rose) Date: Mon, 5 Jan 2015 19:55:01 -0800 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> Message-ID: <2BC6CBEC-989C-420D-85C4-D80E0D883196@oracle.com> On Jan 5, 2015, at 5:58 PM, Vitaly Davidovich wrote: > What's the plan to avoid such copying costs with proposed value types in java? Value types are designed for small structures of a few words, like complex numbers or foreign pointers, not wide aggregates like "struct stat". If there are enough use cases for wide values, we can reduce copying by adding hidden indirections. But building indirections into the design defeats one of our goals, which is memory locality. Locality is obtained by placing a value inside of its containing object or array. Clever bit-sharing indirections, at least for small values, would probably add too much overhead. Value types are not structs, and will be useful for emulating only those C structs which are passed by value. Do you have another use case in mind? ? John From vitalyd at gmail.com Tue Jan 6 04:33:32 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 23:33:32 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: <2BC6CBEC-989C-420D-85C4-D80E0D883196@oracle.com> References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> <2BC6CBEC-989C-420D-85C4-D80E0D883196@oracle.com> Message-ID: The use cases in mind all fall into the "wide" struct case, whether being wide in breadth (lots of fields) or depth (structs embedding other structs). Generally speaking, there are cases where you want to bundle up a bunch of mutable state (primitives mostly, but can be mix of refs as well) and then pass that blob around. In financial markets, there are entities which carry a good deal of timestamps for example, and some of those timestamps may be mutated/set by different phases in a processing pipeline; the terminal phase then processes all of them. Similarly, in graphics code, you may have lots of temporaries that are set up in a render pipeline across different stages before finally being rendered. As I mentioned, mutable struct cases aren't that frequent but quite useful to have when needed. In particular, if the cost of copying the struct around is more expensive than the operations performed on its data, we're back to having mechanical overhead exceeding useful work. In this sense, I feel that .NET got this part right by providing flexibility and some tools (e.g. pass by ref) to handle these instances neatly without sacrificing performance or encapsulation. Agree that taking away locality by playing indirection games isn't a worthwhile tradeoff. It is, however, somewhat unfortunate that java's version of this concept won't be as powerful (or flexible, if you prefer) as .NET. If one could fuse the CLR type system with JVM's runtime (GC, JIT), I think you'd get one heck of a performant managed runtime; one can dream! :) Having said all that, I'll still take immutable value types over the current bloat/indirection situation, so thanks for taking up this effort. Sent from my phone On Jan 5, 2015 10:55 PM, "John Rose" wrote: > On Jan 5, 2015, at 5:58 PM, Vitaly Davidovich wrote: > > What's the plan to avoid such copying costs with proposed value types in > java? > > Value types are designed for small structures of a few words, like complex > numbers or foreign pointers, not wide aggregates like "struct stat". > > If there are enough use cases for wide values, we can reduce copying by > adding hidden indirections. But building indirections into the design > defeats one of our goals, which is memory locality. Locality is obtained > by placing a value inside of its containing object or array. Clever > bit-sharing indirections, at least for small values, would probably add too > much overhead. > > Value types are not structs, and will be useful for emulating only those C > structs which are passed by value. > > Do you have another use case in mind? > > ? John From brian.goetz at oracle.com Tue Jan 6 04:36:15 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 05 Jan 2015 23:36:15 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> Message-ID: <54AB663F.4020501@oracle.com> Right, thanks. For #3, we could easily lift these methods to primitives. Which would freak out some users, to see 1.hashCode() (but this isn't all that much freakier than the existing "".hashCode() which also freaks some people out.) There's a relatively short path to getting this working: - Compiler accepts invocations of these methods on any-typed expressions, and emits calls to the Object methods; - Compiler includes receiver metadata in the bytecode mapping attribute (already done) - Specializer handles the case where you invoke one of these methods on a raw "any T" and rewrites as invocations to (say) Integer.toString(). Probably only a few hours of work. On 1/5/2015 10:52 PM, Michael Barker wrote: > The layer part is indeed not yet implemented. But even a catalogue > of where the code can't be made value-clean would be a useful > contribution. We know that the Collections implementation is full > of stuff that will be hard to any-fy without help from layering or > other tools, but don't yet have a complete catalog. > > > I think I've managed to hit the wall with the current implementation of > Valhalla. I think the "implementation by parts" work or something > similar needs to make progress before moderately complex collections > like HashMap can be implemented. So, sticking points thus far. > > 1) Nulls - checking against null inputs. Ideally no-op and always true > for values. > 2) Nulls - nulling out reference array elements. Need some notion of > "zero" for value types, so that nested value types containing references > don't leak memory. > 3) Methods from Object, e.g. hashCode and equals. At the moment an T> doesn't have any methods, there's no obvious way to get the hash code > of, or equate any-type variables. > > #1 & #2 can be worked around by simply not doing them and suffering the > negative effects (NPEs and memory leaks respectively). I can't find an > easy way to work-around #3. It was mentioned that value types will need > to define hashCode/equals/toString. If this is the case, should the > existing primitives support them too, such all possible types of an > any-typed variable supports said methods? Obviously this impacts a > HashMap implementation heavily as it is all about the hashCode and > equals, but would probably impact an array ArrayList implementation too > as it requires equals to support contains. > > Mike. > From vitalyd at gmail.com Tue Jan 6 04:37:55 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 23:37:55 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> Message-ID: I suppose the workaround for #3 would be to take a Hasher (or whatever name you pick) instance as a ctor arg; this would provide equals/hashCode for your type T. Sent from my phone On Jan 5, 2015 10:53 PM, "Michael Barker" wrote: > > > > The layer part is indeed not yet implemented. But even a catalogue of > > where the code can't be made value-clean would be a useful contribution. > > We know that the Collections implementation is full of stuff that will be > > hard to any-fy without help from layering or other tools, but don't yet > > have a complete catalog. > > > > I think I've managed to hit the wall with the current implementation of > Valhalla. I think the "implementation by parts" work or something similar > needs to make progress before moderately complex collections like HashMap > can be implemented. So, sticking points thus far. > > 1) Nulls - checking against null inputs. Ideally no-op and always true for > values. > 2) Nulls - nulling out reference array elements. Need some notion of > "zero" for value types, so that nested value types containing references > don't leak memory. > 3) Methods from Object, e.g. hashCode and equals. At the moment an > doesn't have any methods, there's no obvious way to get the hash code of, > or equate any-type variables. > > #1 & #2 can be worked around by simply not doing them and suffering the > negative effects (NPEs and memory leaks respectively). I can't find an > easy way to work-around #3. It was mentioned that value types will need to > define hashCode/equals/toString. If this is the case, should the existing > primitives support them too, such all possible types of an any-typed > variable supports said methods? Obviously this impacts a HashMap > implementation heavily as it is all about the hashCode and equals, but > would probably impact an array ArrayList implementation too as it requires > equals to support contains. > > Mike. > From vitalyd at gmail.com Tue Jan 6 04:46:13 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 5 Jan 2015 23:46:13 -0500 Subject: Question on layer/peeling In-Reply-To: <54AB663F.4020501@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> Message-ID: Nothing freaky about this :) in fact, had value types been there since day 1, I'm sure the primitives would've been defined as value types and thus have those methods. To mimic that in today's world via specializer, allowing a call to a method defined on Object against "any T" seems legit. Some people will complain that you can't do the same to a naked primitive (I.e. 1.hashCode) outside the specialization context, though, I suspect ... Sent from my phone On Jan 5, 2015 11:37 PM, "Brian Goetz" wrote: > Right, thanks. For #3, we could easily lift these methods to primitives. > Which would freak out some users, to see > > 1.hashCode() > > (but this isn't all that much freakier than the existing > > "".hashCode() > > which also freaks some people out.) > > There's a relatively short path to getting this working: > - Compiler accepts invocations of these methods on any-typed expressions, > and emits calls to the Object methods; > - Compiler includes receiver metadata in the bytecode mapping attribute > (already done) > - Specializer handles the case where you invoke one of these methods on a > raw "any T" and rewrites as invocations to (say) Integer.toString(). > > Probably only a few hours of work. > > On 1/5/2015 10:52 PM, Michael Barker wrote: > >> The layer part is indeed not yet implemented. But even a catalogue >> of where the code can't be made value-clean would be a useful >> contribution. We know that the Collections implementation is full >> of stuff that will be hard to any-fy without help from layering or >> other tools, but don't yet have a complete catalog. >> >> >> I think I've managed to hit the wall with the current implementation of >> Valhalla. I think the "implementation by parts" work or something >> similar needs to make progress before moderately complex collections >> like HashMap can be implemented. So, sticking points thus far. >> >> 1) Nulls - checking against null inputs. Ideally no-op and always true >> for values. >> 2) Nulls - nulling out reference array elements. Need some notion of >> "zero" for value types, so that nested value types containing references >> don't leak memory. >> 3) Methods from Object, e.g. hashCode and equals. At the moment an > T> doesn't have any methods, there's no obvious way to get the hash code >> of, or equate any-type variables. >> >> #1 & #2 can be worked around by simply not doing them and suffering the >> negative effects (NPEs and memory leaks respectively). I can't find an >> easy way to work-around #3. It was mentioned that value types will need >> to define hashCode/equals/toString. If this is the case, should the >> existing primitives support them too, such all possible types of an >> any-typed variable supports said methods? Obviously this impacts a >> HashMap implementation heavily as it is all about the hashCode and >> equals, but would probably impact an array ArrayList implementation too >> as it requires equals to support contains. >> >> Mike. >> >> From john.r.rose at oracle.com Tue Jan 6 06:26:47 2015 From: john.r.rose at oracle.com (John Rose) Date: Mon, 5 Jan 2015 22:26:47 -0800 Subject: Question on layer/peeling In-Reply-To: <54AB663F.4020501@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> Message-ID: <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> On Jan 5, 2015, at 8:36 PM, Brian Goetz wrote: > > There's a relatively short path to getting this working: > - Compiler accepts invocations of these methods on any-typed expressions, and emits calls to the Object methods; > - Compiler includes receiver metadata in the bytecode mapping attribute (already done) > - Specializer handles the case where you invoke one of these methods on a raw "any T" and rewrites as invocations to (say) Integer.toString(). I.e., Object is an honorary supertype of all types, by virtue of boxing conversions. This trick is a good one, and it can go a bit farther with primitives, if you think about the other supertypes of Integer, Double, etc., especially Comparable. (I'd kind of like to define an interface Objectable to carry toS/eq/hC.) The trick can also be coaxed to give semi-plausible semantics to seemingly bogus expressions like "x == null" or "x == (Object)y" (for any x). If "x" is a primitive, since autobox conversions never produce "null", then "x == null" and "x == (Object)y" are complicated ways of saying "false". But this doesn't help with nulling assignments like "a[i] = null"; there you need "T.null" (my bikeshed preference to "default(T)"). It might possibly help to add interfaces to the existing box types to factor out additional common behaviors, such as "+". Or, it might be an intractable can of worms, to enumerate and refactor all the primitive operations. Check Haskell's prelude definitions on arithmetics, for many interesting examples of bridges too far (for Java). In any case, the value types draft allows values to implement interfaces, which can factor common behaviors across disparate values. For example, a TreeMap will be able to contain value types that implement Comparable. (This fact is independent of whether and how TreeMap instances would be able to specialize to the value types, or would just use the generic erased structure to hold boxed values.) ? John P.S. The "nulling" operation is tricky for the reason Mike points out: You can't tell whether a value type embeds a pointer or not. VarHandles might be able to help with this, with a "resetReference" operation which is elidable if the target type has no references. From peter.levart at gmail.com Tue Jan 6 07:56:15 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 06 Jan 2015 08:56:15 +0100 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: <54AB4535.1040500@oracle.com> References: <54AB4535.1040500@oracle.com> Message-ID: <54AB951F.50203@gmail.com> On 01/06/2015 03:15 AM, Brian Goetz wrote: >> Obviously I may have missed some explanation in which case apologies but >> I'd just like to clarify the situation with respect to migrating >> existing >> value-like classes to value types. So let's suppose I've got a class >> (let's >> take java.time.LocalDate as an example) which morally should be a value >> type but isn't because the class predates the language feature. Its >> final, >> it isn't serializable or defines a serialization delegates, all its >> fields >> are final, the class is immutable. Looks like a great candidate to be >> converted into a value type and the preconditions seem to apply. > > This what we call a /value-based/ class. > > The question is, I believe, will it possible to (compatibly) migrate a > class that was a value-based reference class into a value type. We > certainly hope so, but we don't yet have a story for this, as I > mentioned the other day. > > As John pointed out, we're pretty comfortable with the consequences > for synchronization (both the 310 classes an Optional refer to the > value-based disclaimer, which says "do this and you'll be sorry.") > The hard part about this migration compatibility is: nullability. > (This is where someone repeats the usual commercial for null-safe > types and/or elvii operators.) While null truly makes no sense for > Optional, and I could imagine being willing to hose code that relies > on Optionals being null, this is probably the exception among known > value-based classes (e.g., LocalDate). So while we may have a > migration story for libraries that dispense and receive > formerly-value-based value types, we don't have a story for migrating > client code that might expect nullability for those types. Perhaps to accommodate gradual (opt-in for uses not for declarations) migration path, denoting the name of a class representing value type could still mean the "box" of it and denoting the actual value type would use special syntax. Regards, Peter > > (As to the timing of boxing and unboxing, it would be exactly as > primitives today.) From peter.levart at gmail.com Tue Jan 6 08:21:01 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 06 Jan 2015 09:21:01 +0100 Subject: Question on layer/peeling In-Reply-To: <54AB4732.6080207@oracle.com> References: <54AB4732.6080207@oracle.com> Message-ID: <54AB9AED.6060609@gmail.com> This could be a temporary library trick: public class Default { private T value; private Default() {} public static T value() { return new Default().value; } } with use like: public class Test { public static void main(String[] args) { int i = Default.value(); long l = Default.value(); Object o = Default.value(); } } ...but unfortunately the above Test produces the following runtime exception: Specializing method util/Default$value${0=I}.value()Ljava/lang/Object; with class=[] and method=[I] Specializing util.Default${0=I}; searching for util/Default.class (not found) Specializing util.Default${0=I}; searching for util/Default.class (found) Exception in thread "main" java.lang.BootstrapMethodError: call site initialization exception at java.lang.invoke.CallSite.makeSite(CallSite.java:341) at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307) at java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297) at util.Test.main(Test.java:8) Caused by: java.lang.VerifyError: Bad type on operand stack Exception Details: Location: util/Default$value${0=I}.value()I @7: getfield Reason: Type 'util/Default${0=I}' (current frame, stack[0]) is not assignable to 'util/Default' Current Frame: bci: @7 flags: { } locals: { } stack: { 'util/Default${0=I}' } Bytecode: 0000000: bb00 0959 b700 0db4 0012 ac at sun.misc.Unsafe.defineAnonymousClass(Native Method) at java.lang.invoke.GenericMethodSpecializer.metafactory(GenericMethodSpecializer.java:98) at java.lang.invoke.CallSite.makeSite(CallSite.java:302) ... 3 more Seems like the specialization of static method is not entirely correct here. Regards, Peter On 01/06/2015 03:23 AM, Brian Goetz wrote: > Yes, this is pretty straightforward. In the bucket of "things that > are easy and small, so we'll ignore them until we solve the ones that > are big and difficult.") > > The hardest part is picking a syntax (please, no suggestions!) > > > On 1/5/2015 9:19 PM, Vitaly Davidovich wrote: >> C# has a default (T) keyword to allow generic code to obtain the "zero" >> value for a type param. Something like that for java would be nice. >> >> Sent from my phone >> On Jan 5, 2015 9:15 PM, "Michael Barker" wrote: >> >>> Hi, >>> >>> The SotS talks about the use of 'layer' to create an alternative >>> implementation of methods when the type of an is known to be a >>> reference type. However, the examples only show the use of the layer >>> keyword on an interface definition, where as I've encountered at >>> least one >>> case where the internal implementation needs to differentiate between a >>> reference-type and value-type based collection. The example I'm >>> thinking >>> about is the null-ing out of array elements in a collection (which is >>> obviously a no-op with a value type, but necessity with reference >>> types). Is an interface required in order to define a 'layer' or >>> could it >>> be done within a concrete class? >>> >>> E.g. is the following or something similar possible? If not, how >>> would it >>> be achieved with current spec? >>> >>> class ArrayList { >>> T[] values; >>> int position; >>> >>> void removeLast() { >>> if (position <= 0) { >>> return; >>> } >>> >>> --position; >>> clear(position); >>> } >>> >>> private void clear(int index) { >>> } >>> >>> layer { >>> private void clear(int index) { >>> values[index] = null; >>> } >>> } >>> } >>> >>> Mike. >>> From mikeb01 at gmail.com Tue Jan 6 08:39:20 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Tue, 6 Jan 2015 21:39:20 +1300 Subject: Question on layer/peeling In-Reply-To: <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> Message-ID: > > This trick is a good one, and it can go a bit farther with primitives, if > you think about the other supertypes of Integer, Double, etc., especially > Comparable. (I'd kind of like to define an interface Objectable to carry > toS/eq/hC.) > Could allowing primitives to implement interfaces like Comparable make the following implementations be possible, such that the JIT could generate specialisations for the primitive types? public class Arrays { public void sort(T[] array) { // ... implement sort } public double average(T[] array) { if (array.length == 0) { return Double.NaN; } double total = 0; int count = 0; for (T number : array) { total += number.doubleValue(); } return total / count; } } int[] vals = { ... }; Arrays.sort(vals); double avg = Arrays.average(vals); If so, I'm sure that there could be some interesting applications. Mike. From stef at epardaud.fr Tue Jan 6 09:18:10 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Tue, 06 Jan 2015 10:18:10 +0100 Subject: Fwd: Remi's comments on Any In-Reply-To: References: <54AA752F.90001@univ-mlv.fr> <54AA968E.6080100@univ-mlv.fr> <54AAB5F9.2080504@oracle.com> <54AABC8C.10205@epardaud.fr> Message-ID: <54ABA852.7080205@epardaud.fr> Thanks Ryan, that was helpful :) On 01/05/2015 07:23 PM, Ryan Schmitt wrote: > http://openjdk.java.net/projects/valhalla/ > > http://cr.openjdk.java.net/~jrose/values/values-0.html > > > On Mon, Jan 5, 2015 at 8:32 AM, St?phane ?pardaud > wrote: > > Hi, > > Can I ask where the current ideas/proposal for value types is > located? I'm curious how it integrates with the object hierarchy > and how you can differentiate one value type from another at the > VM level. I think this is relevant to the current discussion, > depending on how/if they're tagged and/or boxed. > > Thanks, cheers. > > From martijnverburg at gmail.com Tue Jan 6 09:35:30 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Tue, 6 Jan 2015 09:35:30 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: References: <54AAE765.6040402@oracle.com> <54AB25E3.1060308@oracle.com> Message-ID: Hi all, On 6 January 2015 at 00:19, Richard Warburton wrote: > Hi Brian, > > It should cover all the conversions outlined in the latest SotS. There are > > a few known things that don't work quite right yet, things like: > > > > - Nonstatic inner classes and local classes (we're close on this, > though) > > - super-calls inside generic methods > > > > But, it should be enough to write an ArrayList-like class that is generic > > in any-T. > > > > Thanks. > > To run it, just clone and build the valhalla repo (make images), and use > > that as your JDK. > > > > Unless I really need sleep (it is after midnight here) there are no links > from the project home page (http://openjdk.java.net/projects/valhalla/) to > the source repositories (http://hg.openjdk.java.net/valhalla). I imagine > this might cause confusion for casual users who don't know the mapping > between project names and repo urls and it would be nice to add it. Or even > link to http://hg.openjdk.java.net/valhalla/valhalla - since that's what > they'll need to hg clone. > To this point I'd like to submit a 'patch' to the website page - is there a source file for the web page somewhere that I can create a patch against? Cheers, Martijn > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > From peter.levart at gmail.com Tue Jan 6 09:41:27 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 06 Jan 2015 10:41:27 +0100 Subject: Question on layer/peeling In-Reply-To: <54AB9AED.6060609@gmail.com> References: <54AB4732.6080207@oracle.com> <54AB9AED.6060609@gmail.com> Message-ID: <54ABADC7.6040809@gmail.com> Hi, Here's another one: public final class Box { private T value; public Box(T value) { this.value = value; } public Box() { // leave default value } public T get() { return value; } @SuppressWarnings("unchecked") @Override public boolean equals(Object obj) { return (this == obj) || (obj != null && this.getClass() == obj.getClass() && // same specialization, right? this.get() == ((Box)obj).get()); // I think the bytecode to compare two references is not specialized here to compare two integers } } public class Test { public static void main(String[] args) { System.out.println(new Box(1).equals(new Box(1))); } } I get the following at run time: Specializing util.Box${0=I}; searching for util/Box.class (not found) Specializing util.Box${0=I}; searching for util/Box.class (found) Exception in thread "main" java.lang.VerifyError: Bad type on operand stack Exception Details: Location: util/Box${0=I}.equals(Ljava/lang/Object;)Z @31: if_acmpne Reason: Type integer (current frame, stack[1]) is not assignable to reference type Current Frame: bci: @31 flags: { } locals: { 'util/Box${0=I}', 'java/lang/Object' } stack: { integer, integer } Bytecode: 0000000: 2a2b a500 202b c600 202a b600 172b b600 0000010: 17a6 0015 2ab6 0019 2bc0 0002 b600 19a6 0000020: 0007 04a7 0004 03ac Stackmap Table: same_frame(@34) same_frame(@38) same_locals_1_stack_item_frame(@39,Integer) at util.Test.main(Test.java:8) Regards, Peter On 01/06/2015 09:21 AM, Peter Levart wrote: > This could be a temporary library trick: > > public class Default { > > private T value; > > private Default() {} > > public static T value() { > return new Default().value; > } > } > > > with use like: > > > public class Test { > public static void main(String[] args) { > int i = Default.value(); > long l = Default.value(); > Object o = Default.value(); > } > } > > > > ...but unfortunately the above Test produces the following runtime > exception: > > > Specializing method util/Default$value${0=I}.value()Ljava/lang/Object; > with class=[] and method=[I] > Specializing util.Default${0=I}; searching for util/Default.class (not > found) > Specializing util.Default${0=I}; searching for util/Default.class (found) > Exception in thread "main" java.lang.BootstrapMethodError: call site > initialization exception > at java.lang.invoke.CallSite.makeSite(CallSite.java:341) > at > java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307) > at > java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297) > at util.Test.main(Test.java:8) > Caused by: java.lang.VerifyError: Bad type on operand stack > Exception Details: > Location: > util/Default$value${0=I}.value()I @7: getfield > Reason: > Type 'util/Default${0=I}' (current frame, stack[0]) is not > assignable to 'util/Default' > Current Frame: > bci: @7 > flags: { } > locals: { } > stack: { 'util/Default${0=I}' } > Bytecode: > 0000000: bb00 0959 b700 0db4 0012 ac > > at sun.misc.Unsafe.defineAnonymousClass(Native Method) > at > java.lang.invoke.GenericMethodSpecializer.metafactory(GenericMethodSpecializer.java:98) > at java.lang.invoke.CallSite.makeSite(CallSite.java:302) > ... 3 more > > > Seems like the specialization of static method is not entirely correct > here. > > Regards, Peter > > On 01/06/2015 03:23 AM, Brian Goetz wrote: >> Yes, this is pretty straightforward. In the bucket of "things that >> are easy and small, so we'll ignore them until we solve the ones that >> are big and difficult.") >> >> The hardest part is picking a syntax (please, no suggestions!) >> >> >> On 1/5/2015 9:19 PM, Vitaly Davidovich wrote: >>> C# has a default (T) keyword to allow generic code to obtain the "zero" >>> value for a type param. Something like that for java would be nice. >>> >>> Sent from my phone >>> On Jan 5, 2015 9:15 PM, "Michael Barker" wrote: >>> >>>> Hi, >>>> >>>> The SotS talks about the use of 'layer' to create an alternative >>>> implementation of methods when the type of an is known to be a >>>> reference type. However, the examples only show the use of the layer >>>> keyword on an interface definition, where as I've encountered at >>>> least one >>>> case where the internal implementation needs to differentiate >>>> between a >>>> reference-type and value-type based collection. The example I'm >>>> thinking >>>> about is the null-ing out of array elements in a collection (which is >>>> obviously a no-op with a value type, but necessity with reference >>>> types). Is an interface required in order to define a 'layer' or >>>> could it >>>> be done within a concrete class? >>>> >>>> E.g. is the following or something similar possible? If not, how >>>> would it >>>> be achieved with current spec? >>>> >>>> class ArrayList { >>>> T[] values; >>>> int position; >>>> >>>> void removeLast() { >>>> if (position <= 0) { >>>> return; >>>> } >>>> >>>> --position; >>>> clear(position); >>>> } >>>> >>>> private void clear(int index) { >>>> } >>>> >>>> layer { >>>> private void clear(int index) { >>>> values[index] = null; >>>> } >>>> } >>>> } >>>> >>>> Mike. >>>> > From scolebourne at joda.org Tue Jan 6 10:11:56 2015 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 6 Jan 2015 10:11:56 +0000 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> Message-ID: On 6 January 2015 at 01:58, Vitaly Davidovich wrote: > Mutable structs aren't used all that often in c# but there are times when > they're very useful. When I've thought about this, the first use case for mutable I came up with was property objects. These allow code like: class Person { public final Property surname; } String surname = person.surname.get(); person.surname.set("MySurname"); Property p = person.surname; Here, Property is a mutable wrapper around the surname, hence the get() and set(). Today, this design is only suitable for the client side, as creating all the wrapper objects creates far too much garbage and churn for a server (tried it, was a car crash). However, despite the appeal of making this work via value types, I'm willing to agree that allowing mutability would have far too many otehr negative consequences. Thus I agree with the "only immutable" design direction. Stephen From maurizio.cimadamore at oracle.com Tue Jan 6 10:57:39 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 06 Jan 2015 10:57:39 +0000 Subject: Question on layer/peeling In-Reply-To: <54ABADC7.6040809@gmail.com> References: <54AB4732.6080207@oracle.com> <54AB9AED.6060609@gmail.com> <54ABADC7.6040809@gmail.com> Message-ID: <54ABBFA3.5020706@oracle.com> Hi Peter, something seems to be broken in the compiler w.r.t. compound conditions - this small test case: class Box { public boolean test(T t, Object obj) { return obj != null && t == t; } } Shows that there's a missing BMA attribute for the IF_ACMPNE: public boolean test(T, java.lang.Object); Code: stack=2, locals=3, args_size=3 0: aload_2 1: ifnull 13 4: aload_1 5: aload_1 6: if_acmpne 13 9: iconst_1 10: goto 14 13: iconst_0 14: ireturn BytecodeMapping: Code_idx Signature 4: TT; 5: TT; That's why the specializer fails. I'll look into this. Maurizio On 06/01/15 09:41, Peter Levart wrote: > Hi, > > Here's another one: > > > public final class Box { > > private T value; > > public Box(T value) { > this.value = value; > } > > public Box() { > // leave default value > } > > public T get() { > return value; > } > > @SuppressWarnings("unchecked") > @Override > public boolean equals(Object obj) { > return (this == obj) || > (obj != null && > this.getClass() == obj.getClass() && // same > specialization, right? > this.get() == ((Box)obj).get()); > // I think the bytecode to compare two references is > not specialized here to compare two integers > } > } > > > public class Test { > public static void main(String[] args) { > System.out.println(new Box(1).equals(new Box(1))); > } > } > > > I get the following at run time: > > > Specializing util.Box${0=I}; searching for util/Box.class (not found) > Specializing util.Box${0=I}; searching for util/Box.class (found) > Exception in thread "main" java.lang.VerifyError: Bad type on operand > stack > Exception Details: > Location: > util/Box${0=I}.equals(Ljava/lang/Object;)Z @31: if_acmpne > Reason: > Type integer (current frame, stack[1]) is not assignable to > reference type > Current Frame: > bci: @31 > flags: { } > locals: { 'util/Box${0=I}', 'java/lang/Object' } > stack: { integer, integer } > Bytecode: > 0000000: 2a2b a500 202b c600 202a b600 172b b600 > 0000010: 17a6 0015 2ab6 0019 2bc0 0002 b600 19a6 > 0000020: 0007 04a7 0004 03ac > Stackmap Table: > same_frame(@34) > same_frame(@38) > same_locals_1_stack_item_frame(@39,Integer) > > at util.Test.main(Test.java:8) > > > Regards, Peter > > On 01/06/2015 09:21 AM, Peter Levart wrote: >> This could be a temporary library trick: >> >> public class Default { >> >> private T value; >> >> private Default() {} >> >> public static T value() { >> return new Default().value; >> } >> } >> >> >> with use like: >> >> >> public class Test { >> public static void main(String[] args) { >> int i = Default.value(); >> long l = Default.value(); >> Object o = Default.value(); >> } >> } >> >> >> >> ...but unfortunately the above Test produces the following runtime >> exception: >> >> >> Specializing method >> util/Default$value${0=I}.value()Ljava/lang/Object; with class=[] and >> method=[I] >> Specializing util.Default${0=I}; searching for util/Default.class >> (not found) >> Specializing util.Default${0=I}; searching for util/Default.class >> (found) >> Exception in thread "main" java.lang.BootstrapMethodError: call site >> initialization exception >> at java.lang.invoke.CallSite.makeSite(CallSite.java:341) >> at >> java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307) >> at >> java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297) >> at util.Test.main(Test.java:8) >> Caused by: java.lang.VerifyError: Bad type on operand stack >> Exception Details: >> Location: >> util/Default$value${0=I}.value()I @7: getfield >> Reason: >> Type 'util/Default${0=I}' (current frame, stack[0]) is not >> assignable to 'util/Default' >> Current Frame: >> bci: @7 >> flags: { } >> locals: { } >> stack: { 'util/Default${0=I}' } >> Bytecode: >> 0000000: bb00 0959 b700 0db4 0012 ac >> >> at sun.misc.Unsafe.defineAnonymousClass(Native Method) >> at >> java.lang.invoke.GenericMethodSpecializer.metafactory(GenericMethodSpecializer.java:98) >> at java.lang.invoke.CallSite.makeSite(CallSite.java:302) >> ... 3 more >> >> >> Seems like the specialization of static method is not entirely >> correct here. >> >> Regards, Peter >> >> On 01/06/2015 03:23 AM, Brian Goetz wrote: >>> Yes, this is pretty straightforward. In the bucket of "things that >>> are easy and small, so we'll ignore them until we solve the ones >>> that are big and difficult.") >>> >>> The hardest part is picking a syntax (please, no suggestions!) >>> >>> >>> On 1/5/2015 9:19 PM, Vitaly Davidovich wrote: >>>> C# has a default (T) keyword to allow generic code to obtain the >>>> "zero" >>>> value for a type param. Something like that for java would be nice. >>>> >>>> Sent from my phone >>>> On Jan 5, 2015 9:15 PM, "Michael Barker" wrote: >>>> >>>>> Hi, >>>>> >>>>> The SotS talks about the use of 'layer' to create an alternative >>>>> implementation of methods when the type of an is known to >>>>> be a >>>>> reference type. However, the examples only show the use of the layer >>>>> keyword on an interface definition, where as I've encountered at >>>>> least one >>>>> case where the internal implementation needs to differentiate >>>>> between a >>>>> reference-type and value-type based collection. The example I'm >>>>> thinking >>>>> about is the null-ing out of array elements in a collection (which is >>>>> obviously a no-op with a value type, but necessity with reference >>>>> types). Is an interface required in order to define a 'layer' or >>>>> could it >>>>> be done within a concrete class? >>>>> >>>>> E.g. is the following or something similar possible? If not, how >>>>> would it >>>>> be achieved with current spec? >>>>> >>>>> class ArrayList { >>>>> T[] values; >>>>> int position; >>>>> >>>>> void removeLast() { >>>>> if (position <= 0) { >>>>> return; >>>>> } >>>>> >>>>> --position; >>>>> clear(position); >>>>> } >>>>> >>>>> private void clear(int index) { >>>>> } >>>>> >>>>> layer { >>>>> private void clear(int index) { >>>>> values[index] = null; >>>>> } >>>>> } >>>>> } >>>>> >>>>> Mike. >>>>> >> > From maurizio.cimadamore at oracle.com Tue Jan 6 11:05:18 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 06 Jan 2015 11:05:18 +0000 Subject: Question on layer/peeling In-Reply-To: <54AB9AED.6060609@gmail.com> References: <54AB4732.6080207@oracle.com> <54AB9AED.6060609@gmail.com> Message-ID: <54ABC16E.5020207@oracle.com> Looks like another compiler issue; here's the bytecode for Default.value: public static T value(); Code: stack=2, locals=0, args_size=0 0: new #2 // class Default 3: dup 4: invokespecial #3 // Method "":()V 7: getfield #4 // Field value:Ljava/lang/Object; 10: areturn BytecodeMapping: Code_idx Signature 0: LDefault; 4: LDefault;::()V 10: TT; As you can see, the GETFIELD is missing the BytecodeMapping entry. I suspect the 'dup' is causing javac to lose the type info associated with the stack operand. I'll look into this. Maurizio On 06/01/15 08:21, Peter Levart wrote: > This could be a temporary library trick: > > public class Default { > > private T value; > > private Default() {} > > public static T value() { > return new Default().value; > } > } > > > with use like: > > > public class Test { > public static void main(String[] args) { > int i = Default.value(); > long l = Default.value(); > Object o = Default.value(); > } > } > > > > ...but unfortunately the above Test produces the following runtime > exception: > > > Specializing method util/Default$value${0=I}.value()Ljava/lang/Object; > with class=[] and method=[I] > Specializing util.Default${0=I}; searching for util/Default.class (not > found) > Specializing util.Default${0=I}; searching for util/Default.class (found) > Exception in thread "main" java.lang.BootstrapMethodError: call site > initialization exception > at java.lang.invoke.CallSite.makeSite(CallSite.java:341) > at > java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307) > at > java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297) > at util.Test.main(Test.java:8) > Caused by: java.lang.VerifyError: Bad type on operand stack > Exception Details: > Location: > util/Default$value${0=I}.value()I @7: getfield > Reason: > Type 'util/Default${0=I}' (current frame, stack[0]) is not > assignable to 'util/Default' > Current Frame: > bci: @7 > flags: { } > locals: { } > stack: { 'util/Default${0=I}' } > Bytecode: > 0000000: bb00 0959 b700 0db4 0012 ac > > at sun.misc.Unsafe.defineAnonymousClass(Native Method) > at > java.lang.invoke.GenericMethodSpecializer.metafactory(GenericMethodSpecializer.java:98) > at java.lang.invoke.CallSite.makeSite(CallSite.java:302) > ... 3 more > > > Seems like the specialization of static method is not entirely correct > here. > > Regards, Peter > > On 01/06/2015 03:23 AM, Brian Goetz wrote: >> Yes, this is pretty straightforward. In the bucket of "things that >> are easy and small, so we'll ignore them until we solve the ones that >> are big and difficult.") >> >> The hardest part is picking a syntax (please, no suggestions!) >> >> >> On 1/5/2015 9:19 PM, Vitaly Davidovich wrote: >>> C# has a default (T) keyword to allow generic code to obtain the "zero" >>> value for a type param. Something like that for java would be nice. >>> >>> Sent from my phone >>> On Jan 5, 2015 9:15 PM, "Michael Barker" wrote: >>> >>>> Hi, >>>> >>>> The SotS talks about the use of 'layer' to create an alternative >>>> implementation of methods when the type of an is known to be a >>>> reference type. However, the examples only show the use of the layer >>>> keyword on an interface definition, where as I've encountered at >>>> least one >>>> case where the internal implementation needs to differentiate >>>> between a >>>> reference-type and value-type based collection. The example I'm >>>> thinking >>>> about is the null-ing out of array elements in a collection (which is >>>> obviously a no-op with a value type, but necessity with reference >>>> types). Is an interface required in order to define a 'layer' or >>>> could it >>>> be done within a concrete class? >>>> >>>> E.g. is the following or something similar possible? If not, how >>>> would it >>>> be achieved with current spec? >>>> >>>> class ArrayList { >>>> T[] values; >>>> int position; >>>> >>>> void removeLast() { >>>> if (position <= 0) { >>>> return; >>>> } >>>> >>>> --position; >>>> clear(position); >>>> } >>>> >>>> private void clear(int index) { >>>> } >>>> >>>> layer { >>>> private void clear(int index) { >>>> values[index] = null; >>>> } >>>> } >>>> } >>>> >>>> Mike. >>>> > From maurizio.cimadamore at oracle.com Tue Jan 6 11:15:11 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 06 Jan 2015 11:15:11 +0000 Subject: Question on layer/peeling In-Reply-To: <54ABC16E.5020207@oracle.com> References: <54AB4732.6080207@oracle.com> <54AB9AED.6060609@gmail.com> <54ABC16E.5020207@oracle.com> Message-ID: <54ABC3BF.9090800@oracle.com> Btw, accessibility is all over the place in the current compiler/specializer, so you might want to avoid using private, for now. Maurizio On 06/01/15 11:05, Maurizio Cimadamore wrote: > Looks like another compiler issue; here's the bytecode for Default.value: > > public static T value(); > Code: > stack=2, locals=0, args_size=0 > 0: new #2 // class Default > 3: dup > 4: invokespecial #3 // Method "":()V > 7: getfield #4 // Field > value:Ljava/lang/Object; > 10: areturn > BytecodeMapping: > Code_idx Signature > 0: LDefault; > 4: LDefault;::()V > 10: TT; > > As you can see, the GETFIELD is missing the BytecodeMapping entry. I > suspect the 'dup' is causing javac to lose the type info associated > with the stack operand. I'll look into this. > > Maurizio > > On 06/01/15 08:21, Peter Levart wrote: >> This could be a temporary library trick: >> >> public class Default { >> >> private T value; >> >> private Default() {} >> >> public static T value() { >> return new Default().value; >> } >> } >> >> >> with use like: >> >> >> public class Test { >> public static void main(String[] args) { >> int i = Default.value(); >> long l = Default.value(); >> Object o = Default.value(); >> } >> } >> >> >> >> ...but unfortunately the above Test produces the following runtime >> exception: >> >> >> Specializing method >> util/Default$value${0=I}.value()Ljava/lang/Object; with class=[] and >> method=[I] >> Specializing util.Default${0=I}; searching for util/Default.class >> (not found) >> Specializing util.Default${0=I}; searching for util/Default.class >> (found) >> Exception in thread "main" java.lang.BootstrapMethodError: call site >> initialization exception >> at java.lang.invoke.CallSite.makeSite(CallSite.java:341) >> at >> java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307) >> at >> java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297) >> at util.Test.main(Test.java:8) >> Caused by: java.lang.VerifyError: Bad type on operand stack >> Exception Details: >> Location: >> util/Default$value${0=I}.value()I @7: getfield >> Reason: >> Type 'util/Default${0=I}' (current frame, stack[0]) is not >> assignable to 'util/Default' >> Current Frame: >> bci: @7 >> flags: { } >> locals: { } >> stack: { 'util/Default${0=I}' } >> Bytecode: >> 0000000: bb00 0959 b700 0db4 0012 ac >> >> at sun.misc.Unsafe.defineAnonymousClass(Native Method) >> at >> java.lang.invoke.GenericMethodSpecializer.metafactory(GenericMethodSpecializer.java:98) >> at java.lang.invoke.CallSite.makeSite(CallSite.java:302) >> ... 3 more >> >> >> Seems like the specialization of static method is not entirely >> correct here. >> >> Regards, Peter >> >> On 01/06/2015 03:23 AM, Brian Goetz wrote: >>> Yes, this is pretty straightforward. In the bucket of "things that >>> are easy and small, so we'll ignore them until we solve the ones >>> that are big and difficult.") >>> >>> The hardest part is picking a syntax (please, no suggestions!) >>> >>> >>> On 1/5/2015 9:19 PM, Vitaly Davidovich wrote: >>>> C# has a default (T) keyword to allow generic code to obtain the >>>> "zero" >>>> value for a type param. Something like that for java would be nice. >>>> >>>> Sent from my phone >>>> On Jan 5, 2015 9:15 PM, "Michael Barker" wrote: >>>> >>>>> Hi, >>>>> >>>>> The SotS talks about the use of 'layer' to create an alternative >>>>> implementation of methods when the type of an is known to >>>>> be a >>>>> reference type. However, the examples only show the use of the layer >>>>> keyword on an interface definition, where as I've encountered at >>>>> least one >>>>> case where the internal implementation needs to differentiate >>>>> between a >>>>> reference-type and value-type based collection. The example I'm >>>>> thinking >>>>> about is the null-ing out of array elements in a collection (which is >>>>> obviously a no-op with a value type, but necessity with reference >>>>> types). Is an interface required in order to define a 'layer' or >>>>> could it >>>>> be done within a concrete class? >>>>> >>>>> E.g. is the following or something similar possible? If not, how >>>>> would it >>>>> be achieved with current spec? >>>>> >>>>> class ArrayList { >>>>> T[] values; >>>>> int position; >>>>> >>>>> void removeLast() { >>>>> if (position <= 0) { >>>>> return; >>>>> } >>>>> >>>>> --position; >>>>> clear(position); >>>>> } >>>>> >>>>> private void clear(int index) { >>>>> } >>>>> >>>>> layer { >>>>> private void clear(int index) { >>>>> values[index] = null; >>>>> } >>>>> } >>>>> } >>>>> >>>>> Mike. >>>>> >> > From maurizio.cimadamore at oracle.com Tue Jan 6 13:00:52 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 06 Jan 2015 13:00:52 +0000 Subject: hg: valhalla/valhalla/langtools: BytecodeMapping generation fixes: Message-ID: <201501061300.t06D0q5V024359@aojmv0008> Changeset: 8331682af2ed Author: mcimadamore Date: 2015-01-06 12:54 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/8331682af2ed BytecodeMapping generation fixes: * missing BMA for eq/ne any comparisons nested within logic operators &&/|| * missing BMA for field/method access immediately following a constructor call * refactor Items code in order to avoid cast to AnyItem in order to access type info associated with a given item * added tests ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java + test/tools/javac/valhalla/typespec/items/tests/TestBinary.java ! test/tools/javac/valhalla/typespec/items/tests/TestInstanceof.java ! test/tools/javac/valhalla/typespec/items/tests/TestNew.java From vitalyd at gmail.com Tue Jan 6 13:33:41 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 6 Jan 2015 08:33:41 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> Message-ID: What are the "far too many" downsides? The lost writes can happen if not careful, but what else? I'm assuming though if mutable value types were possible, then language and VM would allow passing them by ref. Sent from my phone On Jan 6, 2015 5:12 AM, "Stephen Colebourne" wrote: > On 6 January 2015 at 01:58, Vitaly Davidovich wrote: > > Mutable structs aren't used all that often in c# but there are times when > > they're very useful. > > When I've thought about this, the first use case for mutable I came up > with was property objects. These allow code like: > > class Person { > public final Property surname; > } > String surname = person.surname.get(); > person.surname.set("MySurname"); > Property p = person.surname; > > Here, Property is a mutable wrapper around the surname, hence the > get() and set(). Today, this design is only suitable for the client > side, as creating all the wrapper objects creates far too much garbage > and churn for a server (tried it, was a car crash). > > However, despite the appeal of making this work via value types, I'm > willing to agree that allowing mutability would have far too many > otehr negative consequences. Thus I agree with the "only immutable" > design direction. > > Stephen > From sven.reimers at gmail.com Tue Jan 6 15:13:46 2015 From: sven.reimers at gmail.com (Sven Reimers) Date: Tue, 6 Jan 2015 16:13:46 +0100 Subject: java.lang.VerifyError: Bad type on operand stack Message-ID: Hi, as suggested by Brian I tried to get some sample code written and running on top of valhalla. First problem I encountered was that a missing diamond <> on the right hand side (my bad - seems I am really relying on my IDE) was just a warning that I was using a raw type. Running my example I got this: Specializing Valhalla$Point3D${0=I}; searching for Valhalla$Point3D.class (not found) Specializing Valhalla$Point3D${0=I}; searching for Valhalla$Point3D.class (found) Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.VerifyError: Bad type on operand stack Exception Details: Location: Valhalla.main([Ljava/lang/String;)V @36: invokevirtual Reason: Type 'Valhalla$Point3D' (current frame, stack[2]) is not assignable to 'Valhalla$Point3D${0=I}' Current Frame: bci: @36 flags: { } locals: { '[Ljava/lang/String;', 'Valhalla$Point3D' } stack: { 'java/io/PrintStream', 'java/lang/StringBuilder', 'Valhalla$Point3D' } Bytecode: 0000000: bb00 0259 04b8 0003 05b8 0003 06b8 0003 0000010: b700 044c b200 05bb 0006 59b7 0007 1208 0000020: b600 092b b600 0ab6 000b b600 0cb6 000d 0000030: b1 at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2704) at java.lang.Class.privateGetMethodRecursive(Class.java:3049) at java.lang.Class.getMethod0(Class.java:3019) at java.lang.Class.getMethod(Class.java:1787) at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:568) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:535) Example Code (see the missing <> just after Point3D...): public class Valhalla { public static void main (String[] args) { Point3D intPoint = new Point3D(1,2,3); System.out.println("X: " + intPoint.getX()); } public static class Point3D { private T x; private T y; private T z; public Point3D(T x, T y, T z) { this.x = x; this.y = y; this.z = z; } public T getX() { return x; } public T getY() { return y; } public T getZ() { return z; } } } Hope to provide more feedback once I got more code running. Thanks for the impressive work so far. -Sven -- Sven Reimers * Senior Expert Software Architect * NetBeans Dream Team Member: http://dreamteam.netbeans.org * Community Leader NetBeans: http://community.java.net/netbeans Desktop Java: http://community.java.net/javadesktop * JUG Leader JUG Bodensee: http://www.jug-bodensee.de * Duke's Choice Award Winner 2009 * Blog: https://www.java.net//blog/sven * XING: https://www.xing.com/profile/Sven_Reimers8 * LinkedIn: http://www.linkedin.com/in/svenreimers Join the NetBeans Groups: * XING: http://www.xing.com/group-20148.82db20 * NUGM: http://haug-server.dyndns.org/display/NUGM/Home * LinkedIn: http://www.linkedin.com/groups?gid=1860468 http://www.linkedin.com/groups?gid=107402 http://www.linkedin.com/groups?gid=1684717 * Oracle: https://mix.oracle.com/groups/18497 From brian.goetz at oracle.com Tue Jan 6 15:13:38 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 10:13:38 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> Message-ID: <54ABFBA2.4090708@oracle.com> The C# folks concluded in hindsight that mutable structs was a big mistake. See Eric Lippert's blog: http://blogs.msdn.com/b/ericlippert/archive/2011/03/14/to-box-or-not-to-box-that-is-the-question.aspx TL;DR version (last line): "Once again the moral of the story is: mutable value types are enough pure evil to turn you all into hermit crabs, and therefore should be avoided." On 1/6/2015 8:33 AM, Vitaly Davidovich wrote: > What are the "far too many" downsides? The lost writes can happen if not > careful, but what else? I'm assuming though if mutable value types were > possible, then language and VM would allow passing them by ref. > > Sent from my phone > On Jan 6, 2015 5:12 AM, "Stephen Colebourne" wrote: > >> On 6 January 2015 at 01:58, Vitaly Davidovich wrote: >>> Mutable structs aren't used all that often in c# but there are times when >>> they're very useful. >> >> When I've thought about this, the first use case for mutable I came up >> with was property objects. These allow code like: >> >> class Person { >> public final Property surname; >> } >> String surname = person.surname.get(); >> person.surname.set("MySurname"); >> Property p = person.surname; >> >> Here, Property is a mutable wrapper around the surname, hence the >> get() and set(). Today, this design is only suitable for the client >> side, as creating all the wrapper objects creates far too much garbage >> and churn for a server (tried it, was a car crash). >> >> However, despite the appeal of making this work via value types, I'm >> willing to agree that allowing mutability would have far too many >> otehr negative consequences. Thus I agree with the "only immutable" >> design direction. >> >> Stephen >> From maurizio.cimadamore at oracle.com Tue Jan 6 15:10:00 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 06 Jan 2015 15:10:00 +0000 Subject: java.lang.VerifyError: Bad type on operand stack In-Reply-To: References: Message-ID: <54ABFAC8.5000308@oracle.com> Whoops :-) Thanks for the report Maurizio On 06/01/15 15:13, Sven Reimers wrote: > Hi, > > as suggested by Brian I tried to get some sample code written and running > on top of valhalla. > > First problem I encountered was that a missing diamond <> on the right hand > side (my bad - seems I am really relying on my IDE) was just a warning that > I was using a raw type. > > Running my example I got this: > > Specializing Valhalla$Point3D${0=I}; searching for Valhalla$Point3D.class > (not found) > Specializing Valhalla$Point3D${0=I}; searching for Valhalla$Point3D.class > (found) > Error: A JNI error has occurred, please check your installation and try > again > Exception in thread "main" java.lang.VerifyError: Bad type on operand stack > Exception Details: > Location: > Valhalla.main([Ljava/lang/String;)V @36: invokevirtual > Reason: > Type 'Valhalla$Point3D' (current frame, stack[2]) is not assignable to > 'Valhalla$Point3D${0=I}' > Current Frame: > bci: @36 > flags: { } > locals: { '[Ljava/lang/String;', 'Valhalla$Point3D' } > stack: { 'java/io/PrintStream', 'java/lang/StringBuilder', > 'Valhalla$Point3D' } > Bytecode: > 0000000: bb00 0259 04b8 0003 05b8 0003 06b8 0003 > 0000010: b700 044c b200 05bb 0006 59b7 0007 1208 > 0000020: b600 092b b600 0ab6 000b b600 0cb6 000d > 0000030: b1 > > at java.lang.Class.getDeclaredMethods0(Native Method) > at java.lang.Class.privateGetDeclaredMethods(Class.java:2704) > at java.lang.Class.privateGetMethodRecursive(Class.java:3049) > at java.lang.Class.getMethod0(Class.java:3019) > at java.lang.Class.getMethod(Class.java:1787) > at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:568) > at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:535) > > Example Code (see the missing <> just after Point3D...): > > public class Valhalla { > > > public static void main (String[] args) { > > Point3D intPoint = new Point3D(1,2,3); > > System.out.println("X: " + intPoint.getX()); > > } > > public static class Point3D { > private T x; > private T y; > private T z; > > public Point3D(T x, T y, T z) { > this.x = x; > this.y = y; > this.z = z; > } > > public T getX() { > return x; > } > > public T getY() { > return y; > } > > public T getZ() { > return z; > } > > } > > } > > Hope to provide more feedback once I got more code running. > > Thanks for the impressive work so far. > > -Sven > From vitalyd at gmail.com Tue Jan 6 15:24:52 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 6 Jan 2015 10:24:52 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: <54ABFBA2.4090708@oracle.com> References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> <54ABFBA2.4090708@oracle.com> Message-ID: I've seen and read that blog before. I'm not disputing that one needs to be careful in how C# mutable structs are used and as mentioned, it's a rare (but useful) use case. But really, the issue in that blog is more about how compiler desugars/transforms C#'s using block and its poor interaction with value types when someone doesn't fully understand the sugaring mechanics. I suspect the problem Eric (when he was still at MSFT) had was developers not taking the time to fully understand the tool. Yes, it's error prone if you're sleep walking through code, but then again, don't sleep walk through code! So, I'll agree that their use should be restricted, but it's nice to have when needed (akin to using com.sun.misc.Unsafe -- it's a sharp knife that can inflict serious damage if not wielded carefully, but very handy/surgical when called for and used carefully). On Tue, Jan 6, 2015 at 10:13 AM, Brian Goetz wrote: > The C# folks concluded in hindsight that mutable structs was a big > mistake. See Eric Lippert's blog: > > http://blogs.msdn.com/b/ericlippert/archive/2011/03/ > 14/to-box-or-not-to-box-that-is-the-question.aspx > > TL;DR version (last line): "Once again the moral of the story is: mutable > value types are enough pure evil to turn you all into hermit crabs, and > therefore should be avoided." > > > > > > On 1/6/2015 8:33 AM, Vitaly Davidovich wrote: > >> What are the "far too many" downsides? The lost writes can happen if not >> careful, but what else? I'm assuming though if mutable value types were >> possible, then language and VM would allow passing them by ref. >> >> Sent from my phone >> On Jan 6, 2015 5:12 AM, "Stephen Colebourne" >> wrote: >> >> On 6 January 2015 at 01:58, Vitaly Davidovich wrote: >>> >>>> Mutable structs aren't used all that often in c# but there are times >>>> when >>>> they're very useful. >>>> >>> >>> When I've thought about this, the first use case for mutable I came up >>> with was property objects. These allow code like: >>> >>> class Person { >>> public final Property surname; >>> } >>> String surname = person.surname.get(); >>> person.surname.set("MySurname"); >>> Property p = person.surname; >>> >>> Here, Property is a mutable wrapper around the surname, hence the >>> get() and set(). Today, this design is only suitable for the client >>> side, as creating all the wrapper objects creates far too much garbage >>> and churn for a server (tried it, was a car crash). >>> >>> However, despite the appeal of making this work via value types, I'm >>> willing to agree that allowing mutability would have far too many >>> otehr negative consequences. Thus I agree with the "only immutable" >>> design direction. >>> >>> Stephen >>> >>> From brian.goetz at oracle.com Tue Jan 6 16:21:26 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 11:21:26 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> <54ABFBA2.4090708@oracle.com> Message-ID: <54AC0B86.1030701@oracle.com> If your comparison is that "mutable structs are like unsafe", great -- that's a bulletproof argument that value types must not be mutable, and there is no need for further discussion on the topic :) (Pause while Brian gets out his chisel to carve it in stone....done.) On 1/6/2015 10:24 AM, Vitaly Davidovich wrote: > I've seen and read that blog before. I'm not disputing that one needs > to be careful in how C# mutable structs are used and as mentioned, it's > a rare (but useful) use case. But really, the issue in that blog is > more about how compiler desugars/transforms C#'s using block and its > poor interaction with value types when someone doesn't fully understand > the sugaring mechanics. I suspect the problem Eric (when he was still > at MSFT) had was developers not taking the time to fully understand the > tool. Yes, it's error prone if you're sleep walking through code, but > then again, don't sleep walk through code! > > So, I'll agree that their use should be restricted, but it's nice to > have when needed (akin to using com.sun.misc.Unsafe -- it's a sharp > knife that can inflict serious damage if not wielded carefully, but very > handy/surgical when called for and used carefully). > > On Tue, Jan 6, 2015 at 10:13 AM, Brian Goetz > wrote: > > The C# folks concluded in hindsight that mutable structs was a big > mistake. See Eric Lippert's blog: > > http://blogs.msdn.com/b/__ericlippert/archive/2011/03/__14/to-box-or-not-to-box-that-__is-the-question.aspx > > > TL;DR version (last line): "Once again the moral of the story is: > mutable value types are enough pure evil to turn you all into hermit > crabs, and therefore should be avoided." > > > > > > On 1/6/2015 8:33 AM, Vitaly Davidovich wrote: > > What are the "far too many" downsides? The lost writes can > happen if not > careful, but what else? I'm assuming though if mutable value > types were > possible, then language and VM would allow passing them by ref. > > Sent from my phone > On Jan 6, 2015 5:12 AM, "Stephen Colebourne" > > wrote: > > On 6 January 2015 at 01:58, Vitaly Davidovich > > wrote: > > Mutable structs aren't used all that often in c# but > there are times when > they're very useful. > > > When I've thought about this, the first use case for mutable > I came up > with was property objects. These allow code like: > > class Person { > public final Property surname; > } > String surname = person.surname.get(); > person.surname.set("MySurname"__); > Property p = person.surname; > > Here, Property is a mutable wrapper around the surname, > hence the > get() and set(). Today, this design is only suitable for the > client > side, as creating all the wrapper objects creates far too > much garbage > and churn for a server (tried it, was a car crash). > > However, despite the appeal of making this work via value > types, I'm > willing to agree that allowing mutability would have far too > many > otehr negative consequences. Thus I agree with the "only > immutable" > design direction. > > Stephen > > From sven.reimers at gmail.com Tue Jan 6 16:27:26 2015 From: sven.reimers at gmail.com (Sven Reimers) Date: Tue, 6 Jan 2015 17:27:26 +0100 Subject: Fwd: java.lang.VerifyError: Bad type on operand stack In-Reply-To: References: <54ABFAC8.5000308@oracle.com> Message-ID: @Maurizio: sorry forgot the mailing list... ---------- Forwarded message ---------- From: Sven Reimers Date: Tue, Jan 6, 2015 at 5:20 PM Subject: Re: java.lang.VerifyError: Bad type on operand stack To: Maurizio Cimadamore Follow-Up: I tried a few different things and got into this: Point3D integerPoint = new Point3D<>(1,2,3); compiles only if public static class Point3D {... } not for public static class Point3D {... } ErrorMessage from the compiler is: incompatible types: cannot infer type arguments for Point3D<> Point3D integerPoint = new Point3D<>(1,2,3); ^ reason: cannot infer type-variable(s) T (argument mismatch; int cannot be converted to Integer) where T is a type-variable: T extends declared in class Point3D Is this intentional (not sure I found this stated in the draft)? Thanks Sven On Tue, Jan 6, 2015 at 4:10 PM, Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > Whoops :-) > > Thanks for the report > > Maurizio > > > On 06/01/15 15:13, Sven Reimers wrote: > >> Hi, >> >> as suggested by Brian I tried to get some sample code written and running >> on top of valhalla. >> >> First problem I encountered was that a missing diamond <> on the right >> hand >> side (my bad - seems I am really relying on my IDE) was just a warning >> that >> I was using a raw type. >> >> Running my example I got this: >> >> Specializing Valhalla$Point3D${0=I}; searching for Valhalla$Point3D.class >> (not found) >> Specializing Valhalla$Point3D${0=I}; searching for Valhalla$Point3D.class >> (found) >> Error: A JNI error has occurred, please check your installation and try >> again >> Exception in thread "main" java.lang.VerifyError: Bad type on operand >> stack >> Exception Details: >> Location: >> Valhalla.main([Ljava/lang/String;)V @36: invokevirtual >> Reason: >> Type 'Valhalla$Point3D' (current frame, stack[2]) is not assignable >> to >> 'Valhalla$Point3D${0=I}' >> Current Frame: >> bci: @36 >> flags: { } >> locals: { '[Ljava/lang/String;', 'Valhalla$Point3D' } >> stack: { 'java/io/PrintStream', 'java/lang/StringBuilder', >> 'Valhalla$Point3D' } >> Bytecode: >> 0000000: bb00 0259 04b8 0003 05b8 0003 06b8 0003 >> 0000010: b700 044c b200 05bb 0006 59b7 0007 1208 >> 0000020: b600 092b b600 0ab6 000b b600 0cb6 000d >> 0000030: b1 >> >> at java.lang.Class.getDeclaredMethods0(Native Method) >> at java.lang.Class.privateGetDeclaredMethods(Class.java:2704) >> at java.lang.Class.privateGetMethodRecursive(Class.java:3049) >> at java.lang.Class.getMethod0(Class.java:3019) >> at java.lang.Class.getMethod(Class.java:1787) >> at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:568) >> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:535) >> >> Example Code (see the missing <> just after Point3D...): >> >> public class Valhalla { >> >> >> public static void main (String[] args) { >> >> Point3D intPoint = new Point3D(1,2,3); >> >> System.out.println("X: " + intPoint.getX()); >> >> } >> >> public static class Point3D { >> private T x; >> private T y; >> private T z; >> >> public Point3D(T x, T y, T z) { >> this.x = x; >> this.y = y; >> this.z = z; >> } >> >> public T getX() { >> return x; >> } >> >> public T getY() { >> return y; >> } >> >> public T getZ() { >> return z; >> } >> >> } >> >> } >> >> Hope to provide more feedback once I got more code running. >> >> Thanks for the impressive work so far. >> >> -Sven >> >> > -- Sven Reimers * Senior Expert Software Architect * NetBeans Dream Team Member: http://dreamteam.netbeans.org * Community Leader NetBeans: http://community.java.net/netbeans Desktop Java: http://community.java.net/javadesktop * JUG Leader JUG Bodensee: http://www.jug-bodensee.de * Duke's Choice Award Winner 2009 * Blog: https://www.java.net//blog/sven * XING: https://www.xing.com/profile/Sven_Reimers8 * LinkedIn: http://www.linkedin.com/in/svenreimers Join the NetBeans Groups: * XING: http://www.xing.com/group-20148.82db20 * NUGM: http://haug-server.dyndns.org/display/NUGM/Home * LinkedIn: http://www.linkedin.com/groups?gid=1860468 http://www.linkedin.com/groups?gid=107402 http://www.linkedin.com/groups?gid=1684717 * Oracle: https://mix.oracle.com/groups/18497 -- Sven Reimers * Senior Expert Software Architect * NetBeans Dream Team Member: http://dreamteam.netbeans.org * Community Leader NetBeans: http://community.java.net/netbeans Desktop Java: http://community.java.net/javadesktop * JUG Leader JUG Bodensee: http://www.jug-bodensee.de * Duke's Choice Award Winner 2009 * Blog: https://www.java.net//blog/sven * XING: https://www.xing.com/profile/Sven_Reimers8 * LinkedIn: http://www.linkedin.com/in/svenreimers Join the NetBeans Groups: * XING: http://www.xing.com/group-20148.82db20 * NUGM: http://haug-server.dyndns.org/display/NUGM/Home * LinkedIn: http://www.linkedin.com/groups?gid=1860468 http://www.linkedin.com/groups?gid=107402 http://www.linkedin.com/groups?gid=1684717 * Oracle: https://mix.oracle.com/groups/18497 From vitalyd at gmail.com Tue Jan 6 16:40:50 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 6 Jan 2015 11:40:50 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: <54AC0B86.1030701@oracle.com> References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> <54ABFBA2.4090708@oracle.com> <54AC0B86.1030701@oracle.com> Message-ID: I was merely using unsafe as an example of a tool (lib, not language, in this case) that's (relatively) rarely used, but very important for some cases. If you want to provide a Sufficiently Smart Compiler that makes it obsolete, great, but that's not here presently and probably never will be (it'll be good enough for majority of cases though). One thing that really irks me (and I think some others, based on my observations) about some of the discussions around java and its direction is the sentiment that java developers require hand holding, can't be trusted, must be fully abstracted away from the machine, and thus deserve a language with *mandatory* safety belts and restraints. If java wants to stay relevant in high perf domains, that sentiment needs to be dropped; escape hatches/unsafe-ish code must remain available. Every language that I know of which markets itself as being relevant in high perf scenarios allows for having small targeted amounts of unsafe code; there's absolutely nothing wrong with that, and in fact, it's a Good Thing. As for mutable value types, showing developers the types of issues they may face is fine, direct them towards immutability, but ultimately, let them decide! (clearly, mutable value types aren't going to be implemented, I'm not going to change your mind, but I dislike the philosophy behind it). On Tue, Jan 6, 2015 at 11:21 AM, Brian Goetz wrote: > If your comparison is that "mutable structs are like unsafe", great -- > that's a bulletproof argument that value types must not be mutable, and > there is no need for further discussion on the topic :) > > (Pause while Brian gets out his chisel to carve it in stone....done.) > > > On 1/6/2015 10:24 AM, Vitaly Davidovich wrote: > >> I've seen and read that blog before. I'm not disputing that one needs >> to be careful in how C# mutable structs are used and as mentioned, it's >> a rare (but useful) use case. But really, the issue in that blog is >> more about how compiler desugars/transforms C#'s using block and its >> poor interaction with value types when someone doesn't fully understand >> the sugaring mechanics. I suspect the problem Eric (when he was still >> at MSFT) had was developers not taking the time to fully understand the >> tool. Yes, it's error prone if you're sleep walking through code, but >> then again, don't sleep walk through code! >> >> So, I'll agree that their use should be restricted, but it's nice to >> have when needed (akin to using com.sun.misc.Unsafe -- it's a sharp >> knife that can inflict serious damage if not wielded carefully, but very >> handy/surgical when called for and used carefully). >> >> On Tue, Jan 6, 2015 at 10:13 AM, Brian Goetz > > wrote: >> >> The C# folks concluded in hindsight that mutable structs was a big >> mistake. See Eric Lippert's blog: >> >> http://blogs.msdn.com/b/__ericlippert/archive/2011/03/__ >> 14/to-box-or-not-to-box-that-__is-the-question.aspx >> > 14/to-box-or-not-to-box-that-is-the-question.aspx> >> >> TL;DR version (last line): "Once again the moral of the story is: >> mutable value types are enough pure evil to turn you all into hermit >> crabs, and therefore should be avoided." >> >> >> >> >> >> On 1/6/2015 8:33 AM, Vitaly Davidovich wrote: >> >> What are the "far too many" downsides? The lost writes can >> happen if not >> careful, but what else? I'm assuming though if mutable value >> types were >> possible, then language and VM would allow passing them by ref. >> >> Sent from my phone >> On Jan 6, 2015 5:12 AM, "Stephen Colebourne" >> > wrote: >> >> On 6 January 2015 at 01:58, Vitaly Davidovich >> > wrote: >> >> Mutable structs aren't used all that often in c# but >> there are times when >> they're very useful. >> >> >> When I've thought about this, the first use case for mutable >> I came up >> with was property objects. These allow code like: >> >> class Person { >> public final Property surname; >> } >> String surname = person.surname.get(); >> person.surname.set("MySurname"__); >> Property p = person.surname; >> >> Here, Property is a mutable wrapper around the surname, >> hence the >> get() and set(). Today, this design is only suitable for the >> client >> side, as creating all the wrapper objects creates far too >> much garbage >> and churn for a server (tried it, was a car crash). >> >> However, despite the appeal of making this work via value >> types, I'm >> willing to agree that allowing mutability would have far too >> many >> otehr negative consequences. Thus I agree with the "only >> immutable" >> design direction. >> >> Stephen >> >> >> From brian.goetz at oracle.com Tue Jan 6 17:26:29 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 12:26:29 -0500 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> <54ABFBA2.4090708@oracle.com> <54AC0B86.1030701@oracle.com> Message-ID: <54AC1AC5.6030907@oracle.com> Obviously both extremes (encase everyone in concrete, vs force-feeding chainsaws to schoolchildren) are ridiculous. Which means we're in the middle, where judgement has to be used to determine a fair balance. And since its subjective, there will be differences of opinion. (And when there are differences of opinion, it is tempting to frame the "slightly more conservative than you" perspective as reactionary and the "slightly more liberal than you" perspective as communist, but that's not really constructive.) We do have escape hatches like Unsafe. For exactly this purpose. And they are clearly labeled as dangerous (hey, what does "unsafe" mean?) But mainstream mechanisms should be as safe and surprise-free as we can make them. There are arguments on both sides of the "mutable vs immutable" argument, but the "experts need mutable value types for performance" is pretty bogus, because it only looks at one side of performance. Compare this to another similar performance story: direct access to pointers. C programmers often claim that access to raw pointers is their secret weapon. But, if *anyone* has access to raw pointers, the optimizer is crippled *for everyone*. (Many optimizations are only possible if the VM can control all the pointers, such as compacting GC.) So giving a "secret weapon" to a small cadre of elite hax0rs completely reams the other 99.99% of users. Which seems like a terrible tradeoff, especially given that VMs are better than all but the most elite of hax0rs (most of which, I'll point out, are working on VMs.) Mutable structs are basically the same tradeoff. If you have mutable structs, you need a way to pass them by reference. Not only does this add a whole new layer of complexity that Java never had, but now you have the problem of pointers to the *middle* of an object. Said interior pointers make life much harder for the GC. (GC performance on .NET trails badly behind Java, in part because of this very reason.) So again, I think the "experts need mutable structs" argument is effectively throwing the other 99.99% of devs under the bus -- another bad tradeoff. If you're going to make performance arguments of the form "If I had tool X, I could do Y", then you have to include the other side, which is "if no one has tool X, what more can the VM do?" And the answer is often surprising. Now, there are other arguments why we might consider mutable value types (and while this is a reasonable place for them, I'll just cut this off here as I'm not prepared to engage further on this subject now -- I have too many other things going on that I need to focus on, sorry.) On 1/6/2015 11:40 AM, Vitaly Davidovich wrote: > I was merely using unsafe as an example of a tool (lib, not language, in > this case) that's (relatively) rarely used, but very important for some > cases. If you want to provide a Sufficiently Smart Compiler that makes > it obsolete, great, but that's not here presently and probably never > will be (it'll be good enough for majority of cases though). > > One thing that really irks me (and I think some others, based on my > observations) about some of the discussions around java and its > direction is the sentiment that java developers require hand holding, > can't be trusted, must be fully abstracted away from the machine, and > thus deserve a language with *mandatory* safety belts and restraints. > If java wants to stay relevant in high perf domains, that sentiment > needs to be dropped; escape hatches/unsafe-ish code must remain > available. Every language that I know of which markets itself as being > relevant in high perf scenarios allows for having small targeted amounts > of unsafe code; there's absolutely nothing wrong with that, and in fact, > it's a Good Thing. > > As for mutable value types, showing developers the types of issues they > may face is fine, direct them towards immutability, but ultimately, let > them decide! (clearly, mutable value types aren't going to be > implemented, I'm not going to change your mind, but I dislike the > philosophy behind it). > > On Tue, Jan 6, 2015 at 11:21 AM, Brian Goetz > wrote: > > If your comparison is that "mutable structs are like unsafe", great > -- that's a bulletproof argument that value types must not be > mutable, and there is no need for further discussion on the topic :) > > (Pause while Brian gets out his chisel to carve it in stone....done.) > > > On 1/6/2015 10:24 AM, Vitaly Davidovich wrote: > > I've seen and read that blog before. I'm not disputing that one > needs > to be careful in how C# mutable structs are used and as > mentioned, it's > a rare (but useful) use case. But really, the issue in that blog is > more about how compiler desugars/transforms C#'s using block and its > poor interaction with value types when someone doesn't fully > understand > the sugaring mechanics. I suspect the problem Eric (when he was > still > at MSFT) had was developers not taking the time to fully > understand the > tool. Yes, it's error prone if you're sleep walking through > code, but > then again, don't sleep walk through code! > > So, I'll agree that their use should be restricted, but it's nice to > have when needed (akin to using com.sun.misc.Unsafe -- it's a sharp > knife that can inflict serious damage if not wielded carefully, > but very > handy/surgical when called for and used carefully). > > On Tue, Jan 6, 2015 at 10:13 AM, Brian Goetz > > __>> wrote: > > The C# folks concluded in hindsight that mutable structs > was a big > mistake. See Eric Lippert's blog: > > http://blogs.msdn.com/b/____ericlippert/archive/2011/03/____14/to-box-or-not-to-box-that-____is-the-question.aspx > > > > > > TL;DR version (last line): "Once again the moral of the > story is: > mutable value types are enough pure evil to turn you all > into hermit > crabs, and therefore should be avoided." > > > > > > On 1/6/2015 8:33 AM, Vitaly Davidovich wrote: > > What are the "far too many" downsides? The lost writes can > happen if not > careful, but what else? I'm assuming though if mutable > value > types were > possible, then language and VM would allow passing them > by ref. > > Sent from my phone > On Jan 6, 2015 5:12 AM, "Stephen Colebourne" > > >> wrote: > > On 6 January 2015 at 01:58, Vitaly Davidovich > > >> wrote: > > Mutable structs aren't used all that often in > c# but > there are times when > they're very useful. > > > When I've thought about this, the first use case > for mutable > I came up > with was property objects. These allow code like: > > class Person { > public final Property surname; > } > String surname = person.surname.get(); > person.surname.set("MySurname"____); > Property p = person.surname; > > Here, Property is a mutable wrapper around the surname, > hence the > get() and set(). Today, this design is only > suitable for the > client > side, as creating all the wrapper objects creates > far too > much garbage > and churn for a server (tried it, was a car crash). > > However, despite the appeal of making this work via > value > types, I'm > willing to agree that allowing mutability would > have far too > many > otehr negative consequences. Thus I agree with the > "only > immutable" > design direction. > > Stephen > > > From peter.levart at gmail.com Tue Jan 6 17:27:22 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 06 Jan 2015 18:27:22 +0100 Subject: hg: valhalla/valhalla/langtools: BytecodeMapping generation fixes: In-Reply-To: <201501061300.t06D0q5V024359@aojmv0008> References: <201501061300.t06D0q5V024359@aojmv0008> Message-ID: <54AC1AFA.2050008@gmail.com> On 01/06/2015 02:00 PM, maurizio.cimadamore at oracle.com wrote: > Changeset: 8331682af2ed > Author: mcimadamore > Date: 2015-01-06 12:54 +0000 > URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/8331682af2ed > > BytecodeMapping generation fixes: > * missing BMA for eq/ne any comparisons nested within logic operators &&/|| > * missing BMA for field/method access immediately following a constructor call > * refactor Items code in order to avoid cast to AnyItem in order to access type info associated with a given item > * added tests > > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java > + test/tools/javac/valhalla/typespec/items/tests/TestBinary.java > ! test/tools/javac/valhalla/typespec/items/tests/TestInstanceof.java > ! test/tools/javac/valhalla/typespec/items/tests/TestNew.java > Thanks Maurizio, It works now. But I have another (seems like javac) problem. Since layers are not yet implemented (or I just can't seem to figure out how to use them), I tried to use __WhereRef(T) and __WhereVal(T) with inheritance to simulate them. My 1st goal was to create a Box class that would implement Object.equals() in a way that would use Objects.equals() for reference T and '==' for value T. Here's what I came up with: // the Ref "layer" abstract class RefBoxBase { public abstract T get(); public __WhereRef(T) int hashCode() { return Objects.hashCode(get()); } public __WhereRef(T) boolean equals(Object obj) { return (this == obj) || (obj != null && this.getClass() == obj.getClass() && Objects.equals(this.get(), ((RefBoxBase) obj).get()) ); } } // the Val "overlay" public final class Box extends RefBoxBase { private T value; public Box(T value) { this.value = value; } public Box() { // leave default value } public T get() { return value; } public __WhereVal(T) int hashCode() { return 0; // don't know yet how to do something meaningful here } public __WhereVal(T) boolean equals(Object obj) { return (this == obj) || (obj != null && this.getClass() == obj.getClass() && this.get() == ((Box)obj).get()); } } And the test: public class Test { public static void main(String[] args) { System.out.println(new Box(1).equals(new Box(1))); // this compiles fine System.out.println(new Box("a").equals(new Box("a"))); // compilation error } } src/util/Test.java:6: error: bad receiver type Box in restricted method call System.out.println(new Box("a").equals(new Box("a"))); ^ Note: src/util/RefBoxBase.java uses unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. 1 error It's true that Box.equals() is restricted for Val receiver, but I thought that RefBoxBase.equals() would be "uncovered" in this case an inherited in Ref specialization. Regards, Peter From maurizio.cimadamore at oracle.com Tue Jan 6 17:51:09 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 06 Jan 2015 17:51:09 +0000 Subject: hg: valhalla/valhalla/langtools: BytecodeMapping generation fixes: In-Reply-To: <54AC1AFA.2050008@gmail.com> References: <201501061300.t06D0q5V024359@aojmv0008> <54AC1AFA.2050008@gmail.com> Message-ID: <54AC208D.6080505@oracle.com> As you found out, layering support is still missing from the prototype; you have ability to declare RefOnly/ValOnly members but that's pretty much about it. It's still quite powerful though, albeit some things don't work automagically as you'd expect. What you are looking for is a way to write implementation by parts - and that's the bit that's missing from the current implementation (we have a rough proof of concept that does that, but it's not very polished, and the syntax that it supports is very different from the one described in the document). Maurizio On 06/01/15 17:27, Peter Levart wrote: > On 01/06/2015 02:00 PM, maurizio.cimadamore at oracle.com wrote: >> Changeset: 8331682af2ed >> Author: mcimadamore >> Date: 2015-01-06 12:54 +0000 >> URL:http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/8331682af2ed >> >> BytecodeMapping generation fixes: >> * missing BMA for eq/ne any comparisons nested within logic operators &&/|| >> * missing BMA for field/method access immediately following a constructor call >> * refactor Items code in order to avoid cast to AnyItem in order to access type info associated with a given item >> * added tests >> >> ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java >> ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java >> + test/tools/javac/valhalla/typespec/items/tests/TestBinary.java >> ! test/tools/javac/valhalla/typespec/items/tests/TestInstanceof.java >> ! test/tools/javac/valhalla/typespec/items/tests/TestNew.java >> > > Thanks Maurizio, > > It works now. But I have another (seems like javac) problem. Since > layers are not yet implemented (or I just can't seem to figure out how > to use them), I tried to use __WhereRef(T) and __WhereVal(T) with > inheritance to simulate them. My 1st goal was to create a Box > class that would implement Object.equals() in a way that would use > Objects.equals() for reference T and '==' for value T. Here's what I > came up with: > > // the Ref "layer" > > abstract class RefBoxBase { > > public abstract T get(); > > public __WhereRef(T) int hashCode() { > return Objects.hashCode(get()); > } > > public __WhereRef(T) boolean equals(Object obj) { > return (this == obj) || > (obj != null && > this.getClass() == obj.getClass() && > Objects.equals(this.get(), ((RefBoxBase) obj).get()) > ); > } > } > > // the Val "overlay" > > public final class Box extends RefBoxBase { > > private T value; > > public Box(T value) { > this.value = value; > } > > public Box() { > // leave default value > } > > public T get() { > return value; > } > > public __WhereVal(T) int hashCode() { > return 0; // don't know yet how to do something meaningful here > } > > public __WhereVal(T) boolean equals(Object obj) { > return (this == obj) || > (obj != null && > this.getClass() == obj.getClass() && > this.get() == ((Box)obj).get()); > } > } > > > And the test: > > public class Test { > public static void main(String[] args) { > System.out.println(new Box(1).equals(new Box(1))); > // this compiles fine > > System.out.println(new Box("a").equals(new > Box("a"))); // compilation error > } > } > > src/util/Test.java:6: error: bad receiver type Box in > restricted method call > System.out.println(new Box("a").equals(new > Box("a"))); > ^ > Note: src/util/RefBoxBase.java uses unchecked or unsafe operations. > Note: Recompile with -Xlint:unchecked for details. > 1 error > > > It's true that Box.equals() is restricted for Val receiver, but I > thought that RefBoxBase.equals() would be "uncovered" in this case an > inherited in Ref specialization. > > > > Regards, Peter > > From brian.goetz at oracle.com Tue Jan 6 18:09:33 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 13:09:33 -0500 Subject: hg: valhalla/valhalla/langtools: BytecodeMapping generation fixes: In-Reply-To: <54AC208D.6080505@oracle.com> References: <201501061300.t06D0q5V024359@aojmv0008> <54AC1AFA.2050008@gmail.com> <54AC208D.6080505@oracle.com> Message-ID: <54AC24DD.9030804@oracle.com> Also, note: the __WhereRef(T) syntax is intended to be a quick hack, this is definitely not where the syntax is going. It was just what was easy to implement quickly. On 1/6/2015 12:51 PM, Maurizio Cimadamore wrote: > As you found out, layering support is still missing from the prototype; > you have ability to declare RefOnly/ValOnly members but that's pretty > much about it. It's still quite powerful though, albeit some things > don't work automagically as you'd expect. What you are looking for is a > way to write implementation by parts - and that's the bit that's missing > from the current implementation (we have a rough proof of concept that > does that, but it's not very polished, and the syntax that it supports > is very different from the one described in the document). > > Maurizio > > On 06/01/15 17:27, Peter Levart wrote: >> On 01/06/2015 02:00 PM, maurizio.cimadamore at oracle.com wrote: >>> Changeset: 8331682af2ed >>> Author: mcimadamore >>> Date: 2015-01-06 12:54 +0000 >>> URL:http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/8331682af2ed >>> >>> >>> BytecodeMapping generation fixes: >>> * missing BMA for eq/ne any comparisons nested within logic operators >>> &&/|| >>> * missing BMA for field/method access immediately following a >>> constructor call >>> * refactor Items code in order to avoid cast to AnyItem in order to >>> access type info associated with a given item >>> * added tests >>> >>> ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java >>> ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java >>> + test/tools/javac/valhalla/typespec/items/tests/TestBinary.java >>> ! test/tools/javac/valhalla/typespec/items/tests/TestInstanceof.java >>> ! test/tools/javac/valhalla/typespec/items/tests/TestNew.java >>> >> >> Thanks Maurizio, >> >> It works now. But I have another (seems like javac) problem. Since >> layers are not yet implemented (or I just can't seem to figure out how >> to use them), I tried to use __WhereRef(T) and __WhereVal(T) with >> inheritance to simulate them. My 1st goal was to create a Box >> class that would implement Object.equals() in a way that would use >> Objects.equals() for reference T and '==' for value T. Here's what I >> came up with: >> >> // the Ref "layer" >> >> abstract class RefBoxBase { >> >> public abstract T get(); >> >> public __WhereRef(T) int hashCode() { >> return Objects.hashCode(get()); >> } >> >> public __WhereRef(T) boolean equals(Object obj) { >> return (this == obj) || >> (obj != null && >> this.getClass() == obj.getClass() && >> Objects.equals(this.get(), ((RefBoxBase) obj).get()) >> ); >> } >> } >> >> // the Val "overlay" >> >> public final class Box extends RefBoxBase { >> >> private T value; >> >> public Box(T value) { >> this.value = value; >> } >> >> public Box() { >> // leave default value >> } >> >> public T get() { >> return value; >> } >> >> public __WhereVal(T) int hashCode() { >> return 0; // don't know yet how to do something meaningful here >> } >> >> public __WhereVal(T) boolean equals(Object obj) { >> return (this == obj) || >> (obj != null && >> this.getClass() == obj.getClass() && >> this.get() == ((Box)obj).get()); >> } >> } >> >> >> And the test: >> >> public class Test { >> public static void main(String[] args) { >> System.out.println(new Box(1).equals(new Box(1))); >> // this compiles fine >> >> System.out.println(new Box("a").equals(new >> Box("a"))); // compilation error >> } >> } >> >> src/util/Test.java:6: error: bad receiver type Box in >> restricted method call >> System.out.println(new Box("a").equals(new >> Box("a"))); >> ^ >> Note: src/util/RefBoxBase.java uses unchecked or unsafe operations. >> Note: Recompile with -Xlint:unchecked for details. >> 1 error >> >> >> It's true that Box.equals() is restricted for Val receiver, but I >> thought that RefBoxBase.equals() would be "uncovered" in this case an >> inherited in Ref specialization. >> >> >> >> Regards, Peter >> >> > From peter.levart at gmail.com Tue Jan 6 19:50:39 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 06 Jan 2015 20:50:39 +0100 Subject: hg: valhalla/valhalla/langtools: BytecodeMapping generation fixes: In-Reply-To: <54AC24DD.9030804@oracle.com> References: <201501061300.t06D0q5V024359@aojmv0008> <54AC1AFA.2050008@gmail.com> <54AC208D.6080505@oracle.com> <54AC24DD.9030804@oracle.com> Message-ID: <54AC3C8F.2060909@gmail.com> I see, thanks. The problem with my Box example as it came to me after the fact is that the T=reference specialization is not a specialization at all - the javac generated template is used as is for the T=reference case unchanged - therefore it contains the __WhereVal(T) method too and javac is right to refuse to compile a call to it. Perhaps, if I reversed the parts and put __WhereVal(T) method in private superclass and __WhereRef(T) method in subclass, then it should work for T=reference case and probably also for T=primitive case if __WhereRef(T) methods are not included in primitive specialization and javac allows me to call the inherited method from superclass. Will try that. Regards, Peter On 01/06/2015 07:09 PM, Brian Goetz wrote: > Also, note: the __WhereRef(T) syntax is intended to be a quick hack, > this is definitely not where the syntax is going. It was just what > was easy to implement quickly. > > On 1/6/2015 12:51 PM, Maurizio Cimadamore wrote: >> As you found out, layering support is still missing from the prototype; >> you have ability to declare RefOnly/ValOnly members but that's pretty >> much about it. It's still quite powerful though, albeit some things >> don't work automagically as you'd expect. What you are looking for is a >> way to write implementation by parts - and that's the bit that's missing >> from the current implementation (we have a rough proof of concept that >> does that, but it's not very polished, and the syntax that it supports >> is very different from the one described in the document). >> >> Maurizio >> >> On 06/01/15 17:27, Peter Levart wrote: >>> On 01/06/2015 02:00 PM, maurizio.cimadamore at oracle.com wrote: >>>> Changeset: 8331682af2ed >>>> Author: mcimadamore >>>> Date: 2015-01-06 12:54 +0000 >>>> URL:http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/8331682af2ed >>>> >>>> >>>> >>>> BytecodeMapping generation fixes: >>>> * missing BMA for eq/ne any comparisons nested within logic operators >>>> &&/|| >>>> * missing BMA for field/method access immediately following a >>>> constructor call >>>> * refactor Items code in order to avoid cast to AnyItem in order to >>>> access type info associated with a given item >>>> * added tests >>>> >>>> ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java >>>> ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java >>>> + test/tools/javac/valhalla/typespec/items/tests/TestBinary.java >>>> ! test/tools/javac/valhalla/typespec/items/tests/TestInstanceof.java >>>> ! test/tools/javac/valhalla/typespec/items/tests/TestNew.java >>>> >>> >>> Thanks Maurizio, >>> >>> It works now. But I have another (seems like javac) problem. Since >>> layers are not yet implemented (or I just can't seem to figure out how >>> to use them), I tried to use __WhereRef(T) and __WhereVal(T) with >>> inheritance to simulate them. My 1st goal was to create a Box >>> class that would implement Object.equals() in a way that would use >>> Objects.equals() for reference T and '==' for value T. Here's what I >>> came up with: >>> >>> // the Ref "layer" >>> >>> abstract class RefBoxBase { >>> >>> public abstract T get(); >>> >>> public __WhereRef(T) int hashCode() { >>> return Objects.hashCode(get()); >>> } >>> >>> public __WhereRef(T) boolean equals(Object obj) { >>> return (this == obj) || >>> (obj != null && >>> this.getClass() == obj.getClass() && >>> Objects.equals(this.get(), ((RefBoxBase) obj).get()) >>> ); >>> } >>> } >>> >>> // the Val "overlay" >>> >>> public final class Box extends RefBoxBase { >>> >>> private T value; >>> >>> public Box(T value) { >>> this.value = value; >>> } >>> >>> public Box() { >>> // leave default value >>> } >>> >>> public T get() { >>> return value; >>> } >>> >>> public __WhereVal(T) int hashCode() { >>> return 0; // don't know yet how to do something meaningful here >>> } >>> >>> public __WhereVal(T) boolean equals(Object obj) { >>> return (this == obj) || >>> (obj != null && >>> this.getClass() == obj.getClass() && >>> this.get() == ((Box)obj).get()); >>> } >>> } >>> >>> >>> And the test: >>> >>> public class Test { >>> public static void main(String[] args) { >>> System.out.println(new Box(1).equals(new Box(1))); >>> // this compiles fine >>> >>> System.out.println(new Box("a").equals(new >>> Box("a"))); // compilation error >>> } >>> } >>> >>> src/util/Test.java:6: error: bad receiver type Box in >>> restricted method call >>> System.out.println(new Box("a").equals(new >>> Box("a"))); >>> ^ >>> Note: src/util/RefBoxBase.java uses unchecked or unsafe operations. >>> Note: Recompile with -Xlint:unchecked for details. >>> 1 error >>> >>> >>> It's true that Box.equals() is restricted for Val receiver, but I >>> thought that RefBoxBase.equals() would be "uncovered" in this case an >>> inherited in Ref specialization. >>> >>> >>> >>> Regards, Peter >>> >>> >> From mikeb01 at gmail.com Tue Jan 6 19:56:07 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Wed, 7 Jan 2015 08:56:07 +1300 Subject: Clarification on migration to value types and boxed vs unboxed representations In-Reply-To: <54AC1AC5.6030907@oracle.com> References: <77F3A474-B11E-4D8C-BEBF-219D70DB19BB@oracle.com> <54ABFBA2.4090708@oracle.com> <54AC0B86.1030701@oracle.com> <54AC1AC5.6030907@oracle.com> Message-ID: Apologies for continuing this thread, but I wanted to mention that those who are interested in mutable structs should look at Gil Tene and Martin Thompson's Object Layout proposal (http://www.objectlayout.org/) - which is a complimentary feature to value types. If you have questions, the best place to ask would be on the mechanical sympathy mailing list ( https://groups.google.com/forum/#!forum/mechanical-sympathy). Gil and Martin regularly follow that list. Mike. On 7 January 2015 at 06:26, Brian Goetz wrote: > Obviously both extremes (encase everyone in concrete, vs force-feeding > chainsaws to schoolchildren) are ridiculous. Which means we're in the > middle, where judgement has to be used to determine a fair balance. And > since its subjective, there will be differences of opinion. (And when > there are differences of opinion, it is tempting to frame the "slightly > more conservative than you" perspective as reactionary and the "slightly > more liberal than you" perspective as communist, but that's not really > constructive.) > > We do have escape hatches like Unsafe. For exactly this purpose. And > they are clearly labeled as dangerous (hey, what does "unsafe" mean?) But > mainstream mechanisms should be as safe and surprise-free as we can make > them. > > There are arguments on both sides of the "mutable vs immutable" argument, > but the "experts need mutable value types for performance" is pretty bogus, > because it only looks at one side of performance. Compare this to another > similar performance story: direct access to pointers. C programmers often > claim that access to raw pointers is their secret weapon. But, if *anyone* > has access to raw pointers, the optimizer is crippled *for everyone*. > (Many optimizations are only possible if the VM can control all the > pointers, such as compacting GC.) So giving a "secret weapon" to a small > cadre of elite hax0rs completely reams the other 99.99% of users. Which > seems like a terrible tradeoff, especially given that VMs are better than > all but the most elite of hax0rs (most of which, I'll point out, are > working on VMs.) > > Mutable structs are basically the same tradeoff. If you have mutable > structs, you need a way to pass them by reference. Not only does this add > a whole new layer of complexity that Java never had, but now you have the > problem of pointers to the *middle* of an object. Said interior pointers > make life much harder for the GC. (GC performance on .NET trails badly > behind Java, in part because of this very reason.) So again, I think the > "experts need mutable structs" argument is effectively throwing the other > 99.99% of devs under the bus -- another bad tradeoff. > > If you're going to make performance arguments of the form "If I had tool > X, I could do Y", then you have to include the other side, which is "if no > one has tool X, what more can the VM do?" And the answer is often > surprising. > > Now, there are other arguments why we might consider mutable value types > (and while this is a reasonable place for them, I'll just cut this off here > as I'm not prepared to engage further on this subject now -- I have too > many other things going on that I need to focus on, sorry.) > > On 1/6/2015 11:40 AM, Vitaly Davidovich wrote: > >> I was merely using unsafe as an example of a tool (lib, not language, in >> this case) that's (relatively) rarely used, but very important for some >> cases. If you want to provide a Sufficiently Smart Compiler that makes >> it obsolete, great, but that's not here presently and probably never >> will be (it'll be good enough for majority of cases though). >> >> One thing that really irks me (and I think some others, based on my >> observations) about some of the discussions around java and its >> direction is the sentiment that java developers require hand holding, >> can't be trusted, must be fully abstracted away from the machine, and >> thus deserve a language with *mandatory* safety belts and restraints. >> If java wants to stay relevant in high perf domains, that sentiment >> needs to be dropped; escape hatches/unsafe-ish code must remain >> available. Every language that I know of which markets itself as being >> relevant in high perf scenarios allows for having small targeted amounts >> of unsafe code; there's absolutely nothing wrong with that, and in fact, >> it's a Good Thing. >> >> As for mutable value types, showing developers the types of issues they >> may face is fine, direct them towards immutability, but ultimately, let >> them decide! (clearly, mutable value types aren't going to be >> implemented, I'm not going to change your mind, but I dislike the >> philosophy behind it). >> >> On Tue, Jan 6, 2015 at 11:21 AM, Brian Goetz > > wrote: >> >> If your comparison is that "mutable structs are like unsafe", great >> -- that's a bulletproof argument that value types must not be >> mutable, and there is no need for further discussion on the topic :) >> >> (Pause while Brian gets out his chisel to carve it in stone....done.) >> >> >> On 1/6/2015 10:24 AM, Vitaly Davidovich wrote: >> >> I've seen and read that blog before. I'm not disputing that one >> needs >> to be careful in how C# mutable structs are used and as >> mentioned, it's >> a rare (but useful) use case. But really, the issue in that blog >> is >> more about how compiler desugars/transforms C#'s using block and >> its >> poor interaction with value types when someone doesn't fully >> understand >> the sugaring mechanics. I suspect the problem Eric (when he was >> still >> at MSFT) had was developers not taking the time to fully >> understand the >> tool. Yes, it's error prone if you're sleep walking through >> code, but >> then again, don't sleep walk through code! >> >> So, I'll agree that their use should be restricted, but it's nice >> to >> have when needed (akin to using com.sun.misc.Unsafe -- it's a >> sharp >> knife that can inflict serious damage if not wielded carefully, >> but very >> handy/surgical when called for and used carefully). >> >> On Tue, Jan 6, 2015 at 10:13 AM, Brian Goetz >> >> > __>> wrote: >> >> The C# folks concluded in hindsight that mutable structs >> was a big >> mistake. See Eric Lippert's blog: >> >> http://blogs.msdn.com/b/____ericlippert/archive/2011/03/__ >> __14/to-box-or-not-to-box-that-____is-the-question.aspx >> > 14/to-box-or-not-to-box-that-__is-the-question.aspx> >> >> > 14/to-box-or-not-to-box-that-__is-the-question.aspx >> > 14/to-box-or-not-to-box-that-is-the-question.aspx>> >> >> TL;DR version (last line): "Once again the moral of the >> story is: >> mutable value types are enough pure evil to turn you all >> into hermit >> crabs, and therefore should be avoided." >> >> >> >> >> >> On 1/6/2015 8:33 AM, Vitaly Davidovich wrote: >> >> What are the "far too many" downsides? The lost writes >> can >> happen if not >> careful, but what else? I'm assuming though if mutable >> value >> types were >> possible, then language and VM would allow passing them >> by ref. >> >> Sent from my phone >> On Jan 6, 2015 5:12 AM, "Stephen Colebourne" >> >> >> >> wrote: >> >> On 6 January 2015 at 01:58, Vitaly Davidovich >> >> >> wrote: >> >> Mutable structs aren't used all that often in >> c# but >> there are times when >> they're very useful. >> >> >> When I've thought about this, the first use case >> for mutable >> I came up >> with was property objects. These allow code like: >> >> class Person { >> public final Property surname; >> } >> String surname = person.surname.get(); >> person.surname.set("MySurname"____); >> Property p = person.surname; >> >> Here, Property is a mutable wrapper around the >> surname, >> hence the >> get() and set(). Today, this design is only >> suitable for the >> client >> side, as creating all the wrapper objects creates >> far too >> much garbage >> and churn for a server (tried it, was a car crash). >> >> However, despite the appeal of making this work via >> value >> types, I'm >> willing to agree that allowing mutability would >> have far too >> many >> otehr negative consequences. Thus I agree with the >> "only >> immutable" >> design direction. >> >> Stephen >> >> >> >> From brian.goetz at oracle.com Tue Jan 6 19:58:50 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 14:58:50 -0500 Subject: hg: valhalla/valhalla/langtools: BytecodeMapping generation fixes: In-Reply-To: <54AC3C8F.2060909@gmail.com> References: <201501061300.t06D0q5V024359@aojmv0008> <54AC1AFA.2050008@gmail.com> <54AC208D.6080505@oracle.com> <54AC24DD.9030804@oracle.com> <54AC3C8F.2060909@gmail.com> Message-ID: <54AC3E7A.6040907@oracle.com> Right. There's a little more (but not much) plumbing needed here. We've got the part where a ref-specific method is not propagated into the template at specialization time; we need to do the part where val-specific methods are hidden when you load the erased class without specialization. As a short-term hack, we'll probably rename these methods to something like: @Synthetic __$TEMPLATE_IGNORE_ME$__foo(...) where they'll be visible but hopefully not in the way. Stepping back, the point of implementing specialization (hacks or not) is to determine how practical it really is to any-fy our existing generic libraries. My working theory is that if we can anyfy Collections and Streams, we've already seen 99% of what can go wrong, since these libraries have the "implementation tricks" knob turned up to 11 (either for performance or compatibility reasons.) What I expect will happen is, the experiment of trying to convert these libraries will generate a half dozen or so "oh, crap" realizations, some of which will feed back into new requirements. But there's only so far that envisioning things on the whiteboard will get you. On 1/6/2015 2:50 PM, Peter Levart wrote: > I see, thanks. > > The problem with my Box example as it came to me after the fact > is that the T=reference specialization is not a specialization at all - > the javac generated template is used as is for the T=reference case > unchanged - therefore it contains the __WhereVal(T) method too and javac > is right to refuse to compile a call to it. Perhaps, if I reversed the > parts and put __WhereVal(T) method in private superclass and > __WhereRef(T) method in subclass, then it should work for T=reference > case and probably also for T=primitive case if __WhereRef(T) methods are > not included in primitive specialization and javac allows me to call the > inherited method from superclass. > > Will try that. > > > Regards, Peter > > On 01/06/2015 07:09 PM, Brian Goetz wrote: >> Also, note: the __WhereRef(T) syntax is intended to be a quick hack, >> this is definitely not where the syntax is going. It was just what >> was easy to implement quickly. >> >> On 1/6/2015 12:51 PM, Maurizio Cimadamore wrote: >>> As you found out, layering support is still missing from the prototype; >>> you have ability to declare RefOnly/ValOnly members but that's pretty >>> much about it. It's still quite powerful though, albeit some things >>> don't work automagically as you'd expect. What you are looking for is a >>> way to write implementation by parts - and that's the bit that's missing >>> from the current implementation (we have a rough proof of concept that >>> does that, but it's not very polished, and the syntax that it supports >>> is very different from the one described in the document). >>> >>> Maurizio >>> >>> On 06/01/15 17:27, Peter Levart wrote: >>>> On 01/06/2015 02:00 PM, maurizio.cimadamore at oracle.com wrote: >>>>> Changeset: 8331682af2ed >>>>> Author: mcimadamore >>>>> Date: 2015-01-06 12:54 +0000 >>>>> URL:http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/8331682af2ed >>>>> >>>>> >>>>> >>>>> BytecodeMapping generation fixes: >>>>> * missing BMA for eq/ne any comparisons nested within logic operators >>>>> &&/|| >>>>> * missing BMA for field/method access immediately following a >>>>> constructor call >>>>> * refactor Items code in order to avoid cast to AnyItem in order to >>>>> access type info associated with a given item >>>>> * added tests >>>>> >>>>> ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java >>>>> ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java >>>>> + test/tools/javac/valhalla/typespec/items/tests/TestBinary.java >>>>> ! test/tools/javac/valhalla/typespec/items/tests/TestInstanceof.java >>>>> ! test/tools/javac/valhalla/typespec/items/tests/TestNew.java >>>>> >>>> >>>> Thanks Maurizio, >>>> >>>> It works now. But I have another (seems like javac) problem. Since >>>> layers are not yet implemented (or I just can't seem to figure out how >>>> to use them), I tried to use __WhereRef(T) and __WhereVal(T) with >>>> inheritance to simulate them. My 1st goal was to create a Box >>>> class that would implement Object.equals() in a way that would use >>>> Objects.equals() for reference T and '==' for value T. Here's what I >>>> came up with: >>>> >>>> // the Ref "layer" >>>> >>>> abstract class RefBoxBase { >>>> >>>> public abstract T get(); >>>> >>>> public __WhereRef(T) int hashCode() { >>>> return Objects.hashCode(get()); >>>> } >>>> >>>> public __WhereRef(T) boolean equals(Object obj) { >>>> return (this == obj) || >>>> (obj != null && >>>> this.getClass() == obj.getClass() && >>>> Objects.equals(this.get(), ((RefBoxBase) obj).get()) >>>> ); >>>> } >>>> } >>>> >>>> // the Val "overlay" >>>> >>>> public final class Box extends RefBoxBase { >>>> >>>> private T value; >>>> >>>> public Box(T value) { >>>> this.value = value; >>>> } >>>> >>>> public Box() { >>>> // leave default value >>>> } >>>> >>>> public T get() { >>>> return value; >>>> } >>>> >>>> public __WhereVal(T) int hashCode() { >>>> return 0; // don't know yet how to do something meaningful here >>>> } >>>> >>>> public __WhereVal(T) boolean equals(Object obj) { >>>> return (this == obj) || >>>> (obj != null && >>>> this.getClass() == obj.getClass() && >>>> this.get() == ((Box)obj).get()); >>>> } >>>> } >>>> >>>> >>>> And the test: >>>> >>>> public class Test { >>>> public static void main(String[] args) { >>>> System.out.println(new Box(1).equals(new Box(1))); >>>> // this compiles fine >>>> >>>> System.out.println(new Box("a").equals(new >>>> Box("a"))); // compilation error >>>> } >>>> } >>>> >>>> src/util/Test.java:6: error: bad receiver type Box in >>>> restricted method call >>>> System.out.println(new Box("a").equals(new >>>> Box("a"))); >>>> ^ >>>> Note: src/util/RefBoxBase.java uses unchecked or unsafe operations. >>>> Note: Recompile with -Xlint:unchecked for details. >>>> 1 error >>>> >>>> >>>> It's true that Box.equals() is restricted for Val receiver, but I >>>> thought that RefBoxBase.equals() would be "uncovered" in this case an >>>> inherited in Ref specialization. >>>> >>>> >>>> >>>> Regards, Peter >>>> >>>> >>> > From mikeb01 at gmail.com Tue Jan 6 20:38:52 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Wed, 7 Jan 2015 09:38:52 +1300 Subject: Question on layer/peeling In-Reply-To: <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> Message-ID: > > The trick can also be coaxed to give semi-plausible semantics to seemingly > bogus expressions like "x == null" or "x == (Object)y" (for any x). If "x" > is a primitive, since autobox conversions never produce "null", then "x == > null" and "x == (Object)y" are complicated ways of saying "false". But > this doesn't help with nulling assignments like "a[i] = null"; there you > need "T.null" (my bikeshed preference to "default(T)"). > I think allowing x == null (always false) and a[i] = T.null for value types would certainly make the coding simpler as it would avoid layering for a number of very common cases. In the null comparison case I would assume that Hotspot could easily throw away the condition altogether in the specialised class. From john.r.rose at oracle.com Tue Jan 6 20:52:09 2015 From: john.r.rose at oracle.com (John Rose) Date: Tue, 6 Jan 2015 12:52:09 -0800 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> Message-ID: <7D0ADE8D-B060-4F51-8B5B-ED3335D87EAE@oracle.com> On Jan 6, 2015, at 12:39 AM, Michael Barker wrote: > > Could allowing primitives to implement interfaces like Comparable make the following implementations be possible, such that the JIT could generate specialisations for the primitive types? > > public class Arrays > { > public void sort(T[] array) > { > // ... implement sort > } In short, yes. ? John From brian.goetz at oracle.com Tue Jan 6 20:52:08 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 15:52:08 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> Message-ID: <54AC4AF8.4090709@oracle.com> Some sort of T.null / T.default / default(T) is easy and generally seems a no-brainer, just haven't implemented it yet. Assigning semantics to x == null is a nice idea and would indeed reduce the work of anyfication, which is good. It should be trivial, but isn't quite. There are two ways to represent this in bytecode: aload aconst_null if_acmpeq and aload ifnull The latter is easy; replace "ifnull " with "pop" (and "ifnonnull " with "pop; goto ") (And don't forget to use pop2 instead if specializing to long or double.) The former, however, is not quite as easy, since what's on the stack could have come from a much more complicated evaluation that leaves a null on the stack. We'd have to ensure there was no way the former bytecode sequence got generated from javac. On 1/6/2015 3:38 PM, Michael Barker wrote: > The trick can also be coaxed to give semi-plausible semantics to > seemingly bogus expressions like "x == null" or "x == (Object)y" > (for any x). If "x" is a primitive, since autobox conversions never > produce "null", then "x == null" and "x == (Object)y" are > complicated ways of saying "false". But this doesn't help with > nulling assignments like "a[i] = null"; there you need "T.null" (my > bikeshed preference to "default(T)"). > > > I think allowing x == null (always false) and a[i] = T.null for value > types would certainly make the coding simpler as it would avoid layering > for a number of very common cases. In the null comparison case I would > assume that Hotspot could easily throw away the condition altogether in > the specialised class. From timo.kinnunen at gmail.com Tue Jan 6 21:00:10 2015 From: timo.kinnunen at gmail.com (=?utf-8?Q?Timo_Kinnunen?=) Date: Tue, 6 Jan 2015 21:00:10 +0000 Subject: =?utf-8?Q?Re:_Question_on_layer/peeling?= In-Reply-To: <54AC4AF8.4090709@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> , <54AC4AF8.4090709@oracle.com> Message-ID: <54ac4e24.104dc20a.3a08.26ca@mx.google.com> Before assigning semantics to x == null, let?s not forget boxing. What will happen if x is a boxed value which was returned from a call to the non-specialized Map.get(Object key) method? -- Have a nice day, Timo. Sent from Windows Mail From: Brian Goetz Sent: ?Tuesday?, ?January? ?6?, ?2015 ?21?:?52 To: Michael Barker, John Rose Cc: valhalla-dev at openjdk.java.net Some sort of T.null / T.default / default(T) is easy and generally seems a no-brainer, just haven't implemented it yet. Assigning semantics to x == null is a nice idea and would indeed reduce the work of anyfication, which is good. It should be trivial, but isn't quite. There are two ways to represent this in bytecode: aload aconst_null if_acmpeq and aload ifnull The latter is easy; replace "ifnull " with "pop" (and "ifnonnull " with "pop; goto ") (And don't forget to use pop2 instead if specializing to long or double.) The former, however, is not quite as easy, since what's on the stack could have come from a much more complicated evaluation that leaves a null on the stack. We'd have to ensure there was no way the former bytecode sequence got generated from javac. On 1/6/2015 3:38 PM, Michael Barker wrote: > The trick can also be coaxed to give semi-plausible semantics to > seemingly bogus expressions like "x == null" or "x == (Object)y" > (for any x). If "x" is a primitive, since autobox conversions never > produce "null", then "x == null" and "x == (Object)y" are > complicated ways of saying "false". But this doesn't help with > nulling assignments like "a[i] = null"; there you need "T.null" (my > bikeshed preference to "default(T)"). > > > I think allowing x == null (always false) and a[i] = T.null for value > types would certainly make the coding simpler as it would avoid layering > for a number of very common cases. In the null comparison case I would > assume that Hotspot could easily throw away the condition altogether in > the specialised class. From john.r.rose at oracle.com Tue Jan 6 22:09:50 2015 From: john.r.rose at oracle.com (John Rose) Date: Tue, 6 Jan 2015 14:09:50 -0800 Subject: Question on layer/peeling In-Reply-To: <54ac4e24.104dc20a.3a08.26ca@mx.google.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <, <54AC4AF8.4090709@oracle.com> <>> <54ac4e24.104dc20a.3a08.26ca@mx.google.com> Message-ID: On Jan 6, 2015, at 1:00 PM, Timo Kinnunen wrote: > > Before assigning semantics to x == null, let?s not forget boxing. What will happen if x is a boxed value which was returned from a call to the non-specialized Map.get(Object key) method? ? Then the comparison will be false. In any case the type of x there is Object, isn't it, so there's nothing new here? ? John From timo.kinnunen at gmail.com Tue Jan 6 22:14:56 2015 From: timo.kinnunen at gmail.com (=?utf-8?Q?Timo_Kinnunen?=) Date: Tue, 6 Jan 2015 22:14:56 +0000 Subject: =?utf-8?Q?Re:_Question_on_layer/peeling?= In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <,<54AC4AF8.4090709@oracle.com> <>> <54ac4e24.104dc20a.3a08.26ca@mx.google.com>, Message-ID: <54ac609d.ae09b40a.3e3e.5972@mx.google.com> The type in bytecode will be Object, but the type in source code will be V, with V=boxed complex number, for example. The idiom I?m thinking of is Complex complex = numbers.get(nonexistentKey); if(complex == null) return; calculate(complex.re, complex.im); (Complex is a value type, numbers is a Java 5 hash map) -- Have a nice day, Timo. Sent from Windows Mail From: John Rose Sent: ?Tuesday?, ?January? ?6?, ?2015 ?23?:?09 To: Timo Kinnunen Cc: Brian Goetz, Michael Barker, valhalla-dev at openjdk.java.net On Jan 6, 2015, at 1:00 PM, Timo Kinnunen wrote: > > Before assigning semantics to x == null, let?s not forget boxing. What will happen if x is a boxed value which was returned from a call to the non-specialized Map.get(Object key) method? ? Then the comparison will be false. In any case the type of x there is Object, isn't it, so there's nothing new here? ? John From mikeb01 at gmail.com Tue Jan 6 22:31:40 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Wed, 7 Jan 2015 11:31:40 +1300 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> Message-ID: > > I suppose the workaround for #3 would be to take a Hasher (or whatever > name you pick) instance as a ctor arg; this would provide equals/hashCode > for your type T. > Cheers, this worked. I now have a working specialised HashMap[1]. When some of the "implementation by parts" makes an appearance, I'll rework to see if I can remove the custom Hashers. [1] modulo null problems mentioned earlier and not implementing some of the j.u.Map methods, e.g. keySet(), entrySet(). From vitalyd at gmail.com Tue Jan 6 22:32:36 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 6 Jan 2015 17:32:36 -0500 Subject: Question on layer/peeling In-Reply-To: <54ac609d.ae09b40a.3e3e.5972@mx.google.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54ac4e24.104dc20a.3a08.26ca@mx.google.com> <54ac609d.ae09b40a.3e3e.5972@mx.google.com> Message-ID: Wouldn't you get an NPE here since you'd be auto unboxing a null into a value type? I'm a bit confused about what you're trying to demonstrate. Sent from my phone On Jan 6, 2015 5:24 PM, "Timo Kinnunen" wrote: > The type in bytecode will be Object, but the type in source code will be > V, with V=boxed complex number, for example. The idiom I?m thinking of is > > > Complex complex = numbers.get(nonexistentKey); > > if(complex == null) return; > > calculate(complex.re, complex.im); > > > > > > > (Complex is a value type, numbers is a Java 5 hash map) > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > > > > > From: John Rose > Sent: ?Tuesday?, ?January? ?6?, ?2015 ?23?:?09 > To: Timo Kinnunen > Cc: Brian Goetz, Michael Barker, valhalla-dev at openjdk.java.net > > > > > > On Jan 6, 2015, at 1:00 PM, Timo Kinnunen wrote: > > > > Before assigning semantics to x == null, let?s not forget boxing. What > will happen if x is a boxed value which was returned from a call to the > non-specialized Map.get(Object key) method? > > ? Then the comparison will be false. In any case the type of x there is > Object, isn't it, so there's nothing new here? ? John From brian.goetz at oracle.com Tue Jan 6 22:32:27 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 17:32:27 -0500 Subject: Question on layer/peeling In-Reply-To: <54ac609d.ae09b40a.3e3e.5972@mx.google.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <, <54AC4AF8.4090709@oracle.com> <>> <54ac4e24.104dc20a.3a08.26ca@mx.google.com>, <54ac609d.ae09b40a.3e3e.5972@mx.google.com> Message-ID: <54AC627B.2060909@oracle.com> But bytecodes are supplemented by the BytecodeMapping attribute, which reify the (static) instantiated type (as outlined in the SotS writeup, and which you can see by running javap on a class compiled with the valhalla compiler.) On 1/6/2015 5:14 PM, Timo Kinnunen wrote: > The type in bytecode will be Object, but the type in source code will be V, with V=boxed complex number, for example. The idiom I?m thinking of is > > > Complex complex = numbers.get(nonexistentKey); > > if(complex == null) return; > > calculate(complex.re, complex.im); > > > > > > > (Complex is a value type, numbers is a Java 5 hash map) > > From vitalyd at gmail.com Tue Jan 6 22:35:49 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 6 Jan 2015 17:35:49 -0500 Subject: Question on layer/peeling In-Reply-To: <54AC627B.2060909@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54AC4AF8.4090709@oracle.com> <54ac4e24.104dc20a.3a08.26ca@mx.google.com> <54ac609d.ae09b40a.3e3e.5972@mx.google.com> <54AC627B.2060909@oracle.com> Message-ID: I'm confused - are we talking about an any-fied hashmap or current hashmap? If the former, the null comparison will always be false and without having a default (T)/T.null/whatever, I'd expect a box operation to occur for the null check. Sent from my phone On Jan 6, 2015 5:32 PM, "Brian Goetz" wrote: > But bytecodes are supplemented by the BytecodeMapping attribute, which > reify the (static) instantiated type (as outlined in the SotS writeup, and > which you can see by running javap on a class compiled with the valhalla > compiler.) > > On 1/6/2015 5:14 PM, Timo Kinnunen wrote: > >> The type in bytecode will be Object, but the type in source code will be >> V, with V=boxed complex number, for example. The idiom I?m thinking of is >> >> >> Complex complex = numbers.get(nonexistentKey); >> >> if(complex == null) return; >> >> calculate(complex.re, complex.im); >> >> >> >> >> >> >> (Complex is a value type, numbers is a Java 5 hash map) >> >> >> From brian.goetz at oracle.com Tue Jan 6 22:36:46 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 17:36:46 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> Message-ID: <54AC637E.1030701@oracle.com> Great progress! Can you enumerate the methods or code paths that you had to #ifref out, and why? Then it can form a sort of "scoreboard", where we can burn down on the list of accidental hacks as we improve the implementation, eventually converging on the desired (hack-free) result. On 1/6/2015 5:31 PM, Michael Barker wrote: > I suppose the workaround for #3 would be to take a Hasher (or > whatever name you pick) instance as a ctor arg; this would provide > equals/hashCode for your type T. > > Cheers, this worked. I now have a working specialised HashMap[1]. When > some of the "implementation by parts" makes an appearance, I'll rework > to see if I can remove the custom Hashers. > > [1] modulo null problems mentioned earlier and not implementing some of > the j.u.Map methods, e.g. keySet(), entrySet(). From maurizio.cimadamore at oracle.com Tue Jan 6 22:37:38 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 06 Jan 2015 22:37:38 +0000 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> Message-ID: <54AC63B2.6060304@oracle.com> On 06/01/15 20:38, Michael Barker wrote: >> The trick can also be coaxed to give semi-plausible semantics to seemingly >> bogus expressions like "x == null" or "x == (Object)y" (for any x). If "x" >> is a primitive, since autobox conversions never produce "null", then "x == >> null" and "x == (Object)y" are complicated ways of saying "false". But >> this doesn't help with nulling assignments like "a[i] = null"; there you >> need "T.null" (my bikeshed preference to "default(T)"). >> > I think allowing x == null (always false) and a[i] = T.null for value types > would certainly make the coding simpler as it would avoid layering for a > number of very common cases. In the null comparison case I would assume > that Hotspot could easily throw away the condition altogether in the > specialised class. I think we should distinguish between default value and null value; I believe null only makes sense for references - while default value makes sense in a broader sense. You don't want end up mixing the two concepts - as you would end up that i.e. int.null is 0 (zero), which happens to be quite a common value... Maurizio From vitalyd at gmail.com Tue Jan 6 22:43:10 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 6 Jan 2015 17:43:10 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54AC63B2.6060304@oracle.com> Message-ID: I agree; null should stay as a reference-only concept and specialized generic code should use default for the type. By the way, C# solves this by having TryGet type of methods that return a boolean indicating success (I.e. found mapping) and set an out parameter to the value found. You'd need a lightweight tuple or multi return to do the same in java, I think. Sent from my phone On Jan 6, 2015 5:38 PM, "Maurizio Cimadamore" < maurizio.cimadamore at oracle.com> wrote: On 06/01/15 20:38, Michael Barker wrote: > The trick can also be coaxed to give semi-plausible semantics to seemingly >> bogus expressions like "x == null" or "x == (Object)y" (for any x). If >> "x" >> is a primitive, since autobox conversions never produce "null", then "x == >> null" and "x == (Object)y" are complicated ways of saying "false". But >> this doesn't help with nulling assignments like "a[i] = null"; there you >> need "T.null" (my bikeshed preference to "default(T)"). >> >> I think allowing x == null (always false) and a[i] = T.null for value > types > would certainly make the coding simpler as it would avoid layering for a > number of very common cases. In the null comparison case I would assume > that Hotspot could easily throw away the condition altogether in the > specialised class. > I think we should distinguish between default value and null value; I believe null only makes sense for references - while default value makes sense in a broader sense. You don't want end up mixing the two concepts - as you would end up that i.e. int.null is 0 (zero), which happens to be quite a common value... Maurizio From brian.goetz at oracle.com Tue Jan 6 22:43:59 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 17:43:59 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54AC4AF8.4090709@oracle.com> <54ac4e24.104dc20a.3a08.26ca@mx.google.com> <54ac609d.ae09b40a.3e3e.5972@mx.google.com> <54AC627B.2060909@oracle.com> Message-ID: <54AC652F.10308@oracle.com> I think Timo is just a little confused here. (If it was a non-any-fied HashMap, we'd never specialize, you'd just get the today behavior, and the question is moot.) On 1/6/2015 5:35 PM, Vitaly Davidovich wrote: > I'm confused - are we talking about an any-fied hashmap or current > hashmap? If the former, the null comparison will always be false and > without having a default (T)/T.null/whatever, I'd expect a box operation > to occur for the null check. > > Sent from my phone > > On Jan 6, 2015 5:32 PM, "Brian Goetz" > wrote: > > But bytecodes are supplemented by the BytecodeMapping attribute, > which reify the (static) instantiated type (as outlined in the SotS > writeup, and which you can see by running javap on a class compiled > with the valhalla compiler.) > > On 1/6/2015 5:14 PM, Timo Kinnunen wrote: > > The type in bytecode will be Object, but the type in source code > will be V, with V=boxed complex number, for example. The idiom > I?m thinking of is > > > Complex complex = numbers.get(nonexistentKey); > > if(complex == null) return; > > calculate(complex.re , complex.im > ); > > > > > > > (Complex is a value type, numbers is a Java 5 hash map) > > From brian.goetz at oracle.com Tue Jan 6 22:45:20 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 17:45:20 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54AC63B2.6060304@oracle.com> Message-ID: <54AC6580.3070606@oracle.com> > By the way, C# solves this by having TryGet type of methods that return a > boolean indicating success (I.e. found mapping) and set an out parameter to > the value found. You'd need a lightweight tuple or multi return to do the > same in java, I think. Right, there's two ways to fix get(): - Using the getOrDefault(key, sentinel) // added in Java 8 - Using some sort of optional-like return type Not quite at that part of the road yet. From vitalyd at gmail.com Tue Jan 6 22:49:44 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 6 Jan 2015 17:49:44 -0500 Subject: Question on layer/peeling In-Reply-To: <54AC6580.3070606@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54AC63B2.6060304@oracle.com> <54AC6580.3070606@oracle.com> Message-ID: I thought about getOrDefault as well, but the problem is that if you want to get a value *and* find out if it was present in 1 operation, this won't work. For cases where you don't care if the value was present or not and proceeding with default is just as good as getting it from the map, no problem, but that would be a limitation. On Tue, Jan 6, 2015 at 5:45 PM, Brian Goetz wrote: > By the way, C# solves this by having TryGet type of methods that return a >> boolean indicating success (I.e. found mapping) and set an out parameter >> to >> the value found. You'd need a lightweight tuple or multi return to do the >> same in java, I think. >> > > Right, there's two ways to fix get(): > - Using the getOrDefault(key, sentinel) // added in Java 8 > - Using some sort of optional-like return type > > Not quite at that part of the road yet. > > From timo.kinnunen at gmail.com Tue Jan 6 22:52:09 2015 From: timo.kinnunen at gmail.com (=?utf-8?Q?Timo_Kinnunen?=) Date: Tue, 6 Jan 2015 22:52:09 +0000 Subject: =?utf-8?Q?Re:_Question_on_layer/peeling?= In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54ac4e24.104dc20a.3a08.26ca@mx.google.com> <54ac609d.ae09b40a.3e3e.5972@mx.google.com>, Message-ID: <54ac6864.3202c20a.49f8.72f0@mx.google.com> That is my question as well. If Complex was a reference type that code would be null-safe. It would be weird if only changing Complex into a value type caused that code to start throwing NPEs and have the if-statement become dead code. -- Have a nice day, Timo. Sent from Windows Mail From: Vitaly Davidovich Sent: ?Tuesday?, ?January? ?6?, ?2015 ?23?:?32 To: Timo Kinnunen Cc: John Rose, valhalla-dev at openjdk.java.net Wouldn't you get an NPE here since you'd be auto unboxing a null into a value type? I'm a bit confused about what you're trying to demonstrate. Sent from my phone On Jan 6, 2015 5:24 PM, "Timo Kinnunen" wrote: The type in bytecode will be Object, but the type in source code will be V, with V=boxed complex number, for example. The idiom I?m thinking of is Complex complex = numbers.get(nonexistentKey); if(complex == null) return; calculate(complex.re, complex.im); (Complex is a value type, numbers is a Java 5 hash map) -- Have a nice day, Timo. Sent from Windows Mail From: John Rose Sent: ?Tuesday?, ?January? ?6?, ?2015 ?23?:?09 To: Timo Kinnunen Cc: Brian Goetz, Michael Barker, valhalla-dev at openjdk.java.net On Jan 6, 2015, at 1:00 PM, Timo Kinnunen wrote: > > Before assigning semantics to x == null, let?s not forget boxing. What will happen if x is a boxed value which was returned from a call to the non-specialized Map.get(Object key) method? ? Then the comparison will be false. In any case the type of x there is Object, isn't it, so there's nothing new here? ? John From mikeb01 at gmail.com Tue Jan 6 23:04:32 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Wed, 7 Jan 2015 12:04:32 +1300 Subject: Question on layer/peeling In-Reply-To: <54AC637E.1030701@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AC637E.1030701@oracle.com> Message-ID: > > Great progress! Can you enumerate the methods or code paths that you had > to #ifref out, and why? Then it can form a sort of "scoreboard", where we > can burn down on the list of accidental hacks as we improve the > implementation, eventually converging on the desired (hack-free) result. What is '#ifref'? Is that a compiler directive? If so the answer is none. The only real issues were the ones mentioned previously. The code is up on github[1], note that it is not a conversion of java.util.Map, but a conversion of an open addressed hash map similar to one that we use internally (specifically it is a port of the one from the Agrona project[2]). One of the interesting aspects of the implementation is that I have to use a BitSet rather than relying on null to determine if a value is present or now. I'd replace this with an any-ified value type for the key or value once value types are ready. [1] https://github.com/mikeb01/SpecialisedMap/blob/master/src/github/mikeb01/SpecializedOpenHashMap.java [2] https://github.com/real-logic/Agrona/blob/master/src/main/java/uk/co/real_logic/agrona/collections/Long2ObjectHashMap.java From brian.goetz at oracle.com Tue Jan 6 23:09:03 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 06 Jan 2015 18:09:03 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AC637E.1030701@oracle.com> Message-ID: <54AC6B0F.7070900@oracle.com> Sorry, that's the internal joke-name of "__WhereRef(T)". On 1/6/2015 6:04 PM, Michael Barker wrote: > Great progress! Can you enumerate the methods or code paths that > you had to #ifref out, and why? Then it can form a sort of > "scoreboard", where we can burn down on the list of accidental hacks > as we improve the implementation, eventually converging on the > desired (hack-free) result. > > > What is '#ifref'? Is that a compiler directive? If so the answer is > none. The only real issues were the ones mentioned previously. > > The code is up on github[1], note that it is not a conversion of > java.util.Map, but a conversion of an open addressed hash map similar to > one that we use internally (specifically it is a port of the one from > the Agrona project[2]). One of the interesting aspects of the > implementation is that I have to use a BitSet rather than relying on > null to determine if a value is present or now. I'd replace this with > an any-ified value type for the key or value once value types are ready. > > [1] > https://github.com/mikeb01/SpecialisedMap/blob/master/src/github/mikeb01/SpecializedOpenHashMap.java > [2] > https://github.com/real-logic/Agrona/blob/master/src/main/java/uk/co/real_logic/agrona/collections/Long2ObjectHashMap.java From vitalyd at gmail.com Tue Jan 6 23:10:31 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 6 Jan 2015 18:10:31 -0500 Subject: Question on layer/peeling In-Reply-To: <54ac6864.3202c20a.49f8.72f0@mx.google.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54ac4e24.104dc20a.3a08.26ca@mx.google.com> <54ac609d.ae09b40a.3e3e.5972@mx.google.com> <54ac6864.3202c20a.49f8.72f0@mx.google.com> Message-ID: So you'll have this issue if you change your class to be a value type and yet continue to use non any-fied generic classes. I think it's a pipe dream if anyone thinks they'll simply change their class to be a value type, and everything will be peachy. Even if core JDK collection classes get any-fied, you'll still have to make adjustments, such as moving over to the methods that have multi-return (whichever approach Brian & co pick); continuing to use get() will most likely not work properly. More likely, people will have to introduce new value types that mirror their old classes, update their APIs to work with specialized generics properly, deprecate the old classes, and slowly move away from them. Also, in my experience on the .NET platform, there aren't that many value types as a proportion of all types; certain domains (e.g. graphics, numerics) will have higher counts of them, but most "general purpose" apps do not use them heavily (custom structs, that is, I'm not counting the fact that primitives in .NET are actually represented as value types). I suspect the same will be true in java; it's not like all of a sudden everything will be a value type. So for those types that are a good fit, a careful and thought out migration plan will be needed. On Tue, Jan 6, 2015 at 5:52 PM, Timo Kinnunen wrote: > That is my question as well. If Complex was a reference type that code > would be null-safe. It would be weird if only changing Complex into a value > type caused that code to start throwing NPEs and have the if-statement > become dead code. > > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > *From:* Vitaly Davidovich > *Sent:* ?Tuesday?, ?January? ?6?, ?2015 ?23?:?32 > *To:* Timo Kinnunen > *Cc:* John Rose , valhalla-dev at openjdk.java.net > > Wouldn't you get an NPE here since you'd be auto unboxing a null into a > value type? I'm a bit confused about what you're trying to demonstrate. > > Sent from my phone > On Jan 6, 2015 5:24 PM, "Timo Kinnunen" wrote: > >> The type in bytecode will be Object, but the type in source code will be >> V, with V=boxed complex number, for example. The idiom I?m thinking of is >> >> >> Complex complex = numbers.get(nonexistentKey); >> >> if(complex == null) return; >> >> calculate(complex.re, complex.im); >> >> >> >> >> >> >> (Complex is a value type, numbers is a Java 5 hash map) >> >> >> -- >> Have a nice day, >> Timo. >> >> Sent from Windows Mail >> >> >> >> >> >> From: John Rose >> Sent: ?Tuesday?, ?January? ?6?, ?2015 ?23?:?09 >> To: Timo Kinnunen >> Cc: Brian Goetz, Michael Barker, valhalla-dev at openjdk.java.net >> >> >> >> >> >> On Jan 6, 2015, at 1:00 PM, Timo Kinnunen >> wrote: >> > >> > Before assigning semantics to x == null, let?s not forget boxing. What >> will happen if x is a boxed value which was returned from a call to the >> non-specialized Map.get(Object key) method? >> >> ? Then the comparison will be false. In any case the type of x there is >> Object, isn't it, so there's nothing new here? ? John > > From john.r.rose at oracle.com Tue Jan 6 23:25:42 2015 From: john.r.rose at oracle.com (John Rose) Date: Tue, 6 Jan 2015 15:25:42 -0800 Subject: Question on layer/peeling In-Reply-To: <54AC63B2.6060304@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54AC63B2.6060304@oracle.com> Message-ID: <9A3CF9B3-6054-4034-A19E-87E5AEFFC331@oracle.com> On Jan 6, 2015, at 2:37 PM, Maurizio Cimadamore wrote: > On 06/01/15 20:38, Michael Barker wrote: >>> The trick can also be coaxed to give semi-plausible semantics to seemingly >>> bogus expressions like "x == null" or "x == (Object)y" (for any x). If "x" >>> is a primitive, since autobox conversions never produce "null", then "x == >>> null" and "x == (Object)y" are complicated ways of saying "false". But >>> this doesn't help with nulling assignments like "a[i] = null"; there you >>> need "T.null" (my bikeshed preference to "default(T)"). >>> >> I think allowing x == null (always false) and a[i] = T.null for value types >> would certainly make the coding simpler as it would avoid layering for a >> number of very common cases. In the null comparison case I would assume >> that Hotspot could easily throw away the condition altogether in the >> specialised class. > I think we should distinguish between default value and null value; Totally agree; the concepts cannot be merged. (The bikeshed colors may be similar or different; it doesn't matter much.) > I believe null only makes sense for references Totally true for new code. My only point is that "honorary promotion of T to Object" might help retrofit some old code that was written for Objects. In a DWIM-my sort of way. This POV is consistent with having warnings like "x == null did you really mean x == T.null or Optional.none()?". Throwing static typing errors (with similar messages) would of course force the "did you really mean" question to happen earlier. > - while default value makes sense in a broader sense. You don't want end up mixing the two concepts - as you would end up that i.e. int.null is 0 (zero), which happens to be quite a common value... Yes. We are a long way from the place where (int)null gets forced to zero, and we shouldn't go there now. (I don't argue bikeshed colors, and perhaps you don't like mine, which is OK!) ? John From valhalla at jinq.org Wed Jan 7 04:35:00 2015 From: valhalla at jinq.org (Ming-Yee Iu) Date: Tue, 6 Jan 2015 23:35:00 -0500 Subject: Alternatives to Value Types? Message-ID: Stephen Colbourne might have a point about these value types. Are there alternatives to value types that give fewer benefits but don't require such large Java changes? I'm sure you've considered it before and dismissed it, but maybe instead of value types, we could have value "fields"? I have no idea what I'm saying, but maybe instead of "No Identity" being an annotation on types, "No Identity" could be an annotation added to fields? class NoIdentityContainer { @NoIdentity_CopyByValue_HaveGoodLocality T field; } Creating an instance of the class will also create an instance of the T object with locality near to the container instance. Assigning to the field will actually result in a copy of the contents over the existing instance instead of just changing the pointer. Maybe the annotation could be applied to method parameters and return values as well. Objects stay as objects. Primitives stay as primitives. The only classes that have to deal with new "value" semantics are those classes that explicitly use the annotation. Things still break, but in a different way that might be more palatable. From rednaxelafx at gmail.com Wed Jan 7 06:42:06 2015 From: rednaxelafx at gmail.com (Krystal Mok) Date: Tue, 6 Jan 2015 22:42:06 -0800 Subject: Alternatives to Value Types? In-Reply-To: References: Message-ID: Hi Ming-Yee, The idea of annotation on a field for "embedding objects" or "colocating objects" has cropped up once in a while. A similar idea goes like: class Foo { @Value private AtomicLong val = new AtomicLong(0); // "embed" the AtomicLong in Foo } Anyway, I'd like to point out that the ObjectLayout [1] project actually takes care of a concept similar to yours, but retains full object semantics. - Kris [1]: http://objectlayout.org/ On Tue, Jan 6, 2015 at 8:35 PM, Ming-Yee Iu wrote: > Stephen Colbourne might have a point about these value types. Are there > alternatives to value types that give fewer benefits but don't require such > large Java changes? > > I'm sure you've considered it before and dismissed it, but maybe instead of > value types, we could have value "fields"? I have no idea what I'm saying, > but maybe instead of "No Identity" being an annotation on types, "No > Identity" could be an annotation added to fields? > > class NoIdentityContainer { > @NoIdentity_CopyByValue_HaveGoodLocality T field; > } > > Creating an instance of the class will also create an instance of the T > object with locality near to the container instance. Assigning to the field > will actually result in a copy of the contents over the existing instance > instead of just changing the pointer. Maybe the annotation could be applied > to method parameters and return values as well. Objects stay as objects. > Primitives stay as primitives. The only classes that have to deal with new > "value" semantics are those classes that explicitly use the annotation. > Things still break, but in a different way that might be more palatable. > From ali.ebrahimi1781 at gmail.com Wed Jan 7 08:03:04 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Wed, 7 Jan 2015 11:33:04 +0330 Subject: Question on layer/peeling In-Reply-To: <54AC6580.3070606@oracle.com> References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54AC63B2.6060304@oracle.com> <54AC6580.3070606@oracle.com> Message-ID: Or add method Map.Entry getEntry(key) On Wed, Jan 7, 2015 at 2:15 AM, Brian Goetz wrote: > By the way, C# solves this by having TryGet type of methods that return a >> boolean indicating success (I.e. found mapping) and set an out parameter >> to >> the value found. You'd need a lightweight tuple or multi return to do the >> same in java, I think. >> > > Right, there's two ways to fix get(): > - Using the getOrDefault(key, sentinel) // added in Java 8 > - Using some sort of optional-like return type > > Not quite at that part of the road yet. > > -- Best Regards, Ali Ebrahimi From maurizio.cimadamore at oracle.com Wed Jan 7 10:42:52 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 07 Jan 2015 10:42:52 +0000 Subject: Alternatives to Value Types? In-Reply-To: References: Message-ID: <54AD0DAC.1010707@oracle.com> Hi, note that locality of fields is just one of the many problems. A perhaps bigger one is how instances of V (where V is a value class) are laid out in an array, as that tremendously affects performances; if every cell of the array is merely a pointer to some other heap location (where the instance is actually stored), then locality is thrown out of the window. The idea behind current proposal is that, as instances of value classes are kind of ephemeral, they can be 'inlined' directly inside the array - which means no long jump to reach the array elements. If I understand your proposal correctly, being @Value a property of the field rather than one of the class, means that it won't be possible i.e. to store instances of 'Foo' as mere blobs of bytes inside an array. Of course, if you put in place the right set of restriction, you might be able to get there - but it seems to be that those restrictions (most notably, the fact that 'Foo' must be monomorphic) apply more to the concept of class than that of the field. Maurizio On 07/01/15 06:42, Krystal Mok wrote: > Hi Ming-Yee, > > The idea of annotation on a field for "embedding objects" or "colocating > objects" has cropped up once in a while. > > A similar idea goes like: > > class Foo { > @Value private AtomicLong val = new AtomicLong(0); // "embed" the > AtomicLong in Foo > } > > Anyway, I'd like to point out that the ObjectLayout [1] project actually > takes care of a concept similar to yours, but retains full object semantics. > > - Kris > > [1]: http://objectlayout.org/ > > On Tue, Jan 6, 2015 at 8:35 PM, Ming-Yee Iu wrote: > >> Stephen Colbourne might have a point about these value types. Are there >> alternatives to value types that give fewer benefits but don't require such >> large Java changes? >> >> I'm sure you've considered it before and dismissed it, but maybe instead of >> value types, we could have value "fields"? I have no idea what I'm saying, >> but maybe instead of "No Identity" being an annotation on types, "No >> Identity" could be an annotation added to fields? >> >> class NoIdentityContainer { >> @NoIdentity_CopyByValue_HaveGoodLocality T field; >> } >> >> Creating an instance of the class will also create an instance of the T >> object with locality near to the container instance. Assigning to the field >> will actually result in a copy of the contents over the existing instance >> instead of just changing the pointer. Maybe the annotation could be applied >> to method parameters and return values as well. Objects stay as objects. >> Primitives stay as primitives. The only classes that have to deal with new >> "value" semantics are those classes that explicitly use the annotation. >> Things still break, but in a different way that might be more palatable. >> From twhitmore.nz at gmail.com Wed Jan 7 11:14:40 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Thu, 8 Jan 2015 00:14:40 +1300 Subject: 'Equals' in any-typed code: VERY BIG PROPOSAL Message-ID: Hi John, Maurizio, people Thanks for your comments on equals/ null. > You are right that the "==" operator has some very sharp edges when we try to unify references and primitives. (So does "+".) > On references, it is often used invalidly, and has valid uses for null checks, short-cutting calls to x.equals(y), and work with interned types (like enums). Equality checking across values/ reftypes really would benefit from a clean syntax. I feel it is a hack to be writing a null- check, and we should avoid encouraging null to be written in 'any' typed code! Now, am I going to ask you to marry me? No, but I am going to propose an 'eq' operator. Into the Java language. As such: if (node.getKey() eq key) { if (name eq "") { if (product.getColor() eq color) { As well as solving equals for 'any', such an operator would address one of the biggest pain points for millions of application developers working in Java. Millions would cheer, and the world would thank us. I would also consider a 'neq' not-equal operator. These one or two operators are of such benefit & as by far the most important, can easily be justified to stand on their own. if (value neq other.value) { Clean scope, no need to bring in other operators/ comparisons, just a compact & benefical feature.. Could this be the right time? We have a real requirement, an opportunity for long-term benefit, and a release where bold changes can be accepted. Regards Thomas Whitmore From maurizio.cimadamore at oracle.com Wed Jan 7 11:25:44 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 07 Jan 2015 11:25:44 +0000 Subject: Fwd: java.lang.VerifyError: Bad type on operand stack In-Reply-To: References: <54ABFAC8.5000308@oracle.com> Message-ID: <54AD17B8.8050604@oracle.com> I consulted with the inference Gods (they seem to be lurking in Valhalla from time to time) and it would seem that this is actually the correct behavior. The problem is that the constructor is considered perfectly applicable during the 'strict' applicability step, which only uses subtyping - therefore the only allowed inference result there is 'int' - and the target type seems to contradict there. It is possible that we will relax this a bit moving forward by introducing some ad hoc rules, but the status quo seems to be expressive enough, while avoiding being unnecessarily messy. At the same time, the Gods pointed out that inference should only be allowed to do boxing/unboxing tricks when an inference constraints of the kind alpha :> int has been derived from a compatibility check (i.e. because we want to pass an 'int' where 'alpha' is expected). These tricks should not be allowed in the general case - example: T[] clone(T[] arg) { return arg; } int[] arr1 = clone(new int[3]); //ok Integer[] arr2 = clone(new int[3]); //fail Here, the array subtyping rules essentially mean that no boxing can occurr, so the only choice for T is 'int' in the second case, which, again, is incompatible with the target. The compiler gets some of this right and some other things only happen to be right 'by accident' - I'm working on a patch that will clean this up a bit. Maurizio On 06/01/15 16:27, Sven Reimers wrote: > @Maurizio: sorry forgot the mailing list... > > > ---------- Forwarded message ---------- > From: Sven Reimers > Date: Tue, Jan 6, 2015 at 5:20 PM > Subject: Re: java.lang.VerifyError: Bad type on operand stack > To: Maurizio Cimadamore > > > Follow-Up: > > I tried a few different things and got into this: > > Point3D integerPoint = new Point3D<>(1,2,3); > > compiles only if > > public static class Point3D {... } > > not for > > public static class Point3D {... } > > ErrorMessage from the compiler is: > > incompatible types: cannot infer type arguments for Point3D<> > Point3D integerPoint = new Point3D<>(1,2,3); > ^ > reason: cannot infer type-variable(s) T > (argument mismatch; int cannot be converted to Integer) > where T is a type-variable: > T extends declared in class Point3D > > Is this intentional (not sure I found this stated in the draft)? > > Thanks > > Sven > > > On Tue, Jan 6, 2015 at 4:10 PM, Maurizio Cimadamore < > maurizio.cimadamore at oracle.com> wrote: > >> Whoops :-) >> >> Thanks for the report >> >> Maurizio >> >> >> On 06/01/15 15:13, Sven Reimers wrote: >> >>> Hi, >>> >>> as suggested by Brian I tried to get some sample code written and running >>> on top of valhalla. >>> >>> First problem I encountered was that a missing diamond <> on the right >>> hand >>> side (my bad - seems I am really relying on my IDE) was just a warning >>> that >>> I was using a raw type. >>> >>> Running my example I got this: >>> >>> Specializing Valhalla$Point3D${0=I}; searching for Valhalla$Point3D.class >>> (not found) >>> Specializing Valhalla$Point3D${0=I}; searching for Valhalla$Point3D.class >>> (found) >>> Error: A JNI error has occurred, please check your installation and try >>> again >>> Exception in thread "main" java.lang.VerifyError: Bad type on operand >>> stack >>> Exception Details: >>> Location: >>> Valhalla.main([Ljava/lang/String;)V @36: invokevirtual >>> Reason: >>> Type 'Valhalla$Point3D' (current frame, stack[2]) is not assignable >>> to >>> 'Valhalla$Point3D${0=I}' >>> Current Frame: >>> bci: @36 >>> flags: { } >>> locals: { '[Ljava/lang/String;', 'Valhalla$Point3D' } >>> stack: { 'java/io/PrintStream', 'java/lang/StringBuilder', >>> 'Valhalla$Point3D' } >>> Bytecode: >>> 0000000: bb00 0259 04b8 0003 05b8 0003 06b8 0003 >>> 0000010: b700 044c b200 05bb 0006 59b7 0007 1208 >>> 0000020: b600 092b b600 0ab6 000b b600 0cb6 000d >>> 0000030: b1 >>> >>> at java.lang.Class.getDeclaredMethods0(Native Method) >>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2704) >>> at java.lang.Class.privateGetMethodRecursive(Class.java:3049) >>> at java.lang.Class.getMethod0(Class.java:3019) >>> at java.lang.Class.getMethod(Class.java:1787) >>> at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:568) >>> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:535) >>> >>> Example Code (see the missing <> just after Point3D...): >>> >>> public class Valhalla { >>> >>> >>> public static void main (String[] args) { >>> >>> Point3D intPoint = new Point3D(1,2,3); >>> >>> System.out.println("X: " + intPoint.getX()); >>> >>> } >>> >>> public static class Point3D { >>> private T x; >>> private T y; >>> private T z; >>> >>> public Point3D(T x, T y, T z) { >>> this.x = x; >>> this.y = y; >>> this.z = z; >>> } >>> >>> public T getX() { >>> return x; >>> } >>> >>> public T getY() { >>> return y; >>> } >>> >>> public T getZ() { >>> return z; >>> } >>> >>> } >>> >>> } >>> >>> Hope to provide more feedback once I got more code running. >>> >>> Thanks for the impressive work so far. >>> >>> -Sven >>> >>> > From martijnverburg at gmail.com Wed Jan 7 12:51:13 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Wed, 7 Jan 2015 12:51:13 +0000 Subject: 'Equals' in any-typed code: VERY BIG PROPOSAL In-Reply-To: References: Message-ID: Hi Thomas, If you're keen to explore this space I'd recommend taking a look at what it would take to implement these proposed operators and explore their interaction with existing ==/!= and equals() mechanics and how it would all work when dealing with primitives, boxed objects, arrays, nulls and general objects with/without a .equals() defined, not to mention the proposed 'value type' construct.... I know it's tempting to look at these issues from the "Java the language" perspective (with some pseudo syntax), I've fallen for this trap in the past as well! It's infinitely better to propose these ideas from the view of the JVM and it's internal mechanics, which I appreciate takes a lot of time and effort :-). Cheers, Martijn On 7 January 2015 at 11:14, Thomas W wrote: > Hi John, Maurizio, people > > Thanks for your comments on equals/ null. > > > You are right that the "==" operator has some very sharp edges when we > try to unify references and primitives. (So does "+".) > > On references, it is often used invalidly, and has valid uses for null > checks, short-cutting calls to x.equals(y), and work with interned types > (like enums). > > Equality checking across values/ reftypes really would benefit from a clean > syntax. > > I feel it is a hack to be writing a null- check, and we should avoid > encouraging null to be written in 'any' typed code! > > Now, am I going to ask you to marry me? > > No, but I am going to propose an 'eq' operator. Into the Java language. As > such: > > if (node.getKey() eq key) { > if (name eq "") { > if (product.getColor() eq color) { > > As well as solving equals for 'any', such an operator would address one of > the biggest pain points for millions of application developers working in > Java. > > Millions would cheer, and the world would thank us. > > I would also consider a 'neq' not-equal operator. These one or two > operators are of such benefit & as by far the most important, can easily be > justified to stand on their own. > > if (value neq other.value) { > > Clean scope, no need to bring in other operators/ comparisons, just a > compact & benefical feature.. > > Could this be the right time? We have a real requirement, an opportunity > for long-term benefit, and a release where bold changes can be accepted. > > Regards > Thomas Whitmore > From simon at ochsenreither.de Wed Jan 7 13:21:33 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Wed, 07 Jan 2015 14:21:33 +0100 Subject: 'Equals' in any-typed code: VERY BIG PROPOSAL In-Reply-To: References: Message-ID: <54AD32DD.6030100@ochsenreither.de> > No, but I am going to propose an 'eq' operator. Into the Java language. As > such: > > if (node.getKey() eq key) { > if (name eq "") { > if (product.getColor() eq color) { > > As well as solving equals for 'any', such an operator would address one of > the biggest pain points for millions of application developers working in > Java. a) What are the actual semantics? b) How is this different to the approaches already outlines earlier? From maurizio.cimadamore at oracle.com Wed Jan 7 13:21:58 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 07 Jan 2015 13:21:58 +0000 Subject: hg: valhalla/valhalla/langtools: Misc type-checking fixes: Message-ID: <201501071321.t07DLwBt018980@aojmv0008> Changeset: 5afa3cddd11e Author: mcimadamore Date: 2015-01-07 13:21 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/5afa3cddd11e Misc type-checking fixes: * Disallow specialized -> raw unchecked conversion * Iron out inference rules for bound boxing logic * Types.isSpecialized erroneously returns 'true' for unbounded wildcard-parameterized types * Refactor code in Attr relying on Types.isSpecialized * Improve inference diagnostics by suppressing synthetic tvar bounds * Add tests ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Infer.java ! test/tools/javac/valhalla/typespec/Inference01.out + test/tools/javac/valhalla/typespec/Inference05.java + test/tools/javac/valhalla/typespec/Inference05.out + test/tools/javac/valhalla/typespec/Raw.java + test/tools/javac/valhalla/typespec/Raw.out From vitalyd at gmail.com Wed Jan 7 13:38:15 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 08:38:15 -0500 Subject: 'Equals' in any-typed code: VERY BIG PROPOSAL In-Reply-To: References: Message-ID: Why do we need a new special operator? What's wrong with using equals () and having the specializer rewrite that for primitives? For other types (refs and custom value types), it should just delegate to the real equals. Sent from my phone On Jan 7, 2015 6:15 AM, "Thomas W" wrote: > Hi John, Maurizio, people > > Thanks for your comments on equals/ null. > > > You are right that the "==" operator has some very sharp edges when we > try to unify references and primitives. (So does "+".) > > On references, it is often used invalidly, and has valid uses for null > checks, short-cutting calls to x.equals(y), and work with interned types > (like enums). > > Equality checking across values/ reftypes really would benefit from a clean > syntax. > > I feel it is a hack to be writing a null- check, and we should avoid > encouraging null to be written in 'any' typed code! > > Now, am I going to ask you to marry me? > > No, but I am going to propose an 'eq' operator. Into the Java language. As > such: > > if (node.getKey() eq key) { > if (name eq "") { > if (product.getColor() eq color) { > > As well as solving equals for 'any', such an operator would address one of > the biggest pain points for millions of application developers working in > Java. > > Millions would cheer, and the world would thank us. > > I would also consider a 'neq' not-equal operator. These one or two > operators are of such benefit & as by far the most important, can easily be > justified to stand on their own. > > if (value neq other.value) { > > Clean scope, no need to bring in other operators/ comparisons, just a > compact & benefical feature.. > > Could this be the right time? We have a real requirement, an opportunity > for long-term benefit, and a release where bold changes can be accepted. > > Regards > Thomas Whitmore > From peter.levart at gmail.com Wed Jan 7 13:43:27 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 07 Jan 2015 14:43:27 +0100 Subject: alternatives or complements to layers Message-ID: <54AD37FF.9060301@gmail.com> Hi, While experimenting a bit with the current preliminary prototype, I found myself on several occasions wanting to call a method from "normal" API. Say System.out.println(x) with an x of type "any T". Usually a method that already has various overloads or a method taking an Object. Javac refuses: method PrintStream.println(boolean) is not applicable (argument mismatch; T cannot be converted to boolean) method PrintStream.println(char) is not applicable (argument mismatch; T cannot be converted to char) method PrintStream.println(int) is not applicable (argument mismatch; T cannot be converted to int) method PrintStream.println(long) is not applicable (argument mismatch; T cannot be converted to long) method PrintStream.println(float) is not applicable (argument mismatch; T cannot be converted to float) method PrintStream.println(double) is not applicable (argument mismatch; T cannot be converted to double) method PrintStream.println(char[]) is not applicable (argument mismatch; T cannot be converted to char[]) method PrintStream.println(String) is not applicable (argument mismatch; T cannot be converted to String) method PrintStream.println(Object) is not applicable (argument mismatch; T cannot be converted to Object) where T is a type-variable: T extends declared in method test() But we know that whatever instantiation of T we choose, at least one of available overloaded methods would apply. If the set of overloaded methods contains one with parameter of type Object, then any instantiation of parameter of type "any T" would apply to at least one of those methods. At specialization time, the most appropriate method could be "selected". Of course this would not automatically route reference typed parameter to the most appropriate method among those taking various reference types - the one taking Object would always be chosen, but any value typed paremeter could select the most appropriate method (with implicit conversions and/or boxing). This could be viewed as a more natural complement to "layers" for supporting implementation by parts. The specializer would either have to inspect the target class somehow, or javac could provide the set of applicable methods in an attribute associated with invocation. Or perhaps, such invocation could be "specialized" as invokedynamic where the linking to the most appropriate method (with implicit conversions and/or boxing) would be choosen at 1st invocation. What do you think? Regards, Peter From vitalyd at gmail.com Wed Jan 7 13:44:45 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 08:44:45 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54AC63B2.6060304@oracle.com> <54AC6580.3070606@oracle.com> Message-ID: Not sure that's ideal as the implementation may not be using entry objects to store things (e.g. open addressed version with parallel arrays for keys and values); this would cause the get method to allocate an entry for each get operation. If the entry impl is a value type it'll be boxed since it's assigned to an interface. Sent from my phone On Jan 7, 2015 3:03 AM, "Ali Ebrahimi" wrote: > Or add method > Map.Entry getEntry(key) > > On Wed, Jan 7, 2015 at 2:15 AM, Brian Goetz > wrote: > >> By the way, C# solves this by having TryGet type of methods that return a >>> boolean indicating success (I.e. found mapping) and set an out parameter >>> to >>> the value found. You'd need a lightweight tuple or multi return to do >>> the >>> same in java, I think. >>> >> >> Right, there's two ways to fix get(): >> - Using the getOrDefault(key, sentinel) // added in Java 8 >> - Using some sort of optional-like return type >> >> Not quite at that part of the road yet. >> >> > > > -- > > Best Regards, > Ali Ebrahimi > From vitalyd at gmail.com Wed Jan 7 13:54:59 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 08:54:59 -0500 Subject: alternatives or complements to layers In-Reply-To: <54AD37FF.9060301@gmail.com> References: <54AD37FF.9060301@gmail.com> Message-ID: The other option is to fail compilation if any-T context is calling a non any-fied method and require user to do a cast on their T to select the proper overload (including possibly casting to Object). I don't know if we want the method "late bound" like that. Also, suppose the different overloads return different types - the writer of code has to know what the type will be upfront. Sent from my phone On Jan 7, 2015 8:44 AM, "Peter Levart" wrote: > Hi, > > While experimenting a bit with the current preliminary prototype, I found > myself on several occasions wanting to call a method from "normal" API. Say > System.out.println(x) with an x of type "any T". Usually a method that > already has various overloads or a method taking an Object. Javac refuses: > > method PrintStream.println(boolean) is not applicable > (argument mismatch; T cannot be converted to boolean) > method PrintStream.println(char) is not applicable > (argument mismatch; T cannot be converted to char) > method PrintStream.println(int) is not applicable > (argument mismatch; T cannot be converted to int) > method PrintStream.println(long) is not applicable > (argument mismatch; T cannot be converted to long) > method PrintStream.println(float) is not applicable > (argument mismatch; T cannot be converted to float) > method PrintStream.println(double) is not applicable > (argument mismatch; T cannot be converted to double) > method PrintStream.println(char[]) is not applicable > (argument mismatch; T cannot be converted to char[]) > method PrintStream.println(String) is not applicable > (argument mismatch; T cannot be converted to String) > method PrintStream.println(Object) is not applicable > (argument mismatch; T cannot be converted to Object) > where T is a type-variable: > T extends declared in method test() > > > But we know that whatever instantiation of T we choose, at least one of > available overloaded methods would apply. If the set of overloaded methods > contains one with parameter of type Object, then any instantiation of > parameter of type "any T" would apply to at least one of those methods. At > specialization time, the most appropriate method could be "selected". > > Of course this would not automatically route reference typed parameter to > the most appropriate method among those taking various reference types - > the one taking Object would always be chosen, but any value typed paremeter > could select the most appropriate method (with implicit conversions and/or > boxing). > > This could be viewed as a more natural complement to "layers" for > supporting implementation by parts. > > The specializer would either have to inspect the target class somehow, or > javac could provide the set of applicable methods in an attribute > associated with invocation. Or perhaps, such invocation could be > "specialized" as invokedynamic where the linking to the most appropriate > method (with implicit conversions and/or boxing) would be choosen at 1st > invocation. > > What do you think? > > > Regards, Peter > > From simon at ochsenreither.de Wed Jan 7 14:03:36 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Wed, 07 Jan 2015 15:03:36 +0100 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> Message-ID: <54AD3CB8.6020905@ochsenreither.de> > The other option is to fail compilation if any-T context is calling a non > any-fied method and require user to do a cast on their T to select the > proper overload (including possibly casting to Object). I don't know if we > want the method "late bound" like that. Also, suppose the different > overloads return different types - the writer of code has to know what the > type will be upfront. I think the interesting question is "is there a reason why an any-fied parameter couldn't act as a compatible replacement for methods with Object + primitive overloads?". Because that's what any does under the hood already: Create additional methods for non-reference types. From vitalyd at gmail.com Wed Jan 7 14:08:27 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 09:08:27 -0500 Subject: alternatives or complements to layers In-Reply-To: <54AD3CB8.6020905@ochsenreither.de> References: <54AD37FF.9060301@gmail.com> <54AD3CB8.6020905@ochsenreither.de> Message-ID: If you're authoring a generic class and call one of these overloaded methods, which one is called? What's the return value (for non void ones)? The final target method is only known when a user instantiates your class and provides the type, but not at authorship. Sent from my phone On Jan 7, 2015 9:04 AM, "Simon Ochsenreither" wrote: > > The other option is to fail compilation if any-T context is calling a non > > any-fied method and require user to do a cast on their T to select the > > proper overload (including possibly casting to Object). I don't know if > we > > want the method "late bound" like that. Also, suppose the different > > overloads return different types - the writer of code has to know what > the > > type will be upfront. > > I think the interesting question is "is there a reason why an any-fied > parameter couldn't act as a compatible replacement for methods with > Object + primitive overloads?". Because that's what any does under the > hood already: Create additional methods for non-reference types. > From ali.ebrahimi1781 at gmail.com Wed Jan 7 14:14:28 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Wed, 7 Jan 2015 17:44:28 +0330 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54AC63B2.6060304@oracle.com> <54AC6580.3070606@oracle.com> Message-ID: You already have to implement Set> entrySet() On Wed, Jan 7, 2015 at 5:14 PM, Vitaly Davidovich wrote: > Not sure that's ideal as the implementation may not be using entry objects > to store things (e.g. open addressed version with parallel arrays for keys > and values); this would cause the get method to allocate an entry for each > get operation. If the entry impl is a value type it'll be boxed since it's > assigned to an interface. > > Sent from my phone > On Jan 7, 2015 3:03 AM, "Ali Ebrahimi" wrote: > >> Or add method >> Map.Entry getEntry(key) >> >> On Wed, Jan 7, 2015 at 2:15 AM, Brian Goetz >> wrote: >> >>> By the way, C# solves this by having TryGet type of methods that return a >>>> boolean indicating success (I.e. found mapping) and set an out >>>> parameter to >>>> the value found. You'd need a lightweight tuple or multi return to do >>>> the >>>> same in java, I think. >>>> >>> >>> Right, there's two ways to fix get(): >>> - Using the getOrDefault(key, sentinel) // added in Java 8 >>> - Using some sort of optional-like return type >>> >>> Not quite at that part of the road yet. >>> >>> >> >> >> -- >> >> Best Regards, >> Ali Ebrahimi >> > -- Best Regards, Ali Ebrahimi From simon at ochsenreither.de Wed Jan 7 14:16:29 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Wed, 07 Jan 2015 15:16:29 +0100 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD3CB8.6020905@ochsenreither.de> Message-ID: <54AD3FBD.4000500@ochsenreither.de> > If you're authoring a generic class and call one of these overloaded > methods, which one is called? What's the return value (for non void > ones)? The final target method is only known when a user instantiates > your class and provides the type, but not at authorship. Which overloaded methods? The standard rules for generic methods apply. The only question I raised was whether calls without BMA (e. g. existing code) could be made to "just work", so that Object+8-primitive-overloads could be migrated to any-T without breaking existing callers. From peter.levart at gmail.com Wed Jan 7 14:22:20 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 07 Jan 2015 15:22:20 +0100 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> Message-ID: <54AD411C.1090004@gmail.com> On 01/07/2015 02:54 PM, Vitaly Davidovich wrote: > The other option is to fail compilation if any-T context is calling a non > any-fied method This is how it works now. > and require user to do a cast on their T to select the > proper overload (including possibly casting to Object). This would not buy anything. I would like specializations to choose different overloads. > I don't know if we > want the method "late bound" like that. If we take "specializations" as something that is automatically generated on demand what would otherwise have to be written by hand, then the binding "happens" at compile-time - javac verifies that any specialization can choose the most appropriate overload. It's just that each specialization can choose a different overload. > Also, suppose the different > overloads return different types - the writer of code has to know what the > type will be upfront. Well, that's right, but also lambdas have multiple "returns" and we have target typing. Frequently, overloaded methods have all same return type (including void). It would be nice to have such a feature. It feels natural. Regards, Peter > > Sent from my phone > On Jan 7, 2015 8:44 AM, "Peter Levart" wrote: > >> Hi, >> >> While experimenting a bit with the current preliminary prototype, I found >> myself on several occasions wanting to call a method from "normal" API. Say >> System.out.println(x) with an x of type "any T". Usually a method that >> already has various overloads or a method taking an Object. Javac refuses: >> >> method PrintStream.println(boolean) is not applicable >> (argument mismatch; T cannot be converted to boolean) >> method PrintStream.println(char) is not applicable >> (argument mismatch; T cannot be converted to char) >> method PrintStream.println(int) is not applicable >> (argument mismatch; T cannot be converted to int) >> method PrintStream.println(long) is not applicable >> (argument mismatch; T cannot be converted to long) >> method PrintStream.println(float) is not applicable >> (argument mismatch; T cannot be converted to float) >> method PrintStream.println(double) is not applicable >> (argument mismatch; T cannot be converted to double) >> method PrintStream.println(char[]) is not applicable >> (argument mismatch; T cannot be converted to char[]) >> method PrintStream.println(String) is not applicable >> (argument mismatch; T cannot be converted to String) >> method PrintStream.println(Object) is not applicable >> (argument mismatch; T cannot be converted to Object) >> where T is a type-variable: >> T extends declared in method test() >> >> >> But we know that whatever instantiation of T we choose, at least one of >> available overloaded methods would apply. If the set of overloaded methods >> contains one with parameter of type Object, then any instantiation of >> parameter of type "any T" would apply to at least one of those methods. At >> specialization time, the most appropriate method could be "selected". >> >> Of course this would not automatically route reference typed parameter to >> the most appropriate method among those taking various reference types - >> the one taking Object would always be chosen, but any value typed paremeter >> could select the most appropriate method (with implicit conversions and/or >> boxing). >> >> This could be viewed as a more natural complement to "layers" for >> supporting implementation by parts. >> >> The specializer would either have to inspect the target class somehow, or >> javac could provide the set of applicable methods in an attribute >> associated with invocation. Or perhaps, such invocation could be >> "specialized" as invokedynamic where the linking to the most appropriate >> method (with implicit conversions and/or boxing) would be choosen at 1st >> invocation. >> >> What do you think? >> >> >> Regards, Peter >> >> From peter.levart at gmail.com Wed Jan 7 14:24:52 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 07 Jan 2015 15:24:52 +0100 Subject: alternatives or complements to layers In-Reply-To: <54AD3CB8.6020905@ochsenreither.de> References: <54AD37FF.9060301@gmail.com> <54AD3CB8.6020905@ochsenreither.de> Message-ID: <54AD41B4.2090409@gmail.com> On 01/07/2015 03:03 PM, Simon Ochsenreither wrote: >> The other option is to fail compilation if any-T context is calling a non >> any-fied method and require user to do a cast on their T to select the >> proper overload (including possibly casting to Object). I don't know if we >> want the method "late bound" like that. Also, suppose the different >> overloads return different types - the writer of code has to know what the >> type will be upfront. > I think the interesting question is "is there a reason why an any-fied > parameter couldn't act as a compatible replacement for methods with > Object + primitive overloads?". Because that's what any does under the > hood already: Create additional methods for non-reference types. It creates additional method signatures, yes, but not additional bodies that would differ - I said that such a feature could be viewed as a complement to layers. Peter From peter.levart at gmail.com Wed Jan 7 14:39:29 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 07 Jan 2015 15:39:29 +0100 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD3CB8.6020905@ochsenreither.de> Message-ID: <54AD4521.6020403@gmail.com> On 01/07/2015 03:08 PM, Vitaly Davidovich wrote: > If you're authoring a generic class and call one of these overloaded > methods, which one is called? What's the return value (for non void ones)? What's the return value of a lambda expression with multiple returns? What's the return value of "condition ? expression1 : expression2" ? > The final target method is only known when a user instantiates your class > and provides the type, but not at authorship. At authorship you know which method will be chosen for each possible instantiation. There are only a limited number of methods. There has to be a "last-resort" method taking Object parameter(s) or such invocation does not compile. Take for example the overloaded methods of System.out.println(). Which method is choosen in this example: void test(T x) { System.out.println(x); } ...it's always the println(Object) right? With primitive (and value type) instantiations, there are not so many other options. Primitives have more options since they can do implicit widening conversions AND boxing, but other value types will either choose the method taking exact value type if available, the next preference would be the boxed equivalent and finally the fallback to Object. Regards, Peter > > Sent from my phone > On Jan 7, 2015 9:04 AM, "Simon Ochsenreither" > wrote: > >>> The other option is to fail compilation if any-T context is calling a non >>> any-fied method and require user to do a cast on their T to select the >>> proper overload (including possibly casting to Object). I don't know if >> we >>> want the method "late bound" like that. Also, suppose the different >>> overloads return different types - the writer of code has to know what >> the >>> type will be upfront. >> I think the interesting question is "is there a reason why an any-fied >> parameter couldn't act as a compatible replacement for methods with >> Object + primitive overloads?". Because that's what any does under the >> hood already: Create additional methods for non-reference types. >> From vitalyd at gmail.com Wed Jan 7 14:45:26 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 09:45:26 -0500 Subject: Question on layer/peeling In-Reply-To: References: <54AB46C4.3040408@oracle.com> <54AB4CE2.8050208@oracle.com> <54AB663F.4020501@oracle.com> <26BA5E7E-3988-43C3-B9F4-6CF844917858@oracle.com> <54AC63B2.6060304@oracle.com> <54AC6580.3070606@oracle.com> Message-ID: Yes but that's only if you iterate; for maps, lookups/insertions are much more frequent than iteration and I'd most definitely be upset if get causes allocations. Sent from my phone On Jan 7, 2015 9:14 AM, "Ali Ebrahimi" wrote: > You already have to implement Set> entrySet() > > On Wed, Jan 7, 2015 at 5:14 PM, Vitaly Davidovich > wrote: > >> Not sure that's ideal as the implementation may not be using entry >> objects to store things (e.g. open addressed version with parallel arrays >> for keys and values); this would cause the get method to allocate an entry >> for each get operation. If the entry impl is a value type it'll be boxed >> since it's assigned to an interface. >> >> Sent from my phone >> On Jan 7, 2015 3:03 AM, "Ali Ebrahimi" >> wrote: >> >>> Or add method >>> Map.Entry getEntry(key) >>> >>> On Wed, Jan 7, 2015 at 2:15 AM, Brian Goetz >>> wrote: >>> >>>> By the way, C# solves this by having TryGet type of methods that return >>>>> a >>>>> boolean indicating success (I.e. found mapping) and set an out >>>>> parameter to >>>>> the value found. You'd need a lightweight tuple or multi return to do >>>>> the >>>>> same in java, I think. >>>>> >>>> >>>> Right, there's two ways to fix get(): >>>> - Using the getOrDefault(key, sentinel) // added in Java 8 >>>> - Using some sort of optional-like return type >>>> >>>> Not quite at that part of the road yet. >>>> >>>> >>> >>> >>> -- >>> >>> Best Regards, >>> Ali Ebrahimi >>> >> > > > -- > > Best Regards, > Ali Ebrahimi > From vitalyd at gmail.com Wed Jan 7 14:54:32 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 09:54:32 -0500 Subject: alternatives or complements to layers In-Reply-To: <54AD4521.6020403@gmail.com> References: <54AD37FF.9060301@gmail.com> <54AD3CB8.6020905@ochsenreither.de> <54AD4521.6020403@gmail.com> Message-ID: But why make the specializer more complex and somewhat "magical" with fallback rules, widening/conversion operations, etc? Why not add a println (T) version? Sent from my phone On Jan 7, 2015 9:39 AM, "Peter Levart" wrote: > On 01/07/2015 03:08 PM, Vitaly Davidovich wrote: > >> If you're authoring a generic class and call one of these overloaded >> methods, which one is called? What's the return value (for non void ones)? >> > > What's the return value of a lambda expression with multiple returns? > What's the return value of "condition ? expression1 : expression2" ? > > The final target method is only known when a user instantiates your class >> and provides the type, but not at authorship. >> > > At authorship you know which method will be chosen for each possible > instantiation. There are only a limited number of methods. There has to be > a "last-resort" method taking Object parameter(s) or such invocation does > not compile. > > Take for example the overloaded methods of System.out.println(). Which > method is choosen in this example: > > void test(T x) { > System.out.println(x); > } > > ...it's always the println(Object) right? > > With primitive (and value type) instantiations, there are not so many > other options. Primitives have more options since they can do implicit > widening conversions AND boxing, but other value types will either choose > the method taking exact value type if available, the next preference would > be the boxed equivalent and finally the fallback to Object. > > > Regards, Peter > > >> Sent from my phone >> On Jan 7, 2015 9:04 AM, "Simon Ochsenreither" >> wrote: >> >> The other option is to fail compilation if any-T context is calling a non >>>> any-fied method and require user to do a cast on their T to select the >>>> proper overload (including possibly casting to Object). I don't know if >>>> >>> we >>> >>>> want the method "late bound" like that. Also, suppose the different >>>> overloads return different types - the writer of code has to know what >>>> >>> the >>> >>>> type will be upfront. >>>> >>> I think the interesting question is "is there a reason why an any-fied >>> parameter couldn't act as a compatible replacement for methods with >>> Object + primitive overloads?". Because that's what any does under the >>> hood already: Create additional methods for non-reference types. >>> >>> > From peter.levart at gmail.com Wed Jan 7 14:57:35 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 07 Jan 2015 15:57:35 +0100 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD3CB8.6020905@ochsenreither.de> <54AD4521.6020403@gmail.com> Message-ID: <54AD495F.40206@gmail.com> On 01/07/2015 03:54 PM, Vitaly Davidovich wrote: > But why make the specializer more complex and somewhat "magical" with > fallback rules, widening/conversion operations, etc? Why not add a > println (T) version? You could, yes. But how would you implement it? Peter > > Sent from my phone > On Jan 7, 2015 9:39 AM, "Peter Levart" wrote: > >> On 01/07/2015 03:08 PM, Vitaly Davidovich wrote: >> >>> If you're authoring a generic class and call one of these overloaded >>> methods, which one is called? What's the return value (for non void ones)? >>> >> What's the return value of a lambda expression with multiple returns? >> What's the return value of "condition ? expression1 : expression2" ? >> >> The final target method is only known when a user instantiates your class >>> and provides the type, but not at authorship. >>> >> At authorship you know which method will be chosen for each possible >> instantiation. There are only a limited number of methods. There has to be >> a "last-resort" method taking Object parameter(s) or such invocation does >> not compile. >> >> Take for example the overloaded methods of System.out.println(). Which >> method is choosen in this example: >> >> void test(T x) { >> System.out.println(x); >> } >> >> ...it's always the println(Object) right? >> >> With primitive (and value type) instantiations, there are not so many >> other options. Primitives have more options since they can do implicit >> widening conversions AND boxing, but other value types will either choose >> the method taking exact value type if available, the next preference would >> be the boxed equivalent and finally the fallback to Object. >> >> >> Regards, Peter >> >> >>> Sent from my phone >>> On Jan 7, 2015 9:04 AM, "Simon Ochsenreither" >>> wrote: >>> >>> The other option is to fail compilation if any-T context is calling a non >>>>> any-fied method and require user to do a cast on their T to select the >>>>> proper overload (including possibly casting to Object). I don't know if >>>>> >>>> we >>>> >>>>> want the method "late bound" like that. Also, suppose the different >>>>> overloads return different types - the writer of code has to know what >>>>> >>>> the >>>> >>>>> type will be upfront. >>>>> >>>> I think the interesting question is "is there a reason why an any-fied >>>> parameter couldn't act as a compatible replacement for methods with >>>> Object + primitive overloads?". Because that's what any does under the >>>> hood already: Create additional methods for non-reference types. >>>> >>>> From vitalyd at gmail.com Wed Jan 7 15:01:45 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 10:01:45 -0500 Subject: alternatives or complements to layers In-Reply-To: <54AD495F.40206@gmail.com> References: <54AD37FF.9060301@gmail.com> <54AD3CB8.6020905@ochsenreither.de> <54AD4521.6020403@gmail.com> <54AD495F.40206@gmail.com> Message-ID: Personally I'd handle primitives as if they were value types from day 1: they get Object's hashCode, equals, and toString by allowing T.toString and rewriting it to their corresponding XXX.toString. Sent from my phone On Jan 7, 2015 9:57 AM, "Peter Levart" wrote: > On 01/07/2015 03:54 PM, Vitaly Davidovich wrote: > >> But why make the specializer more complex and somewhat "magical" with >> fallback rules, widening/conversion operations, etc? Why not add a > T> >> println (T) version? >> > > You could, yes. But how would you implement it? > > Peter > > >> Sent from my phone >> On Jan 7, 2015 9:39 AM, "Peter Levart" wrote: >> >> On 01/07/2015 03:08 PM, Vitaly Davidovich wrote: >>> >>> If you're authoring a generic class and call one of these overloaded >>>> methods, which one is called? What's the return value (for non void >>>> ones)? >>>> >>>> What's the return value of a lambda expression with multiple returns? >>> What's the return value of "condition ? expression1 : expression2" ? >>> >>> The final target method is only known when a user instantiates your >>> class >>> >>>> and provides the type, but not at authorship. >>>> >>>> At authorship you know which method will be chosen for each possible >>> instantiation. There are only a limited number of methods. There has to >>> be >>> a "last-resort" method taking Object parameter(s) or such invocation does >>> not compile. >>> >>> Take for example the overloaded methods of System.out.println(). Which >>> method is choosen in this example: >>> >>> void test(T x) { >>> System.out.println(x); >>> } >>> >>> ...it's always the println(Object) right? >>> >>> With primitive (and value type) instantiations, there are not so many >>> other options. Primitives have more options since they can do implicit >>> widening conversions AND boxing, but other value types will either choose >>> the method taking exact value type if available, the next preference >>> would >>> be the boxed equivalent and finally the fallback to Object. >>> >>> >>> Regards, Peter >>> >>> >>> Sent from my phone >>>> On Jan 7, 2015 9:04 AM, "Simon Ochsenreither" >>>> wrote: >>>> >>>> The other option is to fail compilation if any-T context is calling a >>>> non >>>> >>>>> any-fied method and require user to do a cast on their T to select the >>>>>> proper overload (including possibly casting to Object). I don't know >>>>>> if >>>>>> >>>>>> we >>>>> >>>>> want the method "late bound" like that. Also, suppose the different >>>>>> overloads return different types - the writer of code has to know what >>>>>> >>>>>> the >>>>> >>>>> type will be upfront. >>>>>> >>>>>> I think the interesting question is "is there a reason why an >>>>> any-fied >>>>> parameter couldn't act as a compatible replacement for methods with >>>>> Object + primitive overloads?". Because that's what any does under the >>>>> hood already: Create additional methods for non-reference types. >>>>> >>>>> >>>>> > From vitalyd at gmail.com Wed Jan 7 15:15:25 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 10:15:25 -0500 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD3CB8.6020905@ochsenreither.de> <54AD4521.6020403@gmail.com> <54AD495F.40206@gmail.com> Message-ID: And for cases where you don't want to materialize a string because the underlying API supports outputting the type char-by-char, I'd probably continue special casing that via type checks (i.e. manual overload resolution). It's not ideal, but (a) fairly rare, (b) confined only to those classes that provide specialized versions, (c) only relevant for printing things, and (d) keeps the specializer relatively simple, I think. I'd hate for the specializer to be more complicated purely for this scenario. On Wed, Jan 7, 2015 at 10:01 AM, Vitaly Davidovich wrote: > Personally I'd handle primitives as if they were value types from day 1: > they get Object's hashCode, equals, and toString by allowing T.toString and > rewriting it to their corresponding XXX.toString. > > Sent from my phone > On Jan 7, 2015 9:57 AM, "Peter Levart" wrote: > >> On 01/07/2015 03:54 PM, Vitaly Davidovich wrote: >> >>> But why make the specializer more complex and somewhat "magical" with >>> fallback rules, widening/conversion operations, etc? Why not add a >> T> >>> println (T) version? >>> >> >> You could, yes. But how would you implement it? >> >> Peter >> >> >>> Sent from my phone >>> On Jan 7, 2015 9:39 AM, "Peter Levart" wrote: >>> >>> On 01/07/2015 03:08 PM, Vitaly Davidovich wrote: >>>> >>>> If you're authoring a generic class and call one of these overloaded >>>>> methods, which one is called? What's the return value (for non void >>>>> ones)? >>>>> >>>>> What's the return value of a lambda expression with multiple returns? >>>> What's the return value of "condition ? expression1 : expression2" ? >>>> >>>> The final target method is only known when a user instantiates your >>>> class >>>> >>>>> and provides the type, but not at authorship. >>>>> >>>>> At authorship you know which method will be chosen for each possible >>>> instantiation. There are only a limited number of methods. There has to >>>> be >>>> a "last-resort" method taking Object parameter(s) or such invocation >>>> does >>>> not compile. >>>> >>>> Take for example the overloaded methods of System.out.println(). Which >>>> method is choosen in this example: >>>> >>>> void test(T x) { >>>> System.out.println(x); >>>> } >>>> >>>> ...it's always the println(Object) right? >>>> >>>> With primitive (and value type) instantiations, there are not so many >>>> other options. Primitives have more options since they can do implicit >>>> widening conversions AND boxing, but other value types will either >>>> choose >>>> the method taking exact value type if available, the next preference >>>> would >>>> be the boxed equivalent and finally the fallback to Object. >>>> >>>> >>>> Regards, Peter >>>> >>>> >>>> Sent from my phone >>>>> On Jan 7, 2015 9:04 AM, "Simon Ochsenreither" >>>>> wrote: >>>>> >>>>> The other option is to fail compilation if any-T context is calling >>>>> a non >>>>> >>>>>> any-fied method and require user to do a cast on their T to select the >>>>>>> proper overload (including possibly casting to Object). I don't >>>>>>> know if >>>>>>> >>>>>>> we >>>>>> >>>>>> want the method "late bound" like that. Also, suppose the different >>>>>>> overloads return different types - the writer of code has to know >>>>>>> what >>>>>>> >>>>>>> the >>>>>> >>>>>> type will be upfront. >>>>>>> >>>>>>> I think the interesting question is "is there a reason why an >>>>>> any-fied >>>>>> parameter couldn't act as a compatible replacement for methods with >>>>>> Object + primitive overloads?". Because that's what any does under the >>>>>> hood already: Create additional methods for non-reference types. >>>>>> >>>>>> >>>>>> >> From peter.levart at gmail.com Wed Jan 7 15:18:16 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 07 Jan 2015 16:18:16 +0100 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD3CB8.6020905@ochsenreither.de> <54AD4521.6020403@gmail.com> <54AD495F.40206@gmail.com> Message-ID: <54AD4E38.8090204@gmail.com> On 01/07/2015 04:01 PM, Vitaly Davidovich wrote: > Personally I'd handle primitives as if they were value types from day 1: > they get Object's hashCode, equals, and toString by allowing T.toString and > rewriting it to their corresponding XXX.toString. Ok, there are alternatives for Object methods, yes. So that available operations on "any T" are at least those. But you have to define them upfront and then they are baked in the language. Without those, you are confined to the world of "any T" methods from which you can't escape to interact with the "real world". I view this hypothetical feature as an alternative or complement to "layers". But it can also be viewed as a kind of "extension (multi)methods" for value types. Peter > > Sent from my phone > On Jan 7, 2015 9:57 AM, "Peter Levart" wrote: > >> On 01/07/2015 03:54 PM, Vitaly Davidovich wrote: >> >>> But why make the specializer more complex and somewhat "magical" with >>> fallback rules, widening/conversion operations, etc? Why not add a >> T> >>> println (T) version? >>> >> You could, yes. But how would you implement it? >> >> Peter >> >> >>> Sent from my phone >>> On Jan 7, 2015 9:39 AM, "Peter Levart" wrote: >>> >>> On 01/07/2015 03:08 PM, Vitaly Davidovich wrote: >>>> If you're authoring a generic class and call one of these overloaded >>>>> methods, which one is called? What's the return value (for non void >>>>> ones)? >>>>> >>>>> What's the return value of a lambda expression with multiple returns? >>>> What's the return value of "condition ? expression1 : expression2" ? >>>> >>>> The final target method is only known when a user instantiates your >>>> class >>>> >>>>> and provides the type, but not at authorship. >>>>> >>>>> At authorship you know which method will be chosen for each possible >>>> instantiation. There are only a limited number of methods. There has to >>>> be >>>> a "last-resort" method taking Object parameter(s) or such invocation does >>>> not compile. >>>> >>>> Take for example the overloaded methods of System.out.println(). Which >>>> method is choosen in this example: >>>> >>>> void test(T x) { >>>> System.out.println(x); >>>> } >>>> >>>> ...it's always the println(Object) right? >>>> >>>> With primitive (and value type) instantiations, there are not so many >>>> other options. Primitives have more options since they can do implicit >>>> widening conversions AND boxing, but other value types will either choose >>>> the method taking exact value type if available, the next preference >>>> would >>>> be the boxed equivalent and finally the fallback to Object. >>>> >>>> >>>> Regards, Peter >>>> >>>> >>>> Sent from my phone >>>>> On Jan 7, 2015 9:04 AM, "Simon Ochsenreither" >>>>> wrote: >>>>> >>>>> The other option is to fail compilation if any-T context is calling a >>>>> non >>>>> >>>>>> any-fied method and require user to do a cast on their T to select the >>>>>>> proper overload (including possibly casting to Object). I don't know >>>>>>> if >>>>>>> >>>>>>> we >>>>>> want the method "late bound" like that. Also, suppose the different >>>>>>> overloads return different types - the writer of code has to know what >>>>>>> >>>>>>> the >>>>>> type will be upfront. >>>>>>> I think the interesting question is "is there a reason why an >>>>>> any-fied >>>>>> parameter couldn't act as a compatible replacement for methods with >>>>>> Object + primitive overloads?". Because that's what any does under the >>>>>> hood already: Create additional methods for non-reference types. >>>>>> >>>>>> >>>>>> From maurizio.cimadamore at oracle.com Wed Jan 7 15:22:40 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 07 Jan 2015 15:22:40 +0000 Subject: alternatives or complements to layers In-Reply-To: <54AD37FF.9060301@gmail.com> References: <54AD37FF.9060301@gmail.com> Message-ID: <54AD4F40.8010809@oracle.com> Hi Peter, I found this often too (and it is frustrating!) when trying to apply some of the valhalla magic on our collection classes (the case I found was System.arraycopy, which suffers from similar issues). I believe the pain we are feeling now is both (a) benign and (b) transient. It is benign because, in one way, it makes you see how 'old' the API look once you fullt embrace the idea of a unified type-system; if we were writing this from scratch, println would probably something like: void println(Z z); And then it would be implemented by parts using peeling. The 'transient' nature of this pain is because we are working with a unified language, but an erasure-based library; I believe that this method (and other similar ones) will have to be rewritten (or an extra 'any' variant added) so that they work uniformly with specialized generics. P.S. We can just consider the set of overloaded println (Object + primitives) 'good enough' for an 'any X'; what if X is an instance of a value class? Maurizio On 07/01/15 13:43, Peter Levart wrote: > Hi, > > While experimenting a bit with the current preliminary prototype, I > found myself on several occasions wanting to call a method from > "normal" API. Say System.out.println(x) with an x of type "any T". > Usually a method that already has various overloads or a method taking > an Object. Javac refuses: > > method PrintStream.println(boolean) is not applicable > (argument mismatch; T cannot be converted to boolean) > method PrintStream.println(char) is not applicable > (argument mismatch; T cannot be converted to char) > method PrintStream.println(int) is not applicable > (argument mismatch; T cannot be converted to int) > method PrintStream.println(long) is not applicable > (argument mismatch; T cannot be converted to long) > method PrintStream.println(float) is not applicable > (argument mismatch; T cannot be converted to float) > method PrintStream.println(double) is not applicable > (argument mismatch; T cannot be converted to double) > method PrintStream.println(char[]) is not applicable > (argument mismatch; T cannot be converted to char[]) > method PrintStream.println(String) is not applicable > (argument mismatch; T cannot be converted to String) > method PrintStream.println(Object) is not applicable > (argument mismatch; T cannot be converted to Object) > where T is a type-variable: > T extends declared in method test() > > > But we know that whatever instantiation of T we choose, at least one > of available overloaded methods would apply. If the set of overloaded > methods contains one with parameter of type Object, then any > instantiation of parameter of type "any T" would apply to at least one > of those methods. At specialization time, the most appropriate method > could be "selected". > > Of course this would not automatically route reference typed parameter > to the most appropriate method among those taking various reference > types - the one taking Object would always be chosen, but any value > typed paremeter could select the most appropriate method (with > implicit conversions and/or boxing). > > This could be viewed as a more natural complement to "layers" for > supporting implementation by parts. > > The specializer would either have to inspect the target class somehow, > or javac could provide the set of applicable methods in an attribute > associated with invocation. Or perhaps, such invocation could be > "specialized" as invokedynamic where the linking to the most > appropriate method (with implicit conversions and/or boxing) would be > choosen at 1st invocation. > > What do you think? > > > Regards, Peter > From vitalyd at gmail.com Wed Jan 7 15:25:24 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 10:25:24 -0500 Subject: alternatives or complements to layers In-Reply-To: <54AD4E38.8090204@gmail.com> References: <54AD37FF.9060301@gmail.com> <54AD3CB8.6020905@ochsenreither.de> <54AD4521.6020403@gmail.com> <54AD495F.40206@gmail.com> <54AD4E38.8090204@gmail.com> Message-ID: Methods that you want to invoke that aren't part of Object should be handled by allowing generic constraints on the T; e.g. " bar(T t) { // call some Foo method on t }" (ignore the syntax as it's unimportant right now). This way you can pass a value type that implements an interface and not cause boxing. On Wed, Jan 7, 2015 at 10:18 AM, Peter Levart wrote: > On 01/07/2015 04:01 PM, Vitaly Davidovich wrote: > >> Personally I'd handle primitives as if they were value types from day 1: >> they get Object's hashCode, equals, and toString by allowing T.toString >> and >> rewriting it to their corresponding XXX.toString. >> > > Ok, there are alternatives for Object methods, yes. So that available > operations on "any T" are at least those. But you have to define them > upfront and then they are baked in the language. Without those, you are > confined to the world of "any T" methods from which you can't escape to > interact with the "real world". > > I view this hypothetical feature as an alternative or complement to > "layers". But it can also be viewed as a kind of "extension (multi)methods" > for value types. > > > Peter > > >> Sent from my phone >> On Jan 7, 2015 9:57 AM, "Peter Levart" wrote: >> >> On 01/07/2015 03:54 PM, Vitaly Davidovich wrote: >>> >>> But why make the specializer more complex and somewhat "magical" with >>>> fallback rules, widening/conversion operations, etc? Why not add a >>> T> >>>> println (T) version? >>>> >>>> You could, yes. But how would you implement it? >>> >>> Peter >>> >>> >>> Sent from my phone >>>> On Jan 7, 2015 9:39 AM, "Peter Levart" wrote: >>>> >>>> On 01/07/2015 03:08 PM, Vitaly Davidovich wrote: >>>> >>>>> If you're authoring a generic class and call one of these overloaded >>>>> >>>>>> methods, which one is called? What's the return value (for non void >>>>>> ones)? >>>>>> >>>>>> What's the return value of a lambda expression with multiple >>>>>> returns? >>>>>> >>>>> What's the return value of "condition ? expression1 : expression2" ? >>>>> >>>>> The final target method is only known when a user instantiates your >>>>> class >>>>> >>>>> and provides the type, but not at authorship. >>>>>> >>>>>> At authorship you know which method will be chosen for each possible >>>>>> >>>>> instantiation. There are only a limited number of methods. There has to >>>>> be >>>>> a "last-resort" method taking Object parameter(s) or such invocation >>>>> does >>>>> not compile. >>>>> >>>>> Take for example the overloaded methods of System.out.println(). Which >>>>> method is choosen in this example: >>>>> >>>>> void test(T x) { >>>>> System.out.println(x); >>>>> } >>>>> >>>>> ...it's always the println(Object) right? >>>>> >>>>> With primitive (and value type) instantiations, there are not so many >>>>> other options. Primitives have more options since they can do implicit >>>>> widening conversions AND boxing, but other value types will either >>>>> choose >>>>> the method taking exact value type if available, the next preference >>>>> would >>>>> be the boxed equivalent and finally the fallback to Object. >>>>> >>>>> >>>>> Regards, Peter >>>>> >>>>> >>>>> Sent from my phone >>>>> >>>>>> On Jan 7, 2015 9:04 AM, "Simon Ochsenreither" >>>>> > >>>>>> wrote: >>>>>> >>>>>> The other option is to fail compilation if any-T context is >>>>>> calling a >>>>>> non >>>>>> >>>>>> any-fied method and require user to do a cast on their T to select >>>>>>> the >>>>>>> >>>>>>>> proper overload (including possibly casting to Object). I don't >>>>>>>> know >>>>>>>> if >>>>>>>> >>>>>>>> we >>>>>>>> >>>>>>> want the method "late bound" like that. Also, suppose the >>>>>>> different >>>>>>> >>>>>>>> overloads return different types - the writer of code has to know >>>>>>>> what >>>>>>>> >>>>>>>> the >>>>>>>> >>>>>>> type will be upfront. >>>>>>> >>>>>>>> I think the interesting question is "is there a reason why an >>>>>>>> >>>>>>> any-fied >>>>>>> parameter couldn't act as a compatible replacement for methods with >>>>>>> Object + primitive overloads?". Because that's what any does under >>>>>>> the >>>>>>> hood already: Create additional methods for non-reference types. >>>>>>> >>>>>>> >>>>>>> >>>>>>> > From vitalyd at gmail.com Wed Jan 7 15:35:25 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 10:35:25 -0500 Subject: alternatives or complements to layers In-Reply-To: <54AD4F40.8010809@oracle.com> References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> Message-ID: > > We can just consider the set of overloaded println (Object + primitives) > 'good enough' for an 'any X'; what if X is an instance of a value class? Well, if you're going to go down the road of having the specializer pick overloads, it should pick the most specific one, preferring non-generic methods over generic ones. If there's a non-generic target method that takes the exact type, it's picked (irrespective if it's value or reference type). If there's a generic method that's applicable, it's picked. Finally, pick Object, and box the value type/primitive. I would not consider widening conversions of primitives as candidates (i.e. T=int, there's no int overload but there is a long one -- I would not consider the long variant as eligible). On Wed, Jan 7, 2015 at 10:22 AM, Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > Hi Peter, > I found this often too (and it is frustrating!) when trying to apply some > of the valhalla magic on our collection classes (the case I found was > System.arraycopy, which suffers from similar issues). I believe the pain we > are feeling now is both (a) benign and (b) transient. It is benign because, > in one way, it makes you see how 'old' the API look once you fullt embrace > the idea of a unified type-system; if we were writing this from scratch, > println would probably something like: > > void println(Z z); > > And then it would be implemented by parts using peeling. > > The 'transient' nature of this pain is because we are working with a > unified language, but an erasure-based library; I believe that this method > (and other similar ones) will have to be rewritten (or an extra 'any' > variant added) so that they work uniformly with specialized generics. > > P.S. > We can just consider the set of overloaded println (Object + primitives) > 'good enough' for an 'any X'; what if X is an instance of a value class? > > Maurizio > > > On 07/01/15 13:43, Peter Levart wrote: > >> Hi, >> >> While experimenting a bit with the current preliminary prototype, I found >> myself on several occasions wanting to call a method from "normal" API. Say >> System.out.println(x) with an x of type "any T". Usually a method that >> already has various overloads or a method taking an Object. Javac refuses: >> >> method PrintStream.println(boolean) is not applicable >> (argument mismatch; T cannot be converted to boolean) >> method PrintStream.println(char) is not applicable >> (argument mismatch; T cannot be converted to char) >> method PrintStream.println(int) is not applicable >> (argument mismatch; T cannot be converted to int) >> method PrintStream.println(long) is not applicable >> (argument mismatch; T cannot be converted to long) >> method PrintStream.println(float) is not applicable >> (argument mismatch; T cannot be converted to float) >> method PrintStream.println(double) is not applicable >> (argument mismatch; T cannot be converted to double) >> method PrintStream.println(char[]) is not applicable >> (argument mismatch; T cannot be converted to char[]) >> method PrintStream.println(String) is not applicable >> (argument mismatch; T cannot be converted to String) >> method PrintStream.println(Object) is not applicable >> (argument mismatch; T cannot be converted to Object) >> where T is a type-variable: >> T extends declared in method test() >> >> >> But we know that whatever instantiation of T we choose, at least one of >> available overloaded methods would apply. If the set of overloaded methods >> contains one with parameter of type Object, then any instantiation of >> parameter of type "any T" would apply to at least one of those methods. At >> specialization time, the most appropriate method could be "selected". >> >> Of course this would not automatically route reference typed parameter to >> the most appropriate method among those taking various reference types - >> the one taking Object would always be chosen, but any value typed paremeter >> could select the most appropriate method (with implicit conversions and/or >> boxing). >> >> This could be viewed as a more natural complement to "layers" for >> supporting implementation by parts. >> >> The specializer would either have to inspect the target class somehow, or >> javac could provide the set of applicable methods in an attribute >> associated with invocation. Or perhaps, such invocation could be >> "specialized" as invokedynamic where the linking to the most appropriate >> method (with implicit conversions and/or boxing) would be choosen at 1st >> invocation. >> >> What do you think? >> >> >> Regards, Peter >> >> > From peter.levart at gmail.com Wed Jan 7 15:37:20 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 07 Jan 2015 16:37:20 +0100 Subject: alternatives or complements to layers In-Reply-To: <54AD4F40.8010809@oracle.com> References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> Message-ID: <54AD52B0.8030503@gmail.com> On 01/07/2015 04:22 PM, Maurizio Cimadamore wrote: > Hi Peter, > I found this often too (and it is frustrating!) when trying to apply > some of the valhalla magic on our collection classes (the case I found > was System.arraycopy, which suffers from similar issues). I believe > the pain we are feeling now is both (a) benign and (b) transient. It > is benign because, in one way, it makes you see how 'old' the API look > once you fullt embrace the idea of a unified type-system; if we were > writing this from scratch, println would probably something like: > > void println(Z z); > > And then it would be implemented by parts using peeling. > > The 'transient' nature of this pain is because we are working with a > unified language, but an erasure-based library; I believe that this > method (and other similar ones) will have to be rewritten (or an extra > 'any' variant added) so that they work uniformly with specialized > generics. Ok, I buy that, but it would mean that existing core libraries would have to get a massive update. > > P.S. > We can just consider the set of overloaded println (Object + > primitives) 'good enough' for an 'any X'; what if X is an instance of > a value class? It was said that an option is for all value types (including primitives) to get 3 Object methods (hashCode, equals, toString). So at least println is doable. Regards, Peter > > Maurizio > > On 07/01/15 13:43, Peter Levart wrote: >> Hi, >> >> While experimenting a bit with the current preliminary prototype, I >> found myself on several occasions wanting to call a method from >> "normal" API. Say System.out.println(x) with an x of type "any T". >> Usually a method that already has various overloads or a method >> taking an Object. Javac refuses: >> >> method PrintStream.println(boolean) is not applicable >> (argument mismatch; T cannot be converted to boolean) >> method PrintStream.println(char) is not applicable >> (argument mismatch; T cannot be converted to char) >> method PrintStream.println(int) is not applicable >> (argument mismatch; T cannot be converted to int) >> method PrintStream.println(long) is not applicable >> (argument mismatch; T cannot be converted to long) >> method PrintStream.println(float) is not applicable >> (argument mismatch; T cannot be converted to float) >> method PrintStream.println(double) is not applicable >> (argument mismatch; T cannot be converted to double) >> method PrintStream.println(char[]) is not applicable >> (argument mismatch; T cannot be converted to char[]) >> method PrintStream.println(String) is not applicable >> (argument mismatch; T cannot be converted to String) >> method PrintStream.println(Object) is not applicable >> (argument mismatch; T cannot be converted to Object) >> where T is a type-variable: >> T extends declared in method test() >> >> >> But we know that whatever instantiation of T we choose, at least one >> of available overloaded methods would apply. If the set of overloaded >> methods contains one with parameter of type Object, then any >> instantiation of parameter of type "any T" would apply to at least >> one of those methods. At specialization time, the most appropriate >> method could be "selected". >> >> Of course this would not automatically route reference typed >> parameter to the most appropriate method among those taking various >> reference types - the one taking Object would always be chosen, but >> any value typed paremeter could select the most appropriate method >> (with implicit conversions and/or boxing). >> >> This could be viewed as a more natural complement to "layers" for >> supporting implementation by parts. >> >> The specializer would either have to inspect the target class >> somehow, or javac could provide the set of applicable methods in an >> attribute associated with invocation. Or perhaps, such invocation >> could be "specialized" as invokedynamic where the linking to the most >> appropriate method (with implicit conversions and/or boxing) would be >> choosen at 1st invocation. >> >> What do you think? >> >> >> Regards, Peter >> > From maurizio.cimadamore at oracle.com Wed Jan 7 15:39:20 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 07 Jan 2015 15:39:20 +0000 Subject: alternatives or complements to layers In-Reply-To: <54AD52B0.8030503@gmail.com> References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> Message-ID: <54AD5328.3080000@oracle.com> On 07/01/15 15:37, Peter Levart wrote: > It was said that an option is for all value types (including > primitives) to get 3 Object methods (hashCode, equals, toString). So > at least println is doable. Right - I was speaking more generally of all methods which are overloaded on the parameter type with Object, int, long, ... Maurizio From vitalyd at gmail.com Wed Jan 7 15:41:03 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 10:41:03 -0500 Subject: alternatives or complements to layers In-Reply-To: <54AD52B0.8030503@gmail.com> References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> Message-ID: Yes, PrintStream.println() appears to materialize strings, so that's doable "as-is". A more interesting case is, e.g., AbstractStringBuilder.append(), which has specialized primitive versions that do not materialize strings. On Wed, Jan 7, 2015 at 10:37 AM, Peter Levart wrote: > On 01/07/2015 04:22 PM, Maurizio Cimadamore wrote: > >> Hi Peter, >> I found this often too (and it is frustrating!) when trying to apply some >> of the valhalla magic on our collection classes (the case I found was >> System.arraycopy, which suffers from similar issues). I believe the pain we >> are feeling now is both (a) benign and (b) transient. It is benign because, >> in one way, it makes you see how 'old' the API look once you fullt embrace >> the idea of a unified type-system; if we were writing this from scratch, >> println would probably something like: >> >> void println(Z z); >> >> And then it would be implemented by parts using peeling. >> >> The 'transient' nature of this pain is because we are working with a >> unified language, but an erasure-based library; I believe that this method >> (and other similar ones) will have to be rewritten (or an extra 'any' >> variant added) so that they work uniformly with specialized generics. >> > > Ok, I buy that, but it would mean that existing core libraries would have > to get a massive update. > > >> P.S. >> We can just consider the set of overloaded println (Object + primitives) >> 'good enough' for an 'any X'; what if X is an instance of a value class? >> > > It was said that an option is for all value types (including primitives) > to get 3 Object methods (hashCode, equals, toString). So at least println > is doable. > > Regards, Peter > > > >> Maurizio >> >> On 07/01/15 13:43, Peter Levart wrote: >> >>> Hi, >>> >>> While experimenting a bit with the current preliminary prototype, I >>> found myself on several occasions wanting to call a method from "normal" >>> API. Say System.out.println(x) with an x of type "any T". Usually a method >>> that already has various overloads or a method taking an Object. Javac >>> refuses: >>> >>> method PrintStream.println(boolean) is not applicable >>> (argument mismatch; T cannot be converted to boolean) >>> method PrintStream.println(char) is not applicable >>> (argument mismatch; T cannot be converted to char) >>> method PrintStream.println(int) is not applicable >>> (argument mismatch; T cannot be converted to int) >>> method PrintStream.println(long) is not applicable >>> (argument mismatch; T cannot be converted to long) >>> method PrintStream.println(float) is not applicable >>> (argument mismatch; T cannot be converted to float) >>> method PrintStream.println(double) is not applicable >>> (argument mismatch; T cannot be converted to double) >>> method PrintStream.println(char[]) is not applicable >>> (argument mismatch; T cannot be converted to char[]) >>> method PrintStream.println(String) is not applicable >>> (argument mismatch; T cannot be converted to String) >>> method PrintStream.println(Object) is not applicable >>> (argument mismatch; T cannot be converted to Object) >>> where T is a type-variable: >>> T extends declared in method test() >>> >>> >>> But we know that whatever instantiation of T we choose, at least one of >>> available overloaded methods would apply. If the set of overloaded methods >>> contains one with parameter of type Object, then any instantiation of >>> parameter of type "any T" would apply to at least one of those methods. At >>> specialization time, the most appropriate method could be "selected". >>> >>> Of course this would not automatically route reference typed parameter >>> to the most appropriate method among those taking various reference types - >>> the one taking Object would always be chosen, but any value typed paremeter >>> could select the most appropriate method (with implicit conversions and/or >>> boxing). >>> >>> This could be viewed as a more natural complement to "layers" for >>> supporting implementation by parts. >>> >>> The specializer would either have to inspect the target class somehow, >>> or javac could provide the set of applicable methods in an attribute >>> associated with invocation. Or perhaps, such invocation could be >>> "specialized" as invokedynamic where the linking to the most appropriate >>> method (with implicit conversions and/or boxing) would be choosen at 1st >>> invocation. >>> >>> What do you think? >>> >>> >>> Regards, Peter >>> >>> >> > From ali.ebrahimi1781 at gmail.com Wed Jan 7 16:11:39 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Wed, 7 Jan 2015 19:41:39 +0330 Subject: alternatives or complements to layers In-Reply-To: <54AD4F40.8010809@oracle.com> References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> Message-ID: Hi, On Wed, Jan 7, 2015 at 6:52 PM, Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > Hi Peter, > > We can just consider the set of overloaded println (Object + primitives) > 'good enough' for an 'any X'; what if X is an instance of a value class? > of course if in future we have (Object + primitives + some value types) If we all agree value types can be boxed/ub-boxed as with primitive types, there is no difference. if value types have Integer.valueOf/parseInt counterpart method compiler/specialize can utilize as needed for boxin/unboxing. > Maurizio Best Regards, Ali Ebrahimi From brian.goetz at oracle.com Wed Jan 7 16:31:37 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jan 2015 11:31:37 -0500 Subject: alternatives or complements to layers In-Reply-To: <54AD37FF.9060301@gmail.com> References: <54AD37FF.9060301@gmail.com> Message-ID: <54AD5F69.6060307@oracle.com> Nice question, Peter. I don't like the idea of automagically figuring it out based on overloads, but I do like the idea that there's a path to turning this bag of overloads into a coherent entity, so we can maximize investment in / compatibility with existing libraries. The naive approach would have us look at a family of methods with the same name and whose parameters follow a certain pattern (i.e., there's a foo(Object) and a foo(P) for at least N primitive types P), and optimistically guessing that they must be manual specialization of the same semantics. You can see why this makes us uncomfortable. But, would it be possible to *say* this in the source code? "Hey, specializer, these foo(X) methods for X=Object,int,long are the same method!" This is closely related to one of the use cases called out in SotS, which is "hand-specialization of a specific instantiation of a generic method". So if you have a method generic in any T: void println(T t) { .. } could you also add manually-written optimizations for known values of T? // This is a specialization of a generic method substituting T=int specialization void println(int i) { .. } The question asked (not answered) in SotS is: can we do this for new code? The follow-on question you are asking, I believe, is: can we compatibly convert existing families of methods to do this too? > Ok, I buy that, but it would mean that existing core libraries would have to get a massive update. Price of entry. When we talk about how expensive this stuff is, this is part of the cost. (Which, BTW, is probably the part that the community could most contribute to.) On 1/7/2015 8:43 AM, Peter Levart wrote: > Hi, > > While experimenting a bit with the current preliminary prototype, I > found myself on several occasions wanting to call a method from "normal" > API. Say System.out.println(x) with an x of type "any T". Usually a > method that already has various overloads or a method taking an Object. > Javac refuses: > > method PrintStream.println(boolean) is not applicable > (argument mismatch; T cannot be converted to boolean) > method PrintStream.println(char) is not applicable > (argument mismatch; T cannot be converted to char) > method PrintStream.println(int) is not applicable > (argument mismatch; T cannot be converted to int) > method PrintStream.println(long) is not applicable > (argument mismatch; T cannot be converted to long) > method PrintStream.println(float) is not applicable > (argument mismatch; T cannot be converted to float) > method PrintStream.println(double) is not applicable > (argument mismatch; T cannot be converted to double) > method PrintStream.println(char[]) is not applicable > (argument mismatch; T cannot be converted to char[]) > method PrintStream.println(String) is not applicable > (argument mismatch; T cannot be converted to String) > method PrintStream.println(Object) is not applicable > (argument mismatch; T cannot be converted to Object) > where T is a type-variable: > T extends declared in method test() > > > But we know that whatever instantiation of T we choose, at least one of > available overloaded methods would apply. If the set of overloaded > methods contains one with parameter of type Object, then any > instantiation of parameter of type "any T" would apply to at least one > of those methods. At specialization time, the most appropriate method > could be "selected". > > Of course this would not automatically route reference typed parameter > to the most appropriate method among those taking various reference > types - the one taking Object would always be chosen, but any value > typed paremeter could select the most appropriate method (with implicit > conversions and/or boxing). > > This could be viewed as a more natural complement to "layers" for > supporting implementation by parts. > > The specializer would either have to inspect the target class somehow, > or javac could provide the set of applicable methods in an attribute > associated with invocation. Or perhaps, such invocation could be > "specialized" as invokedynamic where the linking to the most appropriate > method (with implicit conversions and/or boxing) would be choosen at 1st > invocation. > > What do you think? > > > Regards, Peter > From peter.levart at gmail.com Wed Jan 7 16:36:46 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 07 Jan 2015 17:36:46 +0100 Subject: alternatives or complements to layers In-Reply-To: <54AD5328.3080000@oracle.com> References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> <54AD5328.3080000@oracle.com> Message-ID: <54AD609E.2080000@gmail.com> On 01/07/2015 04:39 PM, Maurizio Cimadamore wrote: > > On 07/01/15 15:37, Peter Levart wrote: >> It was said that an option is for all value types (including >> primitives) to get 3 Object methods (hashCode, equals, toString). So >> at least println is doable. > Right - I was speaking more generally of all methods which are > overloaded on the parameter type with Object, int, long, ... > > Maurizio Depends on how value types interact with interfaces. I think that the in following idiom, for example: interface SomeInterface { method() { ... } } void doSomething(T arg) { if (arg instanceof SomeInterface) { ((SomeInterface)arg).method(); ... } } It would not be necessary to box the 'arg' (in specializations where T is value type) just to invoke an interface method on it. The expression "arg instanceof SomeInterface" could be evaluated at specialization time and replaced with either "true" or "false". The cast ((SomeInterface)arg) could be replaced during specialization with either "throw CCE" or no-op. So value types could expose common behaviour (like Formattable, etc...) even to "any T" methods without resorting to boxing (which specialization is trying to avoid). Peter From vitalyd at gmail.com Wed Jan 7 16:41:30 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 11:41:30 -0500 Subject: alternatives or complements to layers In-Reply-To: <54AD609E.2080000@gmail.com> References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> <54AD5328.3080000@oracle.com> <54AD609E.2080000@gmail.com> Message-ID: I really hope we get proper generic constraints and not have to write instanceof checks for such cases. Also, I want it to be clear from looking at the method definition that it really expects a SomeInterface type internally and not just any plain T. Think of declaring generic methods in an interface that are constrained to be SomeInterface (or subtype) -- this needs to be visible purely from interface, without looking at impl. On Wed, Jan 7, 2015 at 11:36 AM, Peter Levart wrote: > On 01/07/2015 04:39 PM, Maurizio Cimadamore wrote: > >> >> On 07/01/15 15:37, Peter Levart wrote: >> >>> It was said that an option is for all value types (including primitives) >>> to get 3 Object methods (hashCode, equals, toString). So at least println >>> is doable. >>> >> Right - I was speaking more generally of all methods which are overloaded >> on the parameter type with Object, int, long, ... >> >> Maurizio >> > > Depends on how value types interact with interfaces. I think that the in > following idiom, for example: > > > interface SomeInterface { method() { ... } } > > > void doSomething(T arg) { > if (arg instanceof SomeInterface) { > ((SomeInterface)arg).method(); ... > } > } > > > It would not be necessary to box the 'arg' (in specializations where T is > value type) just to invoke an interface method on it. The expression "arg > instanceof SomeInterface" could be evaluated at specialization time and > replaced with either "true" or "false". > > The cast ((SomeInterface)arg) could be replaced during specialization with > either "throw CCE" or no-op. > > So value types could expose common behaviour (like Formattable, etc...) > even to "any T" methods without resorting to boxing (which specialization > is trying to avoid). > > > > Peter > > From brian.goetz at oracle.com Wed Jan 7 16:47:15 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jan 2015 11:47:15 -0500 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> <54AD5328.3080000@oracle.com> <54AD609E.2080000@gmail.com> Message-ID: <54AD6313.5080009@oracle.com> Ideally, yes. Where we don't currently have a story is the interaction of value types, interfaces, and bounds. It seems natural enough to say (from a source-code perspective): value class Bar implements FooAble { ... } and void foo(T t) but these don't necessarily mean exactly what you might think at first, so there are many details in which devils lurk. On 1/7/2015 11:41 AM, Vitaly Davidovich wrote: > I really hope we get proper generic constraints and not have to write > instanceof checks for such cases. Also, I want it to be clear from looking > at the method definition that it really expects a SomeInterface type > internally and not just any plain T. Think of declaring generic methods in > an interface that are constrained to be SomeInterface (or subtype) -- this > needs to be visible purely from interface, without looking at impl. > > On Wed, Jan 7, 2015 at 11:36 AM, Peter Levart > wrote: > >> On 01/07/2015 04:39 PM, Maurizio Cimadamore wrote: >> >>> >>> On 07/01/15 15:37, Peter Levart wrote: >>> >>>> It was said that an option is for all value types (including primitives) >>>> to get 3 Object methods (hashCode, equals, toString). So at least println >>>> is doable. >>>> >>> Right - I was speaking more generally of all methods which are overloaded >>> on the parameter type with Object, int, long, ... >>> >>> Maurizio >>> >> >> Depends on how value types interact with interfaces. I think that the in >> following idiom, for example: >> >> >> interface SomeInterface { method() { ... } } >> >> >> void doSomething(T arg) { >> if (arg instanceof SomeInterface) { >> ((SomeInterface)arg).method(); ... >> } >> } >> >> >> It would not be necessary to box the 'arg' (in specializations where T is >> value type) just to invoke an interface method on it. The expression "arg >> instanceof SomeInterface" could be evaluated at specialization time and >> replaced with either "true" or "false". >> >> The cast ((SomeInterface)arg) could be replaced during specialization with >> either "throw CCE" or no-op. >> >> So value types could expose common behaviour (like Formattable, etc...) >> even to "any T" methods without resorting to boxing (which specialization >> is trying to avoid). >> >> >> >> Peter >> >> From vitalyd at gmail.com Wed Jan 7 16:49:54 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 11:49:54 -0500 Subject: alternatives or complements to layers In-Reply-To: <54AD6313.5080009@oracle.com> References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> <54AD5328.3080000@oracle.com> <54AD609E.2080000@gmail.com> <54AD6313.5080009@oracle.com> Message-ID: Can you elaborate a bit on what the issue is? On Wed, Jan 7, 2015 at 11:47 AM, Brian Goetz wrote: > Ideally, yes. > > Where we don't currently have a story is the interaction of value types, > interfaces, and bounds. It seems natural enough to say (from a source-code > perspective): > > value class Bar implements FooAble { ... } > > and > > void foo(T t) > > but these don't necessarily mean exactly what you might think at first, so > there are many details in which devils lurk. > > > > On 1/7/2015 11:41 AM, Vitaly Davidovich wrote: > >> I really hope we get proper generic constraints and not have to write >> instanceof checks for such cases. Also, I want it to be clear from >> looking >> at the method definition that it really expects a SomeInterface type >> internally and not just any plain T. Think of declaring generic methods >> in >> an interface that are constrained to be SomeInterface (or subtype) -- this >> needs to be visible purely from interface, without looking at impl. >> >> On Wed, Jan 7, 2015 at 11:36 AM, Peter Levart >> wrote: >> >> On 01/07/2015 04:39 PM, Maurizio Cimadamore wrote: >>> >>> >>>> On 07/01/15 15:37, Peter Levart wrote: >>>> >>>> It was said that an option is for all value types (including >>>>> primitives) >>>>> to get 3 Object methods (hashCode, equals, toString). So at least >>>>> println >>>>> is doable. >>>>> >>>>> Right - I was speaking more generally of all methods which are >>>> overloaded >>>> on the parameter type with Object, int, long, ... >>>> >>>> Maurizio >>>> >>>> >>> Depends on how value types interact with interfaces. I think that the in >>> following idiom, for example: >>> >>> >>> interface SomeInterface { method() { ... } } >>> >>> >>> void doSomething(T arg) { >>> if (arg instanceof SomeInterface) { >>> ((SomeInterface)arg).method(); ... >>> } >>> } >>> >>> >>> It would not be necessary to box the 'arg' (in specializations where T is >>> value type) just to invoke an interface method on it. The expression "arg >>> instanceof SomeInterface" could be evaluated at specialization time and >>> replaced with either "true" or "false". >>> >>> The cast ((SomeInterface)arg) could be replaced during specialization >>> with >>> either "throw CCE" or no-op. >>> >>> So value types could expose common behaviour (like Formattable, etc...) >>> even to "any T" methods without resorting to boxing (which specialization >>> is trying to avoid). >>> >>> >>> >>> Peter >>> >>> >>> From ali.ebrahimi1781 at gmail.com Wed Jan 7 17:07:26 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Wed, 7 Jan 2015 20:37:26 +0330 Subject: alternatives or complements to layers In-Reply-To: <54AD6313.5080009@oracle.com> References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> <54AD5328.3080000@oracle.com> <54AD609E.2080000@gmail.com> <54AD6313.5080009@oracle.com> Message-ID: Hi, if value types can implement Interfaces how we can recognize interface FooAble is value type or reference type? FooAble x = test(); boolean b = x instanceof Object; b? On Wed, Jan 7, 2015 at 8:17 PM, Brian Goetz wrote: > Ideally, yes. > > Where we don't currently have a story is the interaction of value types, > interfaces, and bounds. It seems natural enough to say (from a source-code > perspective): > > value class Bar implements FooAble { ... } > > and > > void foo(T t) > > but these don't necessarily mean exactly what you might think at first, so > there are many details in which devils lurk. > > > > On 1/7/2015 11:41 AM, Vitaly Davidovich wrote: > >> I really hope we get proper generic constraints and not have to write >> instanceof checks for such cases. Also, I want it to be clear from >> looking >> at the method definition that it really expects a SomeInterface type >> internally and not just any plain T. Think of declaring generic methods >> in >> an interface that are constrained to be SomeInterface (or subtype) -- this >> needs to be visible purely from interface, without looking at impl. >> >> On Wed, Jan 7, 2015 at 11:36 AM, Peter Levart >> wrote: >> >> On 01/07/2015 04:39 PM, Maurizio Cimadamore wrote: >>> >>> >>>> On 07/01/15 15:37, Peter Levart wrote: >>>> >>>> It was said that an option is for all value types (including >>>>> primitives) >>>>> to get 3 Object methods (hashCode, equals, toString). So at least >>>>> println >>>>> is doable. >>>>> >>>>> Right - I was speaking more generally of all methods which are >>>> overloaded >>>> on the parameter type with Object, int, long, ... >>>> >>>> Maurizio >>>> >>>> >>> Depends on how value types interact with interfaces. I think that the in >>> following idiom, for example: >>> >>> >>> interface SomeInterface { method() { ... } } >>> >>> >>> void doSomething(T arg) { >>> if (arg instanceof SomeInterface) { >>> ((SomeInterface)arg).method(); ... >>> } >>> } >>> >>> >>> It would not be necessary to box the 'arg' (in specializations where T is >>> value type) just to invoke an interface method on it. The expression "arg >>> instanceof SomeInterface" could be evaluated at specialization time and >>> replaced with either "true" or "false". >>> >>> The cast ((SomeInterface)arg) could be replaced during specialization >>> with >>> either "throw CCE" or no-op. >>> >>> So value types could expose common behaviour (like Formattable, etc...) >>> even to "any T" methods without resorting to boxing (which specialization >>> is trying to avoid). >>> >>> >>> >>> Peter >>> >>> >>> -- Best Regards, Ali Ebrahimi From vitalyd at gmail.com Wed Jan 7 17:21:45 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 12:21:45 -0500 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> <54AD5328.3080000@oracle.com> <54AD609E.2080000@gmail.com> <54AD6313.5080009@oracle.com> Message-ID: In this case I'd expect the value type to get boxed and b = true. On Wed, Jan 7, 2015 at 12:07 PM, Ali Ebrahimi wrote: > Hi, > if value types can implement Interfaces how we can recognize interface > FooAble is value type or reference type? > > FooAble x = test(); > boolean b = x instanceof Object; > > b? > > > > On Wed, Jan 7, 2015 at 8:17 PM, Brian Goetz > wrote: > > > Ideally, yes. > > > > Where we don't currently have a story is the interaction of value types, > > interfaces, and bounds. It seems natural enough to say (from a > source-code > > perspective): > > > > value class Bar implements FooAble { ... } > > > > and > > > > void foo(T t) > > > > but these don't necessarily mean exactly what you might think at first, > so > > there are many details in which devils lurk. > > > > > > > > On 1/7/2015 11:41 AM, Vitaly Davidovich wrote: > > > >> I really hope we get proper generic constraints and not have to write > >> instanceof checks for such cases. Also, I want it to be clear from > >> looking > >> at the method definition that it really expects a SomeInterface type > >> internally and not just any plain T. Think of declaring generic methods > >> in > >> an interface that are constrained to be SomeInterface (or subtype) -- > this > >> needs to be visible purely from interface, without looking at impl. > >> > >> On Wed, Jan 7, 2015 at 11:36 AM, Peter Levart > >> wrote: > >> > >> On 01/07/2015 04:39 PM, Maurizio Cimadamore wrote: > >>> > >>> > >>>> On 07/01/15 15:37, Peter Levart wrote: > >>>> > >>>> It was said that an option is for all value types (including > >>>>> primitives) > >>>>> to get 3 Object methods (hashCode, equals, toString). So at least > >>>>> println > >>>>> is doable. > >>>>> > >>>>> Right - I was speaking more generally of all methods which are > >>>> overloaded > >>>> on the parameter type with Object, int, long, ... > >>>> > >>>> Maurizio > >>>> > >>>> > >>> Depends on how value types interact with interfaces. I think that the > in > >>> following idiom, for example: > >>> > >>> > >>> interface SomeInterface { method() { ... } } > >>> > >>> > >>> void doSomething(T arg) { > >>> if (arg instanceof SomeInterface) { > >>> ((SomeInterface)arg).method(); ... > >>> } > >>> } > >>> > >>> > >>> It would not be necessary to box the 'arg' (in specializations where T > is > >>> value type) just to invoke an interface method on it. The expression > "arg > >>> instanceof SomeInterface" could be evaluated at specialization time and > >>> replaced with either "true" or "false". > >>> > >>> The cast ((SomeInterface)arg) could be replaced during specialization > >>> with > >>> either "throw CCE" or no-op. > >>> > >>> So value types could expose common behaviour (like Formattable, etc...) > >>> even to "any T" methods without resorting to boxing (which > specialization > >>> is trying to avoid). > >>> > >>> > >>> > >>> Peter > >>> > >>> > >>> > > > -- > > Best Regards, > Ali Ebrahimi > From ali.ebrahimi1781 at gmail.com Wed Jan 7 17:48:40 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Wed, 7 Jan 2015 21:18:40 +0330 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> <54AD5328.3080000@oracle.com> <54AD609E.2080000@gmail.com> <54AD6313.5080009@oracle.com> Message-ID: So only value boxes can implement interfaces? yes? On Wed, Jan 7, 2015 at 8:51 PM, Vitaly Davidovich wrote: > In this case I'd expect the value type to get boxed and b = true. > > On Wed, Jan 7, 2015 at 12:07 PM, Ali Ebrahimi > wrote: > >> Hi, >> if value types can implement Interfaces how we can recognize interface >> FooAble is value type or reference type? >> >> FooAble x = test(); >> boolean b = x instanceof Object; >> >> b? >> >> >> >> On Wed, Jan 7, 2015 at 8:17 PM, Brian Goetz >> wrote: >> >> > Ideally, yes. >> > >> > Where we don't currently have a story is the interaction of value types, >> > interfaces, and bounds. It seems natural enough to say (from a >> source-code >> > perspective): >> > >> > value class Bar implements FooAble { ... } >> > >> > and >> > >> > void foo(T t) >> > >> > but these don't necessarily mean exactly what you might think at first, >> so >> > there are many details in which devils lurk. >> > >> > >> > >> > On 1/7/2015 11:41 AM, Vitaly Davidovich wrote: >> > >> >> I really hope we get proper generic constraints and not have to write >> >> instanceof checks for such cases. Also, I want it to be clear from >> >> looking >> >> at the method definition that it really expects a SomeInterface type >> >> internally and not just any plain T. Think of declaring generic >> methods >> >> in >> >> an interface that are constrained to be SomeInterface (or subtype) -- >> this >> >> needs to be visible purely from interface, without looking at impl. >> >> >> >> On Wed, Jan 7, 2015 at 11:36 AM, Peter Levart >> >> wrote: >> >> >> >> On 01/07/2015 04:39 PM, Maurizio Cimadamore wrote: >> >>> >> >>> >> >>>> On 07/01/15 15:37, Peter Levart wrote: >> >>>> >> >>>> It was said that an option is for all value types (including >> >>>>> primitives) >> >>>>> to get 3 Object methods (hashCode, equals, toString). So at least >> >>>>> println >> >>>>> is doable. >> >>>>> >> >>>>> Right - I was speaking more generally of all methods which are >> >>>> overloaded >> >>>> on the parameter type with Object, int, long, ... >> >>>> >> >>>> Maurizio >> >>>> >> >>>> >> >>> Depends on how value types interact with interfaces. I think that the >> in >> >>> following idiom, for example: >> >>> >> >>> >> >>> interface SomeInterface { method() { ... } } >> >>> >> >>> >> >>> void doSomething(T arg) { >> >>> if (arg instanceof SomeInterface) { >> >>> ((SomeInterface)arg).method(); ... >> >>> } >> >>> } >> >>> >> >>> >> >>> It would not be necessary to box the 'arg' (in specializations where >> T is >> >>> value type) just to invoke an interface method on it. The expression >> "arg >> >>> instanceof SomeInterface" could be evaluated at specialization time >> and >> >>> replaced with either "true" or "false". >> >>> >> >>> The cast ((SomeInterface)arg) could be replaced during specialization >> >>> with >> >>> either "throw CCE" or no-op. >> >>> >> >>> So value types could expose common behaviour (like Formattable, >> etc...) >> >>> even to "any T" methods without resorting to boxing (which >> specialization >> >>> is trying to avoid). >> >>> >> >>> >> >>> >> >>> Peter >> >>> >> >>> >> >>> >> >> >> -- >> >> Best Regards, >> Ali Ebrahimi >> > > -- Best Regards, Ali Ebrahimi From simon at ochsenreither.de Wed Jan 7 18:00:12 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Wed, 07 Jan 2015 19:00:12 +0100 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> <54AD5328.3080000@oracle.com> <54AD609E.2080000@gmail.com> <54AD6313.5080009@oracle.com> Message-ID: <54AD742C.3030303@ochsenreither.de> > So only value boxes can implement interfaces? yes? I think that would be a bad idea. In fact, C# added an IEquatable interface in 2.0 to work around the need to box value types for a popular method. From vitalyd at gmail.com Wed Jan 7 18:01:27 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 13:01:27 -0500 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> <54AD5328.3080000@oracle.com> <54AD609E.2080000@gmail.com> <54AD6313.5080009@oracle.com> Message-ID: This is how this works in .NET; not saying it has to be the same in java, but i find their treatment fairly sound. Assigning a value type to an interface directly boxes; the box implements the same interface and forwards the calls to the value type itself. To avoid boxing yet work with interfaces, they allow generics to be constrained; when you specify the type as a value type for said generic type, there's no boxing - the CLR specializes the type internally. Sent from my phone On Jan 7, 2015 12:48 PM, "Ali Ebrahimi" wrote: > So only value boxes can implement interfaces? yes? > > On Wed, Jan 7, 2015 at 8:51 PM, Vitaly Davidovich > wrote: > >> In this case I'd expect the value type to get boxed and b = true. >> >> On Wed, Jan 7, 2015 at 12:07 PM, Ali Ebrahimi > > wrote: >> >>> Hi, >>> if value types can implement Interfaces how we can recognize interface >>> FooAble is value type or reference type? >>> >>> FooAble x = test(); >>> boolean b = x instanceof Object; >>> >>> b? >>> >>> >>> >>> On Wed, Jan 7, 2015 at 8:17 PM, Brian Goetz >>> wrote: >>> >>> > Ideally, yes. >>> > >>> > Where we don't currently have a story is the interaction of value >>> types, >>> > interfaces, and bounds. It seems natural enough to say (from a >>> source-code >>> > perspective): >>> > >>> > value class Bar implements FooAble { ... } >>> > >>> > and >>> > >>> > void foo(T t) >>> > >>> > but these don't necessarily mean exactly what you might think at >>> first, so >>> > there are many details in which devils lurk. >>> > >>> > >>> > >>> > On 1/7/2015 11:41 AM, Vitaly Davidovich wrote: >>> > >>> >> I really hope we get proper generic constraints and not have to write >>> >> instanceof checks for such cases. Also, I want it to be clear from >>> >> looking >>> >> at the method definition that it really expects a SomeInterface type >>> >> internally and not just any plain T. Think of declaring generic >>> methods >>> >> in >>> >> an interface that are constrained to be SomeInterface (or subtype) -- >>> this >>> >> needs to be visible purely from interface, without looking at impl. >>> >> >>> >> On Wed, Jan 7, 2015 at 11:36 AM, Peter Levart >> > >>> >> wrote: >>> >> >>> >> On 01/07/2015 04:39 PM, Maurizio Cimadamore wrote: >>> >>> >>> >>> >>> >>>> On 07/01/15 15:37, Peter Levart wrote: >>> >>>> >>> >>>> It was said that an option is for all value types (including >>> >>>>> primitives) >>> >>>>> to get 3 Object methods (hashCode, equals, toString). So at least >>> >>>>> println >>> >>>>> is doable. >>> >>>>> >>> >>>>> Right - I was speaking more generally of all methods which are >>> >>>> overloaded >>> >>>> on the parameter type with Object, int, long, ... >>> >>>> >>> >>>> Maurizio >>> >>>> >>> >>>> >>> >>> Depends on how value types interact with interfaces. I think that >>> the in >>> >>> following idiom, for example: >>> >>> >>> >>> >>> >>> interface SomeInterface { method() { ... } } >>> >>> >>> >>> >>> >>> void doSomething(T arg) { >>> >>> if (arg instanceof SomeInterface) { >>> >>> ((SomeInterface)arg).method(); ... >>> >>> } >>> >>> } >>> >>> >>> >>> >>> >>> It would not be necessary to box the 'arg' (in specializations where >>> T is >>> >>> value type) just to invoke an interface method on it. The expression >>> "arg >>> >>> instanceof SomeInterface" could be evaluated at specialization time >>> and >>> >>> replaced with either "true" or "false". >>> >>> >>> >>> The cast ((SomeInterface)arg) could be replaced during specialization >>> >>> with >>> >>> either "throw CCE" or no-op. >>> >>> >>> >>> So value types could expose common behaviour (like Formattable, >>> etc...) >>> >>> even to "any T" methods without resorting to boxing (which >>> specialization >>> >>> is trying to avoid). >>> >>> >>> >>> >>> >>> >>> >>> Peter >>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Best Regards, >>> Ali Ebrahimi >>> >> >> > > > -- > > Best Regards, > Ali Ebrahimi > From brian.goetz at oracle.com Wed Jan 7 18:12:14 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jan 2015 13:12:14 -0500 Subject: alternatives or complements to layers In-Reply-To: References: <54AD37FF.9060301@gmail.com> <54AD4F40.8010809@oracle.com> <54AD52B0.8030503@gmail.com> <54AD5328.3080000@oracle.com> <54AD609E.2080000@gmail.com> <54AD6313.5080009@oracle.com> Message-ID: <54AD76FE.4020408@oracle.com> (And this is more or less consistent with what we've been envisioning.) On 1/7/2015 1:01 PM, Vitaly Davidovich wrote: > This is how this works in .NET; not saying it has to be the same in > java, but i find their treatment fairly sound. > > Assigning a value type to an interface directly boxes; the box > implements the same interface and forwards the calls to the value type > itself. To avoid boxing yet work with interfaces, they allow generics > to be constrained; when you specify the type as a value type for said > generic type, there's no boxing - the CLR specializes the type internally. > > Sent from my phone > > On Jan 7, 2015 12:48 PM, "Ali Ebrahimi" > wrote: > > So only value boxes can implement interfaces? yes? > > On Wed, Jan 7, 2015 at 8:51 PM, Vitaly Davidovich > wrote: > > In this case I'd expect the value type to get boxed and b = true. > > On Wed, Jan 7, 2015 at 12:07 PM, Ali Ebrahimi > > > wrote: > > Hi, > if value types can implement Interfaces how we can > recognize interface > FooAble is value type or reference type? > > FooAble x = test(); > boolean b = x instanceof Object; > > b? > > > > On Wed, Jan 7, 2015 at 8:17 PM, Brian Goetz > > wrote: > > > Ideally, yes. > > > > Where we don't currently have a story is the interaction > of value types, > > interfaces, and bounds. It seems natural enough to say > (from a source-code > > perspective): > > > > value class Bar implements FooAble { ... } > > > > and > > > > void foo(T t) > > > > but these don't necessarily mean exactly what you might > think at first, so > > there are many details in which devils lurk. > > > > > > > > On 1/7/2015 11:41 AM, Vitaly Davidovich wrote: > > > >> I really hope we get proper generic constraints and not > have to write > >> instanceof checks for such cases. Also, I want it to be > clear from > >> looking > >> at the method definition that it really expects a > SomeInterface type > >> internally and not just any plain T. Think of declaring > generic methods > >> in > >> an interface that are constrained to be SomeInterface > (or subtype) -- this > >> needs to be visible purely from interface, without > looking at impl. > >> > >> On Wed, Jan 7, 2015 at 11:36 AM, Peter Levart > > > >> wrote: > >> > >> On 01/07/2015 04:39 PM, Maurizio Cimadamore wrote: > >>> > >>> > >>>> On 07/01/15 15:37, Peter Levart wrote: > >>>> > >>>> It was said that an option is for all value types > (including > >>>>> primitives) > >>>>> to get 3 Object methods (hashCode, equals, toString). > So at least > >>>>> println > >>>>> is doable. > >>>>> > >>>>> Right - I was speaking more generally of all methods > which are > >>>> overloaded > >>>> on the parameter type with Object, int, long, ... > >>>> > >>>> Maurizio > >>>> > >>>> > >>> Depends on how value types interact with interfaces. I > think that the in > >>> following idiom, for example: > >>> > >>> > >>> interface SomeInterface { method() { ... } } > >>> > >>> > >>> void doSomething(T arg) { > >>> if (arg instanceof SomeInterface) { > >>> ((SomeInterface)arg).method(); ... > >>> } > >>> } > >>> > >>> > >>> It would not be necessary to box the 'arg' (in > specializations where T is > >>> value type) just to invoke an interface method on it. > The expression "arg > >>> instanceof SomeInterface" could be evaluated at > specialization time and > >>> replaced with either "true" or "false". > >>> > >>> The cast ((SomeInterface)arg) could be replaced during > specialization > >>> with > >>> either "throw CCE" or no-op. > >>> > >>> So value types could expose common behaviour (like > Formattable, etc...) > >>> even to "any T" methods without resorting to boxing > (which specialization > >>> is trying to avoid). > >>> > >>> > >>> > >>> Peter > >>> > >>> > >>> > > > -- > > Best Regards, > Ali Ebrahimi > > > > > > -- > > Best Regards, > Ali Ebrahimi > From melbeltagy at yahoo.com Wed Jan 7 18:55:04 2015 From: melbeltagy at yahoo.com (Mohamed M. El-Beltagy) Date: Wed, 7 Jan 2015 18:55:04 +0000 (UTC) Subject: alternatives or complements to layers In-Reply-To: <54AD76FE.4020408@oracle.com> References: <54AD76FE.4020408@oracle.com> Message-ID: <1190945487.1495898.1420656904278.JavaMail.yahoo@jws100105.mail.ne1.yahoo.com> Dear all. Kindly bear with me for just five minutes. I think that I really have to ask this question at this inappropriate timing. Inappropriate cause Alan Bateman stated; implicitly of course; to move on and start testing the proposed solution instead of wasting time discussing alternatives. Which is a decision I highly respect for its timing. But yet, reading all these emails, 100+ in just couple of days, I feel that I must ask. The question simply is: Why not change primitives on the language level so that they would be special objects? Meaning, they would fall back to be an object. Benefits? I would dare to say the following: - No changes on the language itself. i.e., no need to introduce the type "any". Unless of course; I missed other objective other than Generics.- All issues rose during this specific thread regarding APIs' and backward compatibility would simply vanish. - If all primitives are at the end objects; Generics will work like a charm.I don't want to waste more of your time reading the benefits I can think of; I'm sure you'll find even more than I can. Disadvantages? At the top of my head is auto boxing behavior and figuring out a way to differentiate between them. Thank you very much for your time reading this email and at the same time; kindly accept my apologize if you feel that I wasted your time. From melbeltagy at yahoo.com Wed Jan 7 19:07:34 2015 From: melbeltagy at yahoo.com (Mohamed M. El-Beltagy) Date: Wed, 7 Jan 2015 19:07:34 +0000 (UTC) Subject: alternatives or complements to layers In-Reply-To: <1190945487.1495898.1420656904278.JavaMail.yahoo@jws100105.mail.ne1.yahoo.com> References: <54AD76FE.4020408@oracle.com> <1190945487.1495898.1420656904278.JavaMail.yahoo@jws100105.mail.ne1.yahoo.com> Message-ID: <1828364016.3586786.1420657654978.JavaMail.yahoo@jws10085.mail.ne1.yahoo.com> Terribly sorry for wasting your time with the previous email. I just read Ron Pressler?s thread. Kindly accept my apologize. On Wednesday, January 7, 2015 9:55 PM, Mohamed M. El-Beltagy wrote: Dear all. Kindly bear with me for just five minutes. I think that I really have to ask this question at this inappropriate timing. Inappropriate cause Alan Bateman stated; implicitly of course; to move on and start testing the proposed solution instead of wasting time discussing alternatives. Which is a decision I highly respect for its timing. But yet, reading all these emails, 100+ in just couple of days, I feel that I must ask. The question simply is: Why not change primitives on the language level so that they would be special objects? Meaning, they would fall back to be an object. Benefits? I would dare to say the following: - No changes on the language itself. i.e., no need to introduce the type "any". Unless of course; I missed other objective other than Generics.- All issues rose during this specific thread regarding APIs' and backward compatibility would simply vanish. - If all primitives are at the end objects; Generics will work like a charm.I don't want to waste more of your time reading the benefits I can think of; I'm sure you'll find even more than I can. Disadvantages? At the top of my head is auto boxing behavior and figuring out a way to differentiate between them. Thank you very much for your time reading this email and at the same time; kindly accept my apologize if you feel that I wasted your time. ? From ivan.st.ivanov at gmail.com Wed Jan 7 20:21:02 2015 From: ivan.st.ivanov at gmail.com (Ivan St. Ivanov) Date: Wed, 7 Jan 2015 22:21:02 +0200 Subject: Can we also get some feedback on specialization, please? In-Reply-To: References: <54AAE765.6040402@oracle.com> <54AB25E3.1060308@oracle.com> Message-ID: Hi folks, We, from the Bulgarian JUG are also planning to do a hackathon on this. Most probably it will be on January 29th (just before FOSDEM). We'll update the VMs we already have and will also, as usual, write some blogs. As I am taking care of presenting this to my fellows here in Bulgaria and as I am poorly prepared in the language and type theory, would you please tell me where can I read more what is a compiler fiction, specialization vs erasure vs reification and all the things that are discussed in the various thread an in the state of specialization paper? Sorry for the stupid question, but I really want to get my folks here (including myself) up to speed. Thanks, Ivan On Tue, Jan 6, 2015 at 11:35 AM, Martijn Verburg wrote: > Hi all, > > On 6 January 2015 at 00:19, Richard Warburton > > wrote: > > > Hi Brian, > > > > It should cover all the conversions outlined in the latest SotS. There > are > > > a few known things that don't work quite right yet, things like: > > > > > > - Nonstatic inner classes and local classes (we're close on this, > > though) > > > - super-calls inside generic methods > > > > > > But, it should be enough to write an ArrayList-like class that is > generic > > > in any-T. > > > > > > > Thanks. > > > > To run it, just clone and build the valhalla repo (make images), and use > > > that as your JDK. > > > > > > > Unless I really need sleep (it is after midnight here) there are no links > > from the project home page (http://openjdk.java.net/projects/valhalla/) > to > > the source repositories (http://hg.openjdk.java.net/valhalla). I imagine > > this might cause confusion for casual users who don't know the mapping > > between project names and repo urls and it would be nice to add it. Or > even > > link to http://hg.openjdk.java.net/valhalla/valhalla - since that's what > > they'll need to hg clone. > > > > To this point I'd like to submit a 'patch' to the website page - is there a > source file for the web page somewhere that I can create a patch against? > > Cheers, > Martijn > > > > > > regards, > > > > Richard Warburton > > > > http://insightfullogic.com > > @RichardWarburto > > > From sven.reimers at gmail.com Wed Jan 7 21:03:08 2015 From: sven.reimers at gmail.com (Sven Reimers) Date: Wed, 7 Jan 2015 22:03:08 +0100 Subject: Fwd: java.lang.VerifyError: Bad type on operand stack In-Reply-To: <54AD17B8.8050604@oracle.com> References: <54ABFAC8.5000308@oracle.com> <54AD17B8.8050604@oracle.com> Message-ID: Ah, ok got it.. Indeed changing it to the following line compiles fine. Point3D integerPoint = new Point3D<>(Integer.valueOf(1),Integer.valueOf(2),Integer.valueOf(3)); Thanks Sven On Wed, Jan 7, 2015 at 12:25 PM, Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > I consulted with the inference Gods (they seem to be lurking in Valhalla > from time to time) and it would seem that this is actually the correct > behavior. > The problem is that the constructor is considered perfectly applicable > during the 'strict' applicability step, which only uses subtyping - > therefore the only allowed inference result there is 'int' - and the target > type seems to contradict there. It is possible that we will relax this a > bit moving forward by introducing some ad hoc rules, but the status quo > seems to be expressive enough, while avoiding being unnecessarily messy. > > At the same time, the Gods pointed out that inference should only be > allowed to do boxing/unboxing tricks when an inference constraints of the > kind > > alpha :> int > > has been derived from a compatibility check (i.e. because we want to pass > an 'int' where 'alpha' is expected). These tricks should not be allowed in > the general case - example: > > T[] clone(T[] arg) { return arg; } > > int[] arr1 = clone(new int[3]); //ok > Integer[] arr2 = clone(new int[3]); //fail > > Here, the array subtyping rules essentially mean that no boxing can > occurr, so the only choice for T is 'int' in the second case, which, again, > is incompatible with the target. > > The compiler gets some of this right and some other things only happen to > be right 'by accident' - I'm working on a patch that will clean this up a > bit. > > Maurizio > > > On 06/01/15 16:27, Sven Reimers wrote: > >> @Maurizio: sorry forgot the mailing list... >> >> >> ---------- Forwarded message ---------- >> From: Sven Reimers >> Date: Tue, Jan 6, 2015 at 5:20 PM >> Subject: Re: java.lang.VerifyError: Bad type on operand stack >> To: Maurizio Cimadamore >> >> >> Follow-Up: >> >> I tried a few different things and got into this: >> >> Point3D integerPoint = new Point3D<>(1,2,3); >> >> compiles only if >> >> public static class Point3D {... } >> >> not for >> >> public static class Point3D {... } >> >> ErrorMessage from the compiler is: >> >> incompatible types: cannot infer type arguments for Point3D<> >> Point3D integerPoint = new Point3D<>(1,2,3); >> ^ >> reason: cannot infer type-variable(s) T >> (argument mismatch; int cannot be converted to Integer) >> where T is a type-variable: >> T extends declared in class Point3D >> >> Is this intentional (not sure I found this stated in the draft)? >> >> Thanks >> >> Sven >> >> >> On Tue, Jan 6, 2015 at 4:10 PM, Maurizio Cimadamore < >> maurizio.cimadamore at oracle.com> wrote: >> >> Whoops :-) >>> >>> Thanks for the report >>> >>> Maurizio >>> >>> >>> On 06/01/15 15:13, Sven Reimers wrote: >>> >>> Hi, >>>> >>>> as suggested by Brian I tried to get some sample code written and >>>> running >>>> on top of valhalla. >>>> >>>> First problem I encountered was that a missing diamond <> on the right >>>> hand >>>> side (my bad - seems I am really relying on my IDE) was just a warning >>>> that >>>> I was using a raw type. >>>> >>>> Running my example I got this: >>>> >>>> Specializing Valhalla$Point3D${0=I}; searching for >>>> Valhalla$Point3D.class >>>> (not found) >>>> Specializing Valhalla$Point3D${0=I}; searching for >>>> Valhalla$Point3D.class >>>> (found) >>>> Error: A JNI error has occurred, please check your installation and try >>>> again >>>> Exception in thread "main" java.lang.VerifyError: Bad type on operand >>>> stack >>>> Exception Details: >>>> Location: >>>> Valhalla.main([Ljava/lang/String;)V @36: invokevirtual >>>> Reason: >>>> Type 'Valhalla$Point3D' (current frame, stack[2]) is not >>>> assignable >>>> to >>>> 'Valhalla$Point3D${0=I}' >>>> Current Frame: >>>> bci: @36 >>>> flags: { } >>>> locals: { '[Ljava/lang/String;', 'Valhalla$Point3D' } >>>> stack: { 'java/io/PrintStream', 'java/lang/StringBuilder', >>>> 'Valhalla$Point3D' } >>>> Bytecode: >>>> 0000000: bb00 0259 04b8 0003 05b8 0003 06b8 0003 >>>> 0000010: b700 044c b200 05bb 0006 59b7 0007 1208 >>>> 0000020: b600 092b b600 0ab6 000b b600 0cb6 000d >>>> 0000030: b1 >>>> >>>> at java.lang.Class.getDeclaredMethods0(Native Method) >>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2704) >>>> at java.lang.Class.privateGetMethodRecursive(Class.java:3049) >>>> at java.lang.Class.getMethod0(Class.java:3019) >>>> at java.lang.Class.getMethod(Class.java:1787) >>>> at sun.launcher.LauncherHelper.validateMainClass( >>>> LauncherHelper.java:568) >>>> at sun.launcher.LauncherHelper.checkAndLoadMain( >>>> LauncherHelper.java:535) >>>> >>>> Example Code (see the missing <> just after Point3D...): >>>> >>>> public class Valhalla { >>>> >>>> >>>> public static void main (String[] args) { >>>> >>>> Point3D intPoint = new Point3D(1,2,3); >>>> >>>> System.out.println("X: " + intPoint.getX()); >>>> >>>> } >>>> >>>> public static class Point3D { >>>> private T x; >>>> private T y; >>>> private T z; >>>> >>>> public Point3D(T x, T y, T z) { >>>> this.x = x; >>>> this.y = y; >>>> this.z = z; >>>> } >>>> >>>> public T getX() { >>>> return x; >>>> } >>>> >>>> public T getY() { >>>> return y; >>>> } >>>> >>>> public T getZ() { >>>> return z; >>>> } >>>> >>>> } >>>> >>>> } >>>> >>>> Hope to provide more feedback once I got more code running. >>>> >>>> Thanks for the impressive work so far. >>>> >>>> -Sven >>>> >>>> >>>> >> > -- Sven Reimers * Senior Expert Software Architect * NetBeans Dream Team Member: http://dreamteam.netbeans.org * Community Leader NetBeans: http://community.java.net/netbeans Desktop Java: http://community.java.net/javadesktop * JUG Leader JUG Bodensee: http://www.jug-bodensee.de * Duke's Choice Award Winner 2009 * Blog: https://www.java.net//blog/sven * XING: https://www.xing.com/profile/Sven_Reimers8 * LinkedIn: http://www.linkedin.com/in/svenreimers Join the NetBeans Groups: * XING: http://www.xing.com/group-20148.82db20 * NUGM: http://haug-server.dyndns.org/display/NUGM/Home * LinkedIn: http://www.linkedin.com/groups?gid=1860468 http://www.linkedin.com/groups?gid=107402 http://www.linkedin.com/groups?gid=1684717 * Oracle: https://mix.oracle.com/groups/18497 From martijnverburg at gmail.com Wed Jan 7 21:04:07 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Wed, 7 Jan 2015 21:04:07 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: References: <54AAE765.6040402@oracle.com> <54AB25E3.1060308@oracle.com> Message-ID: Hi Ivan, To be honest, it's probably a bit early for your group to jump in for valhalla-dev related sessions. FWIW - In London we're really only going to involve the couple of people who are already on this list and understand the fundamentals of the proposal. Cheers, Martijn On 7 January 2015 at 20:21, Ivan St. Ivanov wrote: > Hi folks, > > We, from the Bulgarian JUG are also planning to do a hackathon on this. > Most probably it will be on January 29th (just before FOSDEM). We'll update > the VMs we already have and will also, as usual, write some blogs. > > As I am taking care of presenting this to my fellows here in Bulgaria and > as I am poorly prepared in the language and type theory, would you please > tell me where can I read more what is a compiler fiction, specialization vs > erasure vs reification and all the things that are discussed in the various > thread an in the state of specialization paper? Sorry for the stupid > question, but I really want to get my folks here (including myself) up to > speed. > > Thanks, > Ivan > > On Tue, Jan 6, 2015 at 11:35 AM, Martijn Verburg > wrote: > >> Hi all, >> >> On 6 January 2015 at 00:19, Richard Warburton < >> richard.warburton at gmail.com> >> wrote: >> >> > Hi Brian, >> > >> > It should cover all the conversions outlined in the latest SotS. There >> are >> > > a few known things that don't work quite right yet, things like: >> > > >> > > - Nonstatic inner classes and local classes (we're close on this, >> > though) >> > > - super-calls inside generic methods >> > > >> > > But, it should be enough to write an ArrayList-like class that is >> generic >> > > in any-T. >> > > >> > >> > Thanks. >> > >> > To run it, just clone and build the valhalla repo (make images), and use >> > > that as your JDK. >> > > >> > >> > Unless I really need sleep (it is after midnight here) there are no >> links >> > from the project home page (http://openjdk.java.net/projects/valhalla/) >> to >> > the source repositories (http://hg.openjdk.java.net/valhalla). I >> imagine >> > this might cause confusion for casual users who don't know the mapping >> > between project names and repo urls and it would be nice to add it. Or >> even >> > link to http://hg.openjdk.java.net/valhalla/valhalla - since that's >> what >> > they'll need to hg clone. >> > >> >> To this point I'd like to submit a 'patch' to the website page - is there >> a >> source file for the web page somewhere that I can create a patch against? >> >> Cheers, >> Martijn >> >> >> > >> > regards, >> > >> > Richard Warburton >> > >> > http://insightfullogic.com >> > @RichardWarburto >> > >> > > From sven.reimers at gmail.com Wed Jan 7 21:07:21 2015 From: sven.reimers at gmail.com (Sven Reimers) Date: Wed, 7 Jan 2015 22:07:21 +0100 Subject: Can we also get some feedback on specialization, please? In-Reply-To: References: <54AAE765.6040402@oracle.com> <54AB25E3.1060308@oracle.com> Message-ID: .. and are ready to use the compiler on the command line I assume... ;-) Sven On Wed, Jan 7, 2015 at 10:04 PM, Martijn Verburg wrote: > Hi Ivan, > > To be honest, it's probably a bit early for your group to jump in for > valhalla-dev related sessions. FWIW - In London we're really only going to > involve the couple of people who are already on this list and understand > the fundamentals of the proposal. > > Cheers, > Martijn > > On 7 January 2015 at 20:21, Ivan St. Ivanov > wrote: > > > Hi folks, > > > > We, from the Bulgarian JUG are also planning to do a hackathon on this. > > Most probably it will be on January 29th (just before FOSDEM). We'll > update > > the VMs we already have and will also, as usual, write some blogs. > > > > As I am taking care of presenting this to my fellows here in Bulgaria and > > as I am poorly prepared in the language and type theory, would you please > > tell me where can I read more what is a compiler fiction, specialization > vs > > erasure vs reification and all the things that are discussed in the > various > > thread an in the state of specialization paper? Sorry for the stupid > > question, but I really want to get my folks here (including myself) up to > > speed. > > > > Thanks, > > Ivan > > > > On Tue, Jan 6, 2015 at 11:35 AM, Martijn Verburg < > martijnverburg at gmail.com > > > wrote: > > > >> Hi all, > >> > >> On 6 January 2015 at 00:19, Richard Warburton < > >> richard.warburton at gmail.com> > >> wrote: > >> > >> > Hi Brian, > >> > > >> > It should cover all the conversions outlined in the latest SotS. > There > >> are > >> > > a few known things that don't work quite right yet, things like: > >> > > > >> > > - Nonstatic inner classes and local classes (we're close on this, > >> > though) > >> > > - super-calls inside generic methods > >> > > > >> > > But, it should be enough to write an ArrayList-like class that is > >> generic > >> > > in any-T. > >> > > > >> > > >> > Thanks. > >> > > >> > To run it, just clone and build the valhalla repo (make images), and > use > >> > > that as your JDK. > >> > > > >> > > >> > Unless I really need sleep (it is after midnight here) there are no > >> links > >> > from the project home page ( > http://openjdk.java.net/projects/valhalla/) > >> to > >> > the source repositories (http://hg.openjdk.java.net/valhalla). I > >> imagine > >> > this might cause confusion for casual users who don't know the mapping > >> > between project names and repo urls and it would be nice to add it. Or > >> even > >> > link to http://hg.openjdk.java.net/valhalla/valhalla - since that's > >> what > >> > they'll need to hg clone. > >> > > >> > >> To this point I'd like to submit a 'patch' to the website page - is > there > >> a > >> source file for the web page somewhere that I can create a patch > against? > >> > >> Cheers, > >> Martijn > >> > >> > >> > > >> > regards, > >> > > >> > Richard Warburton > >> > > >> > http://insightfullogic.com > >> > @RichardWarburto > >> > > >> > > > > > -- Sven Reimers * Senior Expert Software Architect * NetBeans Dream Team Member: http://dreamteam.netbeans.org * Community Leader NetBeans: http://community.java.net/netbeans Desktop Java: http://community.java.net/javadesktop * JUG Leader JUG Bodensee: http://www.jug-bodensee.de * Duke's Choice Award Winner 2009 * Blog: https://www.java.net//blog/sven * XING: https://www.xing.com/profile/Sven_Reimers8 * LinkedIn: http://www.linkedin.com/in/svenreimers Join the NetBeans Groups: * XING: http://www.xing.com/group-20148.82db20 * NUGM: http://haug-server.dyndns.org/display/NUGM/Home * LinkedIn: http://www.linkedin.com/groups?gid=1860468 http://www.linkedin.com/groups?gid=107402 http://www.linkedin.com/groups?gid=1684717 * Oracle: https://mix.oracle.com/groups/18497 From brian.goetz at oracle.com Wed Jan 7 21:08:59 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jan 2015 16:08:59 -0500 Subject: Can we also get some feedback on specialization, please? In-Reply-To: References: <54AAE765.6040402@oracle.com> <54AB25E3.1060308@oracle.com> Message-ID: <54ADA06B.5080800@oracle.com> It's way more raw than that! More like, are ready to work around "oh, it seems they have not yet implemented XXX, I'll try something else". On 1/7/2015 4:07 PM, Sven Reimers wrote: > .. and are ready to use the compiler on the command line I assume... > > ;-) > > Sven > > On Wed, Jan 7, 2015 at 10:04 PM, Martijn Verburg > wrote: > >> Hi Ivan, >> >> To be honest, it's probably a bit early for your group to jump in for >> valhalla-dev related sessions. FWIW - In London we're really only going to >> involve the couple of people who are already on this list and understand >> the fundamentals of the proposal. >> >> Cheers, >> Martijn >> >> On 7 January 2015 at 20:21, Ivan St. Ivanov >> wrote: >> >>> Hi folks, >>> >>> We, from the Bulgarian JUG are also planning to do a hackathon on this. >>> Most probably it will be on January 29th (just before FOSDEM). We'll >> update >>> the VMs we already have and will also, as usual, write some blogs. >>> >>> As I am taking care of presenting this to my fellows here in Bulgaria and >>> as I am poorly prepared in the language and type theory, would you please >>> tell me where can I read more what is a compiler fiction, specialization >> vs >>> erasure vs reification and all the things that are discussed in the >> various >>> thread an in the state of specialization paper? Sorry for the stupid >>> question, but I really want to get my folks here (including myself) up to >>> speed. >>> >>> Thanks, >>> Ivan >>> >>> On Tue, Jan 6, 2015 at 11:35 AM, Martijn Verburg < >> martijnverburg at gmail.com >>>> wrote: >>> >>>> Hi all, >>>> >>>> On 6 January 2015 at 00:19, Richard Warburton < >>>> richard.warburton at gmail.com> >>>> wrote: >>>> >>>>> Hi Brian, >>>>> >>>>> It should cover all the conversions outlined in the latest SotS. >> There >>>> are >>>>>> a few known things that don't work quite right yet, things like: >>>>>> >>>>>> - Nonstatic inner classes and local classes (we're close on this, >>>>> though) >>>>>> - super-calls inside generic methods >>>>>> >>>>>> But, it should be enough to write an ArrayList-like class that is >>>> generic >>>>>> in any-T. >>>>>> >>>>> >>>>> Thanks. >>>>> >>>>> To run it, just clone and build the valhalla repo (make images), and >> use >>>>>> that as your JDK. >>>>>> >>>>> >>>>> Unless I really need sleep (it is after midnight here) there are no >>>> links >>>>> from the project home page ( >> http://openjdk.java.net/projects/valhalla/) >>>> to >>>>> the source repositories (http://hg.openjdk.java.net/valhalla). I >>>> imagine >>>>> this might cause confusion for casual users who don't know the mapping >>>>> between project names and repo urls and it would be nice to add it. Or >>>> even >>>>> link to http://hg.openjdk.java.net/valhalla/valhalla - since that's >>>> what >>>>> they'll need to hg clone. >>>>> >>>> >>>> To this point I'd like to submit a 'patch' to the website page - is >> there >>>> a >>>> source file for the web page somewhere that I can create a patch >> against? >>>> >>>> Cheers, >>>> Martijn >>>> >>>> >>>>> >>>>> regards, >>>>> >>>>> Richard Warburton >>>>> >>>>> http://insightfullogic.com >>>>> @RichardWarburto >>>>> >>>> >>> >>> >> > > > From sven.reimers at gmail.com Wed Jan 7 21:13:34 2015 From: sven.reimers at gmail.com (Sven Reimers) Date: Wed, 7 Jan 2015 22:13:34 +0100 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54ADA06B.5080800@oracle.com> References: <54AAE765.6040402@oracle.com> <54AB25E3.1060308@oracle.com> <54ADA06B.5080800@oracle.com> Message-ID: That sounds good... now I am really feeling at home... ;-) -Sven On Wed, Jan 7, 2015 at 10:08 PM, Brian Goetz wrote: > It's way more raw than that! More like, are ready to work around "oh, it > seems they have not yet implemented XXX, I'll try something else". > > > On 1/7/2015 4:07 PM, Sven Reimers wrote: > >> .. and are ready to use the compiler on the command line I assume... >> >> ;-) >> >> Sven >> >> On Wed, Jan 7, 2015 at 10:04 PM, Martijn Verburg < >> martijnverburg at gmail.com> >> wrote: >> >> Hi Ivan, >>> >>> To be honest, it's probably a bit early for your group to jump in for >>> valhalla-dev related sessions. FWIW - In London we're really only going >>> to >>> involve the couple of people who are already on this list and understand >>> the fundamentals of the proposal. >>> >>> Cheers, >>> Martijn >>> >>> On 7 January 2015 at 20:21, Ivan St. Ivanov >>> wrote: >>> >>> Hi folks, >>>> >>>> We, from the Bulgarian JUG are also planning to do a hackathon on this. >>>> Most probably it will be on January 29th (just before FOSDEM). We'll >>>> >>> update >>> >>>> the VMs we already have and will also, as usual, write some blogs. >>>> >>>> As I am taking care of presenting this to my fellows here in Bulgaria >>>> and >>>> as I am poorly prepared in the language and type theory, would you >>>> please >>>> tell me where can I read more what is a compiler fiction, specialization >>>> >>> vs >>> >>>> erasure vs reification and all the things that are discussed in the >>>> >>> various >>> >>>> thread an in the state of specialization paper? Sorry for the stupid >>>> question, but I really want to get my folks here (including myself) up >>>> to >>>> speed. >>>> >>>> Thanks, >>>> Ivan >>>> >>>> On Tue, Jan 6, 2015 at 11:35 AM, Martijn Verburg < >>>> >>> martijnverburg at gmail.com >>> >>>> wrote: >>>>> >>>> >>>> Hi all, >>>>> >>>>> On 6 January 2015 at 00:19, Richard Warburton < >>>>> richard.warburton at gmail.com> >>>>> wrote: >>>>> >>>>> Hi Brian, >>>>>> >>>>>> It should cover all the conversions outlined in the latest SotS. >>>>>> >>>>> There >>> >>>> are >>>>> >>>>>> a few known things that don't work quite right yet, things like: >>>>>>> >>>>>>> - Nonstatic inner classes and local classes (we're close on this, >>>>>>> >>>>>> though) >>>>>> >>>>>>> - super-calls inside generic methods >>>>>>> >>>>>>> But, it should be enough to write an ArrayList-like class that is >>>>>>> >>>>>> generic >>>>> >>>>>> in any-T. >>>>>>> >>>>>>> >>>>>> Thanks. >>>>>> >>>>>> To run it, just clone and build the valhalla repo (make images), and >>>>>> >>>>> use >>> >>>> that as your JDK. >>>>>>> >>>>>>> >>>>>> Unless I really need sleep (it is after midnight here) there are no >>>>>> >>>>> links >>>>> >>>>>> from the project home page ( >>>>>> >>>>> http://openjdk.java.net/projects/valhalla/) >>> >>>> to >>>>> >>>>>> the source repositories (http://hg.openjdk.java.net/valhalla). I >>>>>> >>>>> imagine >>>>> >>>>>> this might cause confusion for casual users who don't know the mapping >>>>>> between project names and repo urls and it would be nice to add it. Or >>>>>> >>>>> even >>>>> >>>>>> link to http://hg.openjdk.java.net/valhalla/valhalla - since that's >>>>>> >>>>> what >>>>> >>>>>> they'll need to hg clone. >>>>>> >>>>>> >>>>> To this point I'd like to submit a 'patch' to the website page - is >>>>> >>>> there >>> >>>> a >>>>> source file for the web page somewhere that I can create a patch >>>>> >>>> against? >>> >>>> >>>>> Cheers, >>>>> Martijn >>>>> >>>>> >>>>> >>>>>> regards, >>>>>> >>>>>> Richard Warburton >>>>>> >>>>>> http://insightfullogic.com >>>>>> @RichardWarburto >>>>>> >>>>>> >>>>> >>>> >>>> >>> >> >> >> -- Sven Reimers * Senior Expert Software Architect * NetBeans Dream Team Member: http://dreamteam.netbeans.org * Community Leader NetBeans: http://community.java.net/netbeans Desktop Java: http://community.java.net/javadesktop * JUG Leader JUG Bodensee: http://www.jug-bodensee.de * Duke's Choice Award Winner 2009 * Blog: https://www.java.net//blog/sven * XING: https://www.xing.com/profile/Sven_Reimers8 * LinkedIn: http://www.linkedin.com/in/svenreimers Join the NetBeans Groups: * XING: http://www.xing.com/group-20148.82db20 * NUGM: http://haug-server.dyndns.org/display/NUGM/Home * LinkedIn: http://www.linkedin.com/groups?gid=1860468 http://www.linkedin.com/groups?gid=107402 http://www.linkedin.com/groups?gid=1684717 * Oracle: https://mix.oracle.com/groups/18497 From mikeb01 at gmail.com Wed Jan 7 21:19:04 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Thu, 8 Jan 2015 10:19:04 +1300 Subject: Casting reference array to any-T array. Message-ID: Hi, A small thing that tripped me up yesterday. The following code: package github.mikeb01; public class C { T[] values; public C() { values = (T[]) new Object[0]; } public static void main(String[] args) { new C(); } } Compiles without error, but fails a runtime with: Specializing github.mikeb01.C${0=I}; searching for github/mikeb01/C.class (not found) Specializing github.mikeb01.C${0=I}; searching for github/mikeb01/C.class (found) Exception in thread "main" java.lang.IllegalStateException: [I at valhalla.specializer.SignatureSpecializer.asClassName(SignatureSpecializer.java:79) at valhalla.specializer.SignatureSpecializer$ForType.getDescAsClassName(SignatureSpecializer.java:156) Should that be a compile time failure, with an invalid cast from a reference array to any-T array? Mike. From brian.goetz at oracle.com Wed Jan 7 21:24:02 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jan 2015 16:24:02 -0500 Subject: Casting reference array to any-T array. In-Reply-To: References: Message-ID: <54ADA3F2.6050102@oracle.com> Yes, this is definitely unsound (and probably we should help by failing at compile time rather than runtime.) To fix your code, use the "new T[n]" syntax that gives you an array of the erasure of T for ref T and primitive arrays for primitive T (and an unchecked warning.) On 1/7/2015 4:19 PM, Michael Barker wrote: > Hi, > > A small thing that tripped me up yesterday. The following code: > > package github.mikeb01; > > public class C > { > T[] values; > > public C() > { > values = (T[]) new Object[0]; > } > > public static void main(String[] args) > { > new C(); > } > } > > Compiles without error, but fails a runtime with: > > Specializing github.mikeb01.C${0=I}; searching for github/mikeb01/C.class > (not found) > Specializing github.mikeb01.C${0=I}; searching for github/mikeb01/C.class > (found) > Exception in thread "main" java.lang.IllegalStateException: [I > at > valhalla.specializer.SignatureSpecializer.asClassName(SignatureSpecializer.java:79) > at > valhalla.specializer.SignatureSpecializer$ForType.getDescAsClassName(SignatureSpecializer.java:156) > > Should that be a compile time failure, with an invalid cast from a > reference array to any-T array? > > Mike. > From ivan.st.ivanov at gmail.com Wed Jan 7 21:25:08 2015 From: ivan.st.ivanov at gmail.com (Ivan St. Ivanov) Date: Wed, 7 Jan 2015 23:25:08 +0200 Subject: Can we also get some feedback on specialization, please? In-Reply-To: References: <54AAE765.6040402@oracle.com> <54AB25E3.1060308@oracle.com> <54ADA06B.5080800@oracle.com> Message-ID: Hi folks, I was really expecting writing some client class (in Sublime) that uses the new functionality, i.e. List or MyClass and compiling that against an image which is first cloned from the repo that Richard shared with us, then configured and built. I am going to try that myself first. And this is what I thought Brian had in mind with: So my question is: has anyone tried to write a program with specialized generics with the valhalla implementation? Well, we'd really appreciate it. And I see that Sven has already tried that and discussed some [usage] issues already in another thread. We will not hate, neither flame. We just want to help. The same way that LJC do. :) So, again to my question, is there any source, where I can teach myself about those fictions, specializations, etc. :) Cheers, Ivan On Wed, Jan 7, 2015 at 11:13 PM, Sven Reimers wrote: > That sounds good... now I am really feeling at home... > > ;-) > > -Sven > > On Wed, Jan 7, 2015 at 10:08 PM, Brian Goetz > wrote: > > > It's way more raw than that! More like, are ready to work around "oh, it > > seems they have not yet implemented XXX, I'll try something else". > > > > > > On 1/7/2015 4:07 PM, Sven Reimers wrote: > > > >> .. and are ready to use the compiler on the command line I assume... > >> > >> ;-) > >> > >> Sven > >> > >> On Wed, Jan 7, 2015 at 10:04 PM, Martijn Verburg < > >> martijnverburg at gmail.com> > >> wrote: > >> > >> Hi Ivan, > >>> > >>> To be honest, it's probably a bit early for your group to jump in for > >>> valhalla-dev related sessions. FWIW - In London we're really only going > >>> to > >>> involve the couple of people who are already on this list and > understand > >>> the fundamentals of the proposal. > >>> > >>> Cheers, > >>> Martijn > >>> > >>> On 7 January 2015 at 20:21, Ivan St. Ivanov > >>> wrote: > >>> > >>> Hi folks, > >>>> > >>>> We, from the Bulgarian JUG are also planning to do a hackathon on > this. > >>>> Most probably it will be on January 29th (just before FOSDEM). We'll > >>>> > >>> update > >>> > >>>> the VMs we already have and will also, as usual, write some blogs. > >>>> > >>>> As I am taking care of presenting this to my fellows here in Bulgaria > >>>> and > >>>> as I am poorly prepared in the language and type theory, would you > >>>> please > >>>> tell me where can I read more what is a compiler fiction, > specialization > >>>> > >>> vs > >>> > >>>> erasure vs reification and all the things that are discussed in the > >>>> > >>> various > >>> > >>>> thread an in the state of specialization paper? Sorry for the stupid > >>>> question, but I really want to get my folks here (including myself) up > >>>> to > >>>> speed. > >>>> > >>>> Thanks, > >>>> Ivan > >>>> > >>>> On Tue, Jan 6, 2015 at 11:35 AM, Martijn Verburg < > >>>> > >>> martijnverburg at gmail.com > >>> > >>>> wrote: > >>>>> > >>>> > >>>> Hi all, > >>>>> > >>>>> On 6 January 2015 at 00:19, Richard Warburton < > >>>>> richard.warburton at gmail.com> > >>>>> wrote: > >>>>> > >>>>> Hi Brian, > >>>>>> > >>>>>> It should cover all the conversions outlined in the latest SotS. > >>>>>> > >>>>> There > >>> > >>>> are > >>>>> > >>>>>> a few known things that don't work quite right yet, things like: > >>>>>>> > >>>>>>> - Nonstatic inner classes and local classes (we're close on this, > >>>>>>> > >>>>>> though) > >>>>>> > >>>>>>> - super-calls inside generic methods > >>>>>>> > >>>>>>> But, it should be enough to write an ArrayList-like class that is > >>>>>>> > >>>>>> generic > >>>>> > >>>>>> in any-T. > >>>>>>> > >>>>>>> > >>>>>> Thanks. > >>>>>> > >>>>>> To run it, just clone and build the valhalla repo (make images), and > >>>>>> > >>>>> use > >>> > >>>> that as your JDK. > >>>>>>> > >>>>>>> > >>>>>> Unless I really need sleep (it is after midnight here) there are no > >>>>>> > >>>>> links > >>>>> > >>>>>> from the project home page ( > >>>>>> > >>>>> http://openjdk.java.net/projects/valhalla/) > >>> > >>>> to > >>>>> > >>>>>> the source repositories (http://hg.openjdk.java.net/valhalla). I > >>>>>> > >>>>> imagine > >>>>> > >>>>>> this might cause confusion for casual users who don't know the > mapping > >>>>>> between project names and repo urls and it would be nice to add it. > Or > >>>>>> > >>>>> even > >>>>> > >>>>>> link to http://hg.openjdk.java.net/valhalla/valhalla - since that's > >>>>>> > >>>>> what > >>>>> > >>>>>> they'll need to hg clone. > >>>>>> > >>>>>> > >>>>> To this point I'd like to submit a 'patch' to the website page - is > >>>>> > >>>> there > >>> > >>>> a > >>>>> source file for the web page somewhere that I can create a patch > >>>>> > >>>> against? > >>> > >>>> > >>>>> Cheers, > >>>>> Martijn > >>>>> > >>>>> > >>>>> > >>>>>> regards, > >>>>>> > >>>>>> Richard Warburton > >>>>>> > >>>>>> http://insightfullogic.com > >>>>>> @RichardWarburto > >>>>>> > >>>>>> > >>>>> > >>>> > >>>> > >>> > >> > >> > >> > > > -- > Sven Reimers > > * Senior Expert Software Architect > * NetBeans Dream Team Member: http://dreamteam.netbeans.org > * Community Leader NetBeans: http://community.java.net/netbeans > Desktop Java: > http://community.java.net/javadesktop > * JUG Leader JUG Bodensee: http://www.jug-bodensee.de > * Duke's Choice Award Winner 2009 > * Blog: https://www.java.net//blog/sven > > * XING: https://www.xing.com/profile/Sven_Reimers8 > * LinkedIn: http://www.linkedin.com/in/svenreimers > > Join the NetBeans Groups: > * XING: http://www.xing.com/group-20148.82db20 > * NUGM: http://haug-server.dyndns.org/display/NUGM/Home > * LinkedIn: http://www.linkedin.com/groups?gid=1860468 > http://www.linkedin.com/groups?gid=107402 > http://www.linkedin.com/groups?gid=1684717 > * Oracle: https://mix.oracle.com/groups/18497 > From vitalyd at gmail.com Wed Jan 7 21:28:27 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 16:28:27 -0500 Subject: Casting reference array to any-T array. In-Reply-To: <54ADA3F2.6050102@oracle.com> References: <54ADA3F2.6050102@oracle.com> Message-ID: Timely response - I was just writing to confirm that "new T[]" construct will be supported. As a side note, is the plan to continue erasing T for ref types or will the specializer ultimately create a non-erased T array for them? On Wed, Jan 7, 2015 at 4:24 PM, Brian Goetz wrote: > Yes, this is definitely unsound (and probably we should help by failing at > compile time rather than runtime.) > > To fix your code, use the "new T[n]" syntax that gives you an array of the > erasure of T for ref T and primitive arrays for primitive T (and an > unchecked warning.) > > > On 1/7/2015 4:19 PM, Michael Barker wrote: > >> Hi, >> >> A small thing that tripped me up yesterday. The following code: >> >> package github.mikeb01; >> >> public class C >> { >> T[] values; >> >> public C() >> { >> values = (T[]) new Object[0]; >> } >> >> public static void main(String[] args) >> { >> new C(); >> } >> } >> >> Compiles without error, but fails a runtime with: >> >> Specializing github.mikeb01.C${0=I}; searching for github/mikeb01/C.class >> (not found) >> Specializing github.mikeb01.C${0=I}; searching for github/mikeb01/C.class >> (found) >> Exception in thread "main" java.lang.IllegalStateException: [I >> at >> valhalla.specializer.SignatureSpecializer.asClassName( >> SignatureSpecializer.java:79) >> at >> valhalla.specializer.SignatureSpecializer$ForType.getDescAsClassName( >> SignatureSpecializer.java:156) >> >> Should that be a compile time failure, with an invalid cast from a >> reference array to any-T array? >> >> Mike. >> >> From mikeb01 at gmail.com Wed Jan 7 21:28:29 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Thu, 8 Jan 2015 10:28:29 +1300 Subject: Casting reference array to any-T array. In-Reply-To: <54ADA3F2.6050102@oracle.com> References: <54ADA3F2.6050102@oracle.com> Message-ID: > > Yes, this is definitely unsound (and probably we should help by failing at > compile time rather than runtime.) > > To fix your code, use the "new T[n]" syntax that gives you an array of the > erasure of T for ref T and primitive arrays for primitive T (and an > unchecked warning.) Yep, I eventually figured out what I was missing. Just wanted to get it added to feedback on specialization. Mike. From martijnverburg at gmail.com Wed Jan 7 21:34:27 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Wed, 7 Jan 2015 21:34:27 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: References: <54AAE765.6040402@oracle.com> <54AB25E3.1060308@oracle.com> <54ADA06B.5080800@oracle.com> Message-ID: Hi Ivan, You'll want to start at: http://openjdk.java.net/projects/valhalla/ and read/watch the four presentations there :-). Cheers, Martijn On 7 January 2015 at 21:25, Ivan St. Ivanov wrote: > Hi folks, > > I was really expecting writing some client class (in Sublime) that uses the > new functionality, i.e. List or MyClass and compiling that > against an image which is first cloned from the repo that Richard shared > with us, then configured and built. I am going to try that myself first. > And this is what I thought Brian had in mind with: > > So my question is: has anyone tried to write a program with specialized > generics with the valhalla implementation? Well, we'd really appreciate > it. > > And I see that Sven has already tried that and discussed some [usage] > issues already in another thread. We will not hate, neither flame. We just > want to help. The same way that LJC do. :) > > So, again to my question, is there any source, where I can teach myself > about those fictions, specializations, etc. :) > > Cheers, > Ivan > > > On Wed, Jan 7, 2015 at 11:13 PM, Sven Reimers > wrote: > > > That sounds good... now I am really feeling at home... > > > > ;-) > > > > -Sven > > > > On Wed, Jan 7, 2015 at 10:08 PM, Brian Goetz > > wrote: > > > > > It's way more raw than that! More like, are ready to work around "oh, > it > > > seems they have not yet implemented XXX, I'll try something else". > > > > > > > > > On 1/7/2015 4:07 PM, Sven Reimers wrote: > > > > > >> .. and are ready to use the compiler on the command line I assume... > > >> > > >> ;-) > > >> > > >> Sven > > >> > > >> On Wed, Jan 7, 2015 at 10:04 PM, Martijn Verburg < > > >> martijnverburg at gmail.com> > > >> wrote: > > >> > > >> Hi Ivan, > > >>> > > >>> To be honest, it's probably a bit early for your group to jump in for > > >>> valhalla-dev related sessions. FWIW - In London we're really only > going > > >>> to > > >>> involve the couple of people who are already on this list and > > understand > > >>> the fundamentals of the proposal. > > >>> > > >>> Cheers, > > >>> Martijn > > >>> > > >>> On 7 January 2015 at 20:21, Ivan St. Ivanov < > ivan.st.ivanov at gmail.com> > > >>> wrote: > > >>> > > >>> Hi folks, > > >>>> > > >>>> We, from the Bulgarian JUG are also planning to do a hackathon on > > this. > > >>>> Most probably it will be on January 29th (just before FOSDEM). We'll > > >>>> > > >>> update > > >>> > > >>>> the VMs we already have and will also, as usual, write some blogs. > > >>>> > > >>>> As I am taking care of presenting this to my fellows here in > Bulgaria > > >>>> and > > >>>> as I am poorly prepared in the language and type theory, would you > > >>>> please > > >>>> tell me where can I read more what is a compiler fiction, > > specialization > > >>>> > > >>> vs > > >>> > > >>>> erasure vs reification and all the things that are discussed in the > > >>>> > > >>> various > > >>> > > >>>> thread an in the state of specialization paper? Sorry for the stupid > > >>>> question, but I really want to get my folks here (including myself) > up > > >>>> to > > >>>> speed. > > >>>> > > >>>> Thanks, > > >>>> Ivan > > >>>> > > >>>> On Tue, Jan 6, 2015 at 11:35 AM, Martijn Verburg < > > >>>> > > >>> martijnverburg at gmail.com > > >>> > > >>>> wrote: > > >>>>> > > >>>> > > >>>> Hi all, > > >>>>> > > >>>>> On 6 January 2015 at 00:19, Richard Warburton < > > >>>>> richard.warburton at gmail.com> > > >>>>> wrote: > > >>>>> > > >>>>> Hi Brian, > > >>>>>> > > >>>>>> It should cover all the conversions outlined in the latest SotS. > > >>>>>> > > >>>>> There > > >>> > > >>>> are > > >>>>> > > >>>>>> a few known things that don't work quite right yet, things like: > > >>>>>>> > > >>>>>>> - Nonstatic inner classes and local classes (we're close on > this, > > >>>>>>> > > >>>>>> though) > > >>>>>> > > >>>>>>> - super-calls inside generic methods > > >>>>>>> > > >>>>>>> But, it should be enough to write an ArrayList-like class that is > > >>>>>>> > > >>>>>> generic > > >>>>> > > >>>>>> in any-T. > > >>>>>>> > > >>>>>>> > > >>>>>> Thanks. > > >>>>>> > > >>>>>> To run it, just clone and build the valhalla repo (make images), > and > > >>>>>> > > >>>>> use > > >>> > > >>>> that as your JDK. > > >>>>>>> > > >>>>>>> > > >>>>>> Unless I really need sleep (it is after midnight here) there are > no > > >>>>>> > > >>>>> links > > >>>>> > > >>>>>> from the project home page ( > > >>>>>> > > >>>>> http://openjdk.java.net/projects/valhalla/) > > >>> > > >>>> to > > >>>>> > > >>>>>> the source repositories (http://hg.openjdk.java.net/valhalla). I > > >>>>>> > > >>>>> imagine > > >>>>> > > >>>>>> this might cause confusion for casual users who don't know the > > mapping > > >>>>>> between project names and repo urls and it would be nice to add > it. > > Or > > >>>>>> > > >>>>> even > > >>>>> > > >>>>>> link to http://hg.openjdk.java.net/valhalla/valhalla - since > that's > > >>>>>> > > >>>>> what > > >>>>> > > >>>>>> they'll need to hg clone. > > >>>>>> > > >>>>>> > > >>>>> To this point I'd like to submit a 'patch' to the website page - is > > >>>>> > > >>>> there > > >>> > > >>>> a > > >>>>> source file for the web page somewhere that I can create a patch > > >>>>> > > >>>> against? > > >>> > > >>>> > > >>>>> Cheers, > > >>>>> Martijn > > >>>>> > > >>>>> > > >>>>> > > >>>>>> regards, > > >>>>>> > > >>>>>> Richard Warburton > > >>>>>> > > >>>>>> http://insightfullogic.com > > >>>>>> @RichardWarburto > > >>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>>> > > >>> > > >> > > >> > > >> > > > > > > -- > > Sven Reimers > > > > * Senior Expert Software Architect > > * NetBeans Dream Team Member: http://dreamteam.netbeans.org > > * Community Leader NetBeans: http://community.java.net/netbeans > > Desktop Java: > > http://community.java.net/javadesktop > > * JUG Leader JUG Bodensee: http://www.jug-bodensee.de > > * Duke's Choice Award Winner 2009 > > * Blog: https://www.java.net//blog/sven > > > > * XING: https://www.xing.com/profile/Sven_Reimers8 > > * LinkedIn: http://www.linkedin.com/in/svenreimers > > > > Join the NetBeans Groups: > > * XING: http://www.xing.com/group-20148.82db20 > > * NUGM: http://haug-server.dyndns.org/display/NUGM/Home > > * LinkedIn: http://www.linkedin.com/groups?gid=1860468 > > http://www.linkedin.com/groups?gid=107402 > > http://www.linkedin.com/groups?gid=1684717 > > * Oracle: https://mix.oracle.com/groups/18497 > > > From brian.goetz at oracle.com Wed Jan 7 21:48:26 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jan 2015 16:48:26 -0500 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> Message-ID: <54ADA9AA.6080802@oracle.com> > Timely response - I was just writing to confirm that "new T[]" construct > will be supported. Tentatively so. We still have to figure out how to generate the right set of warnings, and the right conditions for suppressing those warnings, since we don't want to undermine the invariant that "if a program compiles with no (suppressed or not) raw or unchecked warnings, then the casts inserted by the compiler will not fail." As an alternative to doing it in the language, we will also consider instead providing pre-peeled library methods like: T[] newArray(int) > As a side note, is the plan to continue erasing T > for ref types or will the specializer ultimately create a non-erased T > array for them? The current functionality continues with the erasure plan. However, I wouldn't mind doing better! From vitalyd at gmail.com Wed Jan 7 21:58:04 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 16:58:04 -0500 Subject: Casting reference array to any-T array. In-Reply-To: <54ADA9AA.6080802@oracle.com> References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: > > Tentatively so. We still have to figure out how to generate the right set > of warnings, and the right conditions for suppressing those warnings, since > we don't want to undermine the invariant that "if a program compiles with > no (suppressed or not) raw or unchecked warnings, then the casts inserted > by the compiler will not fail." > As an alternative to doing it in the language, we will also consider > instead providing pre-peeled library methods like: > T[] newArray(int) Ideally, "new T[]" would be made work as it's cleaner/"purer", although a lib method isn't too big of a deal. > The current functionality continues with the erasure plan. However, I > wouldn't mind doing better! Yeah, I can't immediately think of a critical reason why it can't stay erased. For JIT optimizer, having a narrower upper bound on the type may make its life easier, although I don't know if it'll have any material difference. The one question is what reflection will do (and any code based on reflection, such as custom serialization, code generation, etc): T[] getArray() { return new T[]; } Object someArray = getArray(); someArray.getClass().getComponentType() == int.class someArray = getArray(); someArray.getClass().getComponentType() == MyValueType.class someArray = getArray(); // Foo is a class or interface, doesn't matter someArray.getClass().getComponentType() == Object.class The asymmetry would be unfortunate, I think. On Wed, Jan 7, 2015 at 4:48 PM, Brian Goetz wrote: > Timely response - I was just writing to confirm that "new T[]" construct >> will be supported. >> > > Tentatively so. We still have to figure out how to generate the right set > of warnings, and the right conditions for suppressing those warnings, since > we don't want to undermine the invariant that "if a program compiles with > no (suppressed or not) raw or unchecked warnings, then the casts inserted > by the compiler will not fail." > > As an alternative to doing it in the language, we will also consider > instead providing pre-peeled library methods like: > > T[] newArray(int) > > As a side note, is the plan to continue erasing T >> for ref types or will the specializer ultimately create a non-erased T >> array for them? >> > > The current functionality continues with the erasure plan. However, I > wouldn't mind doing better! > From mikeb01 at gmail.com Wed Jan 7 22:09:39 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Thu, 8 Jan 2015 11:09:39 +1300 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: > > The current functionality continues with the erasure plan. However, I >> wouldn't mind doing better! > > > Yeah, I can't immediately think of a critical reason why it can't stay > erased. For JIT optimizer, having a narrower upper bound on the type may > make its life easier, although I don't know if it'll have any material > difference. The one question is what reflection will do (and any code > based on reflection, such as custom serialization, code generation, etc): > (Caveat, I'm not a compiler expert so this is a bit of a guess.) One possible place where this could be used with within the optimiser. E.g. if Hotspot could see a specialised HashMap instead of an erased one, then it could determine that calls to hashCode and equals would be mono-morphic and apply more aggressive in-lining. This could lead to jump in performance across a broad ranges of apps (hands up who uses Strings and HashMaps :-). My understand is that the mega-morhpic dispatch (of hashCode/equals) is one of the more significant costs within HashMap. If that was possible then it would be pretty cool! Mike. From vitalyd at gmail.com Wed Jan 7 22:12:26 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 17:12:26 -0500 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: Right, but the reason I'm doubtful that this will have any impact is because the JIT already does type profiling, and the runtime types it sees (and the statistics around that) won't change due to erasure. My "make its life easier" comment was a guess that perhaps some code paths in the optimizer don't need to be taken (e.g. don't look at profiling info if it now knows statically that an array is composed of final classes). On Wed, Jan 7, 2015 at 5:09 PM, Michael Barker wrote: > The current functionality continues with the erasure plan. However, I >>> wouldn't mind doing better! >> >> >> Yeah, I can't immediately think of a critical reason why it can't stay >> erased. For JIT optimizer, having a narrower upper bound on the type may >> make its life easier, although I don't know if it'll have any material >> difference. The one question is what reflection will do (and any code >> based on reflection, such as custom serialization, code generation, etc): >> > > (Caveat, I'm not a compiler expert so this is a bit of a guess.) > > One possible place where this could be used with within the optimiser. > E.g. if Hotspot could see a specialised HashMap instead of > an erased one, then it could determine that calls to hashCode and equals > would be mono-morphic and apply more aggressive in-lining. This could lead > to jump in performance across a broad ranges of apps (hands up who uses > Strings and HashMaps :-). My understand is that the mega-morhpic dispatch > (of hashCode/equals) is one of the more significant costs within HashMap. > > If that was possible then it would be pretty cool! > > Mike. > From mikeb01 at gmail.com Wed Jan 7 22:23:00 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Thu, 8 Jan 2015 11:23:00 +1300 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: My understand is that it does type profiling at the callsite and something like HashMap.hash() will encounter such wide variety of classes that it will rarely be anything other than fully mega-morphic. My guess was that if there was specialised class for a specific reference type then this could become mono-morphic. On 8 January 2015 at 11:12, Vitaly Davidovich wrote: > Right, but the reason I'm doubtful that this will have any impact is > because the JIT already does type profiling, and the runtime types it sees > (and the statistics around that) won't change due to erasure. My "make its > life easier" comment was a guess that perhaps some code paths in the > optimizer don't need to be taken (e.g. don't look at profiling info if it > now knows statically that an array is composed of final classes). > > On Wed, Jan 7, 2015 at 5:09 PM, Michael Barker wrote: > >> The current functionality continues with the erasure plan. However, I >>>> wouldn't mind doing better! >>> >>> >>> Yeah, I can't immediately think of a critical reason why it can't stay >>> erased. For JIT optimizer, having a narrower upper bound on the type may >>> make its life easier, although I don't know if it'll have any material >>> difference. The one question is what reflection will do (and any code >>> based on reflection, such as custom serialization, code generation, etc): >>> >> >> (Caveat, I'm not a compiler expert so this is a bit of a guess.) >> >> One possible place where this could be used with within the optimiser. >> E.g. if Hotspot could see a specialised HashMap instead of >> an erased one, then it could determine that calls to hashCode and equals >> would be mono-morphic and apply more aggressive in-lining. This could lead >> to jump in performance across a broad ranges of apps (hands up who uses >> Strings and HashMaps :-). My understand is that the mega-morhpic dispatch >> (of hashCode/equals) is one of the more significant costs within HashMap. >> >> If that was possible then it would be pretty cool! >> >> Mike. >> > > From vitalyd at gmail.com Wed Jan 7 22:32:58 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 17:32:58 -0500 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: Ah, you're talking about specialized classes as a whole (I was referring to just the arrays aspect). Yes, if it were to specialize every single type, then you'd get better type information. Downside is you now explode the number of method definitions in the runtime. In .NET, for example, generic methods are not specialized for reference types, in part for this reason I believe. Generally speaking, the downside to creating distinct structures per type is the explosion in the number of types at runtime. I encourage you to read this oldish blog post by Joe Duffy (MSFT engineer): http://joeduffyblog.com/2011/10/23/on-generics-and-some-of-the-associated-overheads/ On Wed, Jan 7, 2015 at 5:23 PM, Michael Barker wrote: > My understand is that it does type profiling at the callsite and something > like HashMap.hash() will encounter such wide variety of classes that it > will rarely be anything other than fully mega-morphic. My guess was that > if there was specialised class for a specific reference type then this > could become mono-morphic. > > On 8 January 2015 at 11:12, Vitaly Davidovich wrote: > >> Right, but the reason I'm doubtful that this will have any impact is >> because the JIT already does type profiling, and the runtime types it sees >> (and the statistics around that) won't change due to erasure. My "make its >> life easier" comment was a guess that perhaps some code paths in the >> optimizer don't need to be taken (e.g. don't look at profiling info if it >> now knows statically that an array is composed of final classes). >> >> On Wed, Jan 7, 2015 at 5:09 PM, Michael Barker wrote: >> >>> The current functionality continues with the erasure plan. However, I >>>>> wouldn't mind doing better! >>>> >>>> >>>> Yeah, I can't immediately think of a critical reason why it can't stay >>>> erased. For JIT optimizer, having a narrower upper bound on the type may >>>> make its life easier, although I don't know if it'll have any material >>>> difference. The one question is what reflection will do (and any code >>>> based on reflection, such as custom serialization, code generation, etc): >>>> >>> >>> (Caveat, I'm not a compiler expert so this is a bit of a guess.) >>> >>> One possible place where this could be used with within the optimiser. >>> E.g. if Hotspot could see a specialised HashMap instead of >>> an erased one, then it could determine that calls to hashCode and equals >>> would be mono-morphic and apply more aggressive in-lining. This could lead >>> to jump in performance across a broad ranges of apps (hands up who uses >>> Strings and HashMaps :-). My understand is that the mega-morhpic dispatch >>> (of hashCode/equals) is one of the more significant costs within HashMap. >>> >>> If that was possible then it would be pretty cool! >>> >>> Mike. >>> >> >> > From sven.reimers at gmail.com Wed Jan 7 22:33:27 2015 From: sven.reimers at gmail.com (Sven Reimers) Date: Wed, 7 Jan 2015 23:33:27 +0100 Subject: Question regarding inheritance and any Message-ID: Hi, trying some simple inheritance I tried public class Valhalla { public static void main (String[] args) { List intPoints = new ArrayList<>(); System.out.println(intPoints); } public static interface List { } public static class ArrayList implements List { } } If I remove the any from the ArrayList I get the following compile error: incompatible types: cannot infer type arguments for ArrayList<> List intPoints = new ArrayList<>(); ^ reason: no instance(s) of type variable(s) T exist so that ArrayList conforms to List where T is a type-variable: T extends Object declared in class ArrayList 1 error Having given this failure some thought I am now thinking this is intentional so that every implementation of an anyfied super class/interface has to actively opt in to provide support for anyfied types. Is this the intention? Thanks for clarifying and your patience answering all those questions... -Sven -- Sven Reimers * Senior Expert Software Architect * NetBeans Dream Team Member: http://dreamteam.netbeans.org * Community Leader NetBeans: http://community.java.net/netbeans Desktop Java: http://community.java.net/javadesktop * JUG Leader JUG Bodensee: http://www.jug-bodensee.de * Duke's Choice Award Winner 2009 * Blog: https://www.java.net//blog/sven * XING: https://www.xing.com/profile/Sven_Reimers8 * LinkedIn: http://www.linkedin.com/in/svenreimers Join the NetBeans Groups: * XING: http://www.xing.com/group-20148.82db20 * NUGM: http://haug-server.dyndns.org/display/NUGM/Home * LinkedIn: http://www.linkedin.com/groups?gid=1860468 http://www.linkedin.com/groups?gid=107402 http://www.linkedin.com/groups?gid=1684717 * Oracle: https://mix.oracle.com/groups/18497 From richard.warburton at gmail.com Wed Jan 7 22:38:32 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Wed, 7 Jan 2015 22:38:32 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: References: <54AAE765.6040402@oracle.com> <54AB25E3.1060308@oracle.com> Message-ID: Hi Ivan, We, from the Bulgarian JUG are also planning to do a hackathon on this. > Most probably it will be on January 29th (just before FOSDEM). We'll update > the VMs we already have and will also, as usual, write some blogs. > There's probably a pretty steep learning curve, but best of luck with your efforts. I imagine it'll be helpful. As I am taking care of presenting this to my fellows here in Bulgaria and > as I am poorly prepared in the language and type theory, would you please > tell me where can I read more what is a compiler fiction, specialization vs > erasure vs reification and all the things that are discussed in the various > thread an in the state of specialization paper? Sorry for the stupid > question, but I really want to get my folks here (including myself) up to > speed. > There are no stupid questions. Actually that's a lie - but these aren't stupid questions. Afaict in this case the term "Compiler Fiction" is being used to denote a concept which exists at some level of abstraction but not at a lower level. For example you could describe a while loop as a compiler fiction because when you hit bytecode control flow is represented by gotos. Obviously removing some abstractions is a good thing and whilst others tend to leak through to the source of the translation. You'll have to make your own mind up as to whether adding or not adding an Any type is a good idea or a bad one. As to specialization - there are solid links from http://openjdk.java.net/projects/valhalla/. I don't really know much more than reading those topics. Erasure and generics have quite a few general articles which Google is more adequately qualified to index than I am. If you are finding it difficult to understand the documents linked but want to understand them. For example if you don't know what a "subtyping relationship" is then I don't have an easy answer for you. Years ago I bought and read http://www.cis.upenn.edu/~bcpierce/tapl/ which is an excellent book, but not an easy or quick read. I've also forgotten most of it though - so I do have to ask stupid questions from time to time myself ;) It would be good if someone wrote a quick introduction to this kind of thing for professional developers. *I don't think any of that should stop you from running a hackday. Just write some code that looks like it should work and then email the list saying what was confusing or broken.* regards, Richard Warburton http://insightfullogic.com @RichardWarburto From twhitmore.nz at gmail.com Wed Jan 7 23:07:43 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Thu, 8 Jan 2015 12:07:43 +1300 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: References: Message-ID: Hi Simon, Vitaly, people, Lots of people have been talking about adapting Collections for 'any' type. Going forwards, I do not believe we should be writing code that explicitly uses/ or checks for nulls (though we may be backwards-compatible for a while) any more than we should be writing or calling a Duck.woof() method. To answering a few questions: > a) What are the actual semantics? For ref-types: x eq y --> (x != null && x.equals(y)) For value-types: x eq y --> (x == y) > b) How is this different to the approaches already outlines earlier? It provides a basic logical building-block (equality check) at the language level, in a way that works both for primitives/valuetypes and for nullable references. It addresses a very big long-standing pain point in Java application development. It avoids writing/ or encouraging "Duck.woof()" style-hacks -- null checks -- into a domain that does not uniformly support them. > Why do we need a new special operator? What's wrong with using equals () and having the specializer rewrite that for primitives? > What's wrong with using equals() and having the specializer rewrite that for primitives? For other types > (refs and custom value types), it should just delegate to the real equals. To have a proper logical equals in one operator, which is independent of implementation & null-safe for reftypes. What's wrong is writing null-checks & nulls, in a domain where nulls may not exist. We are not dealing with Dog, we are dealing with Animal and there should be no 'woof()' method. > What's wrong with using equals() and having the specializer rewrite that for primitives? For other types > (refs and custom value types), it should just delegate to the real equals. What's wrong with an operator that logically has exactly the right meaning, without exposing details that don't exist in the domain? It also provides a vast benefit for application programmers, as this kind of requirement is ridiculously common in hundreds of millions of lines of application code. Regards, Thomas Whitmore From brian.goetz at oracle.com Wed Jan 7 23:16:29 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jan 2015 18:16:29 -0500 Subject: Question regarding inheritance and any In-Reply-To: References: Message-ID: <54ADBE4D.6000107@oracle.com> Right. First, type hierarchies have to be any-fied from the top down. You can have interface A { } interface B extends A { } and interface A { } interface B extends A { } but not interface A { } interface B extends A { } If you think of this in terms of a type bound (any T == Object|Value), this is the equivalent of "type parameter T is not within its bound." Similarly, at the use site, the compiler will reject new A() if int is not within the "bound" of A's tvar. On 1/7/2015 5:33 PM, Sven Reimers wrote: > Hi, > > trying some simple inheritance I tried > > public class Valhalla { > > public static void main (String[] args) { > > List intPoints = new ArrayList<>(); > System.out.println(intPoints); > > } > > public static interface List { } > > public static class ArrayList implements List { } > > } > > If I remove the any from the ArrayList I get the following compile > error: > > incompatible types: cannot infer type arguments for ArrayList<> > List intPoints = new ArrayList<>(); > ^ > reason: no instance(s) of type variable(s) T exist so that ArrayList > conforms to List > where T is a type-variable: > T extends Object declared in class ArrayList > 1 error > > Having given this failure some thought I am now thinking this is > intentional so that every implementation of an anyfied super > class/interface has to actively opt in to provide support for anyfied types. > > Is this the intention? > > Thanks for clarifying and your patience answering all those questions... > > -Sven > From twhitmore.nz at gmail.com Wed Jan 7 23:21:47 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Thu, 8 Jan 2015 12:21:47 +1300 Subject: valhalla-dev Digest, Vol 7, Issue 26 In-Reply-To: References: Message-ID: Hi Peter, Vitaly, Thanks for your interesting ideas. Vitaly says: > Personally I'd handle primitives as if they were value types from day 1: > they get Object's hashCode, equals, and toString by allowing T.toString and > rewriting it to their corresponding XXX.toString. Sounds great, we need good basic integration of primitives & value types. Let's do it. Peter says: > While experimenting a bit with the current preliminary prototype, I found > myself on several occasions wanting to call a method from "normal" API. Say > System.out.println(x) with an x of type "any T". Usually a method that > already has various overloads or a method taking an Object. Javac refuses: > > PrintStream.println(boolean) > PrintStream.println(char) > PrintStream.println(int) > .. > PrintStream.println(String) > PrintStream.println(Object) This pattern is absolutely typical of existing "vari-typed" Java code & libraries. It's the only efficient way to receive an argument that may be of (logically) any type, and process that argument. I absolutely think this is something which we should target, and the specializer should be able to do. Essentially, this is our specialized code's interface to "the rest of the Java world". It really should be able to be efficient. Equals, hashCode and toString only take us so far.. there are a world of other interactions, which -- if designed for performance -- will be written in a similar manner to these. Regards, Thomas Whitmore From vitalyd at gmail.com Wed Jan 7 23:33:35 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 18:33:35 -0500 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: References: Message-ID: > > For value-types: > x eq y --> (x == y) You mean for primitives? Custom value types won't work with == as there's no operator overloading. Getting back to the null aspect though, here's my take ... I think we should not introduce a new operator such as eq when we already have a "universal" equals. What you really want to say is something like this: void foo(T t) { if (T.default.equals(t)) System.out.println("got default/null"); else System.out.println("got non-default/null"); } Now, the obvious problem here is that this NPEs on reference types since T.default yields a null and you can't reverse the comparison because t may be null. So, what you'd like here is similar to what .NET does. Introduce an IEqualityComparer-like JDK interface, and provide a way to obtain a "default" comparer for a given type. In .NET, there's a generic class called EqualityComparer, which has a static property called Default (e.g. EqualityComparer.Default). That internally (in static constructor) figures out, based on inspecting T's capabilities and type, which is the default equality comparer and then hands out that sole instance on each Default access. Ultimately, what you want here is instead of doing T.default.equals(t), you want some other class to define an " boolean equals(T x, T y)" which can internally handle one side being null (for ref types). tldr; well behaved classes already should be handling null inputs in their equals() impl, so we should find a way to reuse that instead of introducing a new operator/keyword. On Wed, Jan 7, 2015 at 6:07 PM, Thomas W wrote: > Hi Simon, Vitaly, people, > > Lots of people have been talking about adapting Collections for 'any' > type. Going forwards, I do not believe we should be writing code that > explicitly uses/ or checks for nulls (though we may be backwards-compatible > for a while) any more than we should be writing or calling a Duck.woof() > method. > > To answering a few questions: > > > a) What are the actual semantics? > > For ref-types: > x eq y --> (x != null && x.equals(y)) > For value-types: > x eq y --> (x == y) > > > b) How is this different to the approaches already outlines earlier? > > It provides a basic logical building-block (equality check) at the > language level, in a way that works both for primitives/valuetypes and for > nullable references. It addresses a very big long-standing pain point in > Java application development. It avoids writing/ or encouraging > "Duck.woof()" style-hacks -- null checks -- into a domain that does not > uniformly support them. > > > Why do we need a new special operator? What's wrong with using equals > () and having the specializer rewrite that for primitives? > > What's wrong with using equals() and having the specializer rewrite > that for primitives? For other types > > (refs and custom value types), it should just delegate to the real > equals. > > To have a proper logical equals in one operator, which is independent of > implementation & null-safe for reftypes. > > What's wrong is writing null-checks & nulls, in a domain where nulls may > not exist. > We are not dealing with Dog, we are dealing with Animal and there should > be no 'woof()' method. > > > What's wrong with using equals() and having the specializer rewrite > that for primitives? For other types > > (refs and custom value types), it should just delegate to the real > equals. > > What's wrong with an operator that logically has exactly the right > meaning, without exposing details that don't exist in the domain? > It also provides a vast benefit for application programmers, as this kind > of requirement is ridiculously common in hundreds of millions of lines of > application code. > > > Regards, > Thomas Whitmore > From maurizio.cimadamore at oracle.com Thu Jan 8 00:00:05 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 08 Jan 2015 00:00:05 +0000 Subject: Casting reference array to any-T array. In-Reply-To: <54ADA3F2.6050102@oracle.com> References: <54ADA3F2.6050102@oracle.com> Message-ID: <54ADC885.6090902@oracle.com> On 07/01/15 21:24, Brian Goetz wrote: > Yes, this is definitely unsound (and probably we should help by > failing at compile time rather than runtime.) Yep - this is a javac bug; I'll look into this. Maurizio > > To fix your code, use the "new T[n]" syntax that gives you an array of > the erasure of T for ref T and primitive arrays for primitive T (and > an unchecked warning.) > > On 1/7/2015 4:19 PM, Michael Barker wrote: >> Hi, >> >> A small thing that tripped me up yesterday. The following code: >> >> package github.mikeb01; >> >> public class C >> { >> T[] values; >> >> public C() >> { >> values = (T[]) new Object[0]; >> } >> >> public static void main(String[] args) >> { >> new C(); >> } >> } >> >> Compiles without error, but fails a runtime with: >> >> Specializing github.mikeb01.C${0=I}; searching for >> github/mikeb01/C.class >> (not found) >> Specializing github.mikeb01.C${0=I}; searching for >> github/mikeb01/C.class >> (found) >> Exception in thread "main" java.lang.IllegalStateException: [I >> at >> valhalla.specializer.SignatureSpecializer.asClassName(SignatureSpecializer.java:79) >> >> at >> valhalla.specializer.SignatureSpecializer$ForType.getDescAsClassName(SignatureSpecializer.java:156) >> >> >> Should that be a compile time failure, with an invalid cast from a >> reference array to any-T array? >> >> Mike. >> From vitalyd at gmail.com Thu Jan 8 00:06:38 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 19:06:38 -0500 Subject: valhalla-dev Digest, Vol 7, Issue 26 In-Reply-To: References: Message-ID: So say I'm authoring a generic class and call System.out.println() - how do I know which method is going to be called? Suppose println returned different types depending on overload - what's my return value type? I can't target type it because that implies I know what the generic type will be and I don't, I'm just shipping a generic library. Now if System.out.println has a generic overload, I know where I'm calling and what I'm getting back. But there's no specialization here, this is plain generic dispatch no different from today. Sent from my phone On Jan 7, 2015 6:21 PM, "Thomas W" wrote: > Hi Peter, Vitaly, > > Thanks for your interesting ideas. Vitaly says: > > > Personally I'd handle primitives as if they were value types from day 1: > > they get Object's hashCode, equals, and toString by allowing T.toString > and > > rewriting it to their corresponding XXX.toString. > > Sounds great, we need good basic integration of primitives & value types. > Let's do it. > > Peter says: > > > While experimenting a bit with the current preliminary prototype, I found > > myself on several occasions wanting to call a method from "normal" API. > Say > > System.out.println(x) with an x of type "any T". Usually a method that > > already has various overloads or a method taking an Object. Javac > refuses: > > > > PrintStream.println(boolean) > > PrintStream.println(char) > > PrintStream.println(int) > > .. > > PrintStream.println(String) > > PrintStream.println(Object) > > This pattern is absolutely typical of existing "vari-typed" Java code & > libraries. It's the only efficient way to receive an argument that may be > of (logically) any type, and process that argument. > I absolutely think this is something which we should target, and the > specializer should be able to do. > > Essentially, this is our specialized code's interface to "the rest of the > Java world". It really should be able to be efficient. > > Equals, hashCode and toString only take us so far.. there are a world of > other interactions, which -- if designed for performance -- will be written > in a similar manner to these. > > > Regards, > Thomas Whitmore > From twhitmore.nz at gmail.com Thu Jan 8 00:14:30 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Thu, 8 Jan 2015 13:14:30 +1300 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: References: Message-ID: Ok, good points. To clarify: For ref-types: x eq y --> (x == y || (x != null && x.equals(y))) // 'null eq null' should be true. For primitive types: x eq y --> x == y For value-types: x eq y --> x.equals(y) > Getting back to the null aspect though, here's my take ... I'm laughing.. but my point is absolutely, that we should get *away* from it. Nulls do not belong at this abstracted level... Neither does Duck.woof(). > we already have a "universal" equals. We don't have a universal equals operator, that's the problem. We have an equals() method on Object, which requires additional null-checking to use on nullable reftypes; and also imposes an "asymmetrical syntax" vis a vis it's arguments. *There is no universal "equals" operator at the logical level.* Simple as that. > What you really want to say is something like this: > > void foo(T t) { > if (T.default.equals(t)) > System.out.println("got default/null"); > else > System.out.println("got non-default/null"); > } No, actually I don't want to say anything like the above. I want to write clear code at a logical level, which applies at the level of the domain "any-typed values" without ugly hacks. Basically in Java, I want to express simple business requirements, and slightly-more-interesting framework code, in a clear & straightforward way. Let's try a better example: public boolean removeItem (T item) { for (int i = 0; i < size; i++) { if (elements[i] eq item) { fastRemove(i); // found; remove it. return true; } } return false; // not found. } The example you provided was an example of checking for a "sentinel" value -- something I've been (separately from the EQ proposal) trying to clearly recognize. However, I expect such check is always the responsibility of the client code -- Collections or framework should only answer sentinel, never be checking for it. Therefore it is somewhat less likely to be genericized & any-typed. If we did wish to write such a check, however: public void processSomething (String key) { T result = map.get( key); if (result eq T.sentinel) { System.out.println("key presumed not present"); return; } //.. continue processing } > Now, the obvious problem here is that this NPEs on reference types since T.default yields a null and you can't reverse the comparison because t may be null. So, what you'd like here is similar to what .NET does. Introduce an > IEqualityComparer-like JDK interface, and provide a way to obtain a "default" comparer for a given type. In .NET, there's a generic class called EqualityComparer, which has a static property called Default (e.g. > EqualityComparer.Default). That internally (in static constructor) figures out, based on inspecting T's capabilities and type, which is the default equality comparer and then hands out that sole instance on each Default access. > Ultimately, what you want here is instead of doing T.default.equals(t), you want some other class to define an " boolean equals(T x, T y)" which can internally handle one side being null (for ref types). Treating "Equals" as a strategy is an interesting idea. Is it faster than just specializing appropriate bytecode? I'd still like a clean 'eq' operator to call this internally. > Well behaved classes already should be handling null inputs in their equals() impl, so we should find a way to reuse that instead of introducing a new operator/keyword. Well, I noticed we needed an "x == y" check to ensure "null eq null" answered true. There are quite a few wrinkles with Equals comparison.. one of the benefits of bytecode/ monomorphic implementation for reftypes, might be that Hotspot can optimize some of the checks away. Thanks for your comments! Regards, Thomas On Thu, Jan 8, 2015 at 12:33 PM, Vitaly Davidovich wrote: > For value-types: >> x eq y --> (x == y) > > > You mean for primitives? Custom value types won't work with == as there's > no operator overloading. > > Getting back to the null aspect though, here's my take ... > > I think we should not introduce a new operator such as eq when we already > have a "universal" equals. What you really want to say is something like > this: > > void foo(T t) { > if (T.default.equals(t)) > System.out.println("got default/null"); > else > System.out.println("got non-default/null"); > } > > Now, the obvious problem here is that this NPEs on reference types since > T.default yields a null and you can't reverse the comparison because t may > be null. So, what you'd like here is similar to what .NET does. Introduce > an IEqualityComparer-like JDK interface, and provide a way to obtain a > "default" comparer for a given type. In .NET, there's a generic class > called EqualityComparer, which has a static property called Default (e.g. > EqualityComparer.Default). That internally (in static constructor) > figures out, based on inspecting T's capabilities and type, which is the > default equality comparer and then hands out that sole instance on each > Default access. Ultimately, what you want here is instead of doing > T.default.equals(t), you want some other class to define an " > boolean equals(T x, T y)" which can internally handle one side being null > (for ref types). > > tldr; well behaved classes already should be handling null inputs in their > equals() impl, so we should find a way to reuse that instead of introducing > a new operator/keyword. > > > On Wed, Jan 7, 2015 at 6:07 PM, Thomas W wrote: > >> Hi Simon, Vitaly, people, >> >> Lots of people have been talking about adapting Collections for 'any' >> type. Going forwards, I do not believe we should be writing code that >> explicitly uses/ or checks for nulls (though we may be backwards-compatible >> for a while) any more than we should be writing or calling a Duck.woof() >> method. >> >> To answering a few questions: >> >> > a) What are the actual semantics? >> >> For ref-types: >> x eq y --> (x != null && x.equals(y)) >> For value-types: >> x eq y --> (x == y) >> >> > b) How is this different to the approaches already outlines earlier? >> >> It provides a basic logical building-block (equality check) at the >> language level, in a way that works both for primitives/valuetypes and for >> nullable references. It addresses a very big long-standing pain point in >> Java application development. It avoids writing/ or encouraging >> "Duck.woof()" style-hacks -- null checks -- into a domain that does not >> uniformly support them. >> >> > Why do we need a new special operator? What's wrong with using equals >> () and having the specializer rewrite that for primitives? >> > What's wrong with using equals() and having the specializer rewrite >> that for primitives? For other types >> > (refs and custom value types), it should just delegate to the real >> equals. >> >> To have a proper logical equals in one operator, which is independent of >> implementation & null-safe for reftypes. >> >> What's wrong is writing null-checks & nulls, in a domain where nulls may >> not exist. >> We are not dealing with Dog, we are dealing with Animal and there should >> be no 'woof()' method. >> >> > What's wrong with using equals() and having the specializer rewrite >> that for primitives? For other types >> > (refs and custom value types), it should just delegate to the real >> equals. >> >> What's wrong with an operator that logically has exactly the right >> meaning, without exposing details that don't exist in the domain? >> It also provides a vast benefit for application programmers, as this kind >> of requirement is ridiculously common in hundreds of millions of lines of >> application code. >> >> >> Regards, >> Thomas Whitmore >> > > From vitalyd at gmail.com Thu Jan 8 00:23:22 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 19:23:22 -0500 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: References: Message-ID: So i thought having equality externalized means you don't deal with it in the generic class - you call a black box passing a default and target item, and get an answer. Now yes the default for ref types is null but for better or worse null isn't going away in java so pretending it doesn't exist is just silly. As for sentinels (note this is different from default) we discussed this in the other thread in terms of how to deal with them in things like hashmap. Sent from my phone On Jan 7, 2015 7:14 PM, "Thomas W" wrote: > Ok, good points. To clarify: > > For ref-types: > x eq y --> (x == y || (x != null && x.equals(y))) // > 'null eq null' should be true. > For primitive types: > x eq y --> x == y > For value-types: > x eq y --> x.equals(y) > > > Getting back to the null aspect though, here's my take ... > > I'm laughing.. but my point is absolutely, that we should get *away* from > it. Nulls do not belong at this abstracted level... > Neither does Duck.woof(). > > > we already have a "universal" equals. > > We don't have a universal equals operator, that's the problem. We have an > equals() method on Object, which requires additional null-checking to use > on nullable reftypes; and also imposes an "asymmetrical syntax" vis a vis > it's arguments. > > *There is no universal "equals" operator at the logical level.* > Simple as that. > > > What you really want to say is something like this: > > > > void foo(T t) { > > if (T.default.equals(t)) > > System.out.println("got default/null"); > > else > > System.out.println("got non-default/null"); > > } > > No, actually I don't want to say anything like the above. I want to > write clear code at a logical level, which applies at the level of the > domain "any-typed values" without ugly hacks. > Basically in Java, I want to express simple business requirements, and > slightly-more-interesting framework code, in a clear & straightforward way. > > Let's try a better example: > > > public boolean removeItem (T item) { > for (int i = 0; i < size; i++) { > if (elements[i] eq item) { > fastRemove(i); // found; remove it. > return true; > } > } > return false; // not found. > } > > The example you provided was an example of checking for a "sentinel" value > -- something I've been (separately from the EQ proposal) trying to clearly > recognize. > However, I expect such check is always the responsibility of the client > code -- Collections or framework should only answer sentinel, never be > checking for it. Therefore it is somewhat less likely to be genericized & > any-typed. > > If we did wish to write such a check, however: > > > public void processSomething (String key) { > T result = map.get( key); > if (result eq T.sentinel) { > System.out.println("key presumed not present"); > return; > } > //.. continue processing > } > > > Now, the obvious problem here is that this NPEs on reference types since > T.default yields a null and you can't reverse the comparison because t may > be null. So, what you'd like here is similar to what .NET does. Introduce > an > > IEqualityComparer-like JDK interface, and provide a way to obtain a > "default" comparer for a given type. In .NET, there's a generic class > called EqualityComparer, which has a static property called Default (e.g. > > EqualityComparer.Default). That internally (in static constructor) > figures out, based on inspecting T's capabilities and type, which is the > default equality comparer and then hands out that sole instance on each > Default access. > > Ultimately, what you want here is instead of doing T.default.equals(t), > you want some other class to define an " boolean equals(T x, T y)" > which can internally handle one side being null (for ref types). > > Treating "Equals" as a strategy is an interesting idea. Is it faster than > just specializing appropriate bytecode? > I'd still like a clean 'eq' operator to call this internally. > > > Well behaved classes already should be handling null inputs in their > equals() impl, so we should find a way to reuse that instead of introducing > a new operator/keyword. > > Well, I noticed we needed an "x == y" check to ensure "null eq null" > answered true. > > There are quite a few wrinkles with Equals comparison.. one of the > benefits of bytecode/ monomorphic implementation for reftypes, might be > that Hotspot can optimize some of the checks away. > > Thanks for your comments! > > > Regards, > Thomas > > On Thu, Jan 8, 2015 at 12:33 PM, Vitaly Davidovich > wrote: > >> For value-types: >>> x eq y --> (x == y) >> >> >> You mean for primitives? Custom value types won't work with == as there's >> no operator overloading. >> >> Getting back to the null aspect though, here's my take ... >> >> I think we should not introduce a new operator such as eq when we already >> have a "universal" equals. What you really want to say is something like >> this: >> >> void foo(T t) { >> if (T.default.equals(t)) >> System.out.println("got default/null"); >> else >> System.out.println("got non-default/null"); >> } >> >> Now, the obvious problem here is that this NPEs on reference types since >> T.default yields a null and you can't reverse the comparison because t may >> be null. So, what you'd like here is similar to what .NET does. Introduce >> an IEqualityComparer-like JDK interface, and provide a way to obtain a >> "default" comparer for a given type. In .NET, there's a generic class >> called EqualityComparer, which has a static property called Default (e.g. >> EqualityComparer.Default). That internally (in static constructor) >> figures out, based on inspecting T's capabilities and type, which is the >> default equality comparer and then hands out that sole instance on each >> Default access. Ultimately, what you want here is instead of doing >> T.default.equals(t), you want some other class to define an " >> boolean equals(T x, T y)" which can internally handle one side being null >> (for ref types). >> >> tldr; well behaved classes already should be handling null inputs in >> their equals() impl, so we should find a way to reuse that instead of >> introducing a new operator/keyword. >> >> >> On Wed, Jan 7, 2015 at 6:07 PM, Thomas W wrote: >> >>> Hi Simon, Vitaly, people, >>> >>> Lots of people have been talking about adapting Collections for 'any' >>> type. Going forwards, I do not believe we should be writing code that >>> explicitly uses/ or checks for nulls (though we may be backwards-compatible >>> for a while) any more than we should be writing or calling a Duck.woof() >>> method. >>> >>> To answering a few questions: >>> >>> > a) What are the actual semantics? >>> >>> For ref-types: >>> x eq y --> (x != null && x.equals(y)) >>> For value-types: >>> x eq y --> (x == y) >>> >>> > b) How is this different to the approaches already outlines earlier? >>> >>> It provides a basic logical building-block (equality check) at the >>> language level, in a way that works both for primitives/valuetypes and for >>> nullable references. It addresses a very big long-standing pain point in >>> Java application development. It avoids writing/ or encouraging >>> "Duck.woof()" style-hacks -- null checks -- into a domain that does not >>> uniformly support them. >>> >>> > Why do we need a new special operator? What's wrong with using equals >>> () and having the specializer rewrite that for primitives? >>> > What's wrong with using equals() and having the specializer rewrite >>> that for primitives? For other types >>> > (refs and custom value types), it should just delegate to the real >>> equals. >>> >>> To have a proper logical equals in one operator, which is independent >>> of implementation & null-safe for reftypes. >>> >>> What's wrong is writing null-checks & nulls, in a domain where nulls may >>> not exist. >>> We are not dealing with Dog, we are dealing with Animal and there should >>> be no 'woof()' method. >>> >>> > What's wrong with using equals() and having the specializer rewrite >>> that for primitives? For other types >>> > (refs and custom value types), it should just delegate to the real >>> equals. >>> >>> What's wrong with an operator that logically has exactly the right >>> meaning, without exposing details that don't exist in the domain? >>> It also provides a vast benefit for application programmers, as this >>> kind of requirement is ridiculously common in hundreds of millions of lines >>> of application code. >>> >>> >>> Regards, >>> Thomas Whitmore >>> >> >> > From twhitmore.nz at gmail.com Thu Jan 8 00:24:15 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Thu, 8 Jan 2015 13:24:15 +1300 Subject: valhalla-dev Digest, Vol 7, Issue 26 In-Reply-To: References: Message-ID: Hi Vitaly, good questions. Luckily there are good answers :) > So say I'm authoring a generic class and call System.out.println() - how do I know which method is going to be called? The most specific one, matching the specialized type of your 'any T' parameter. > Suppose println returned different types depending on overload - what's my return value type? I can't target type it because that implies I know what the generic type will be and I don't, I'm just shipping a generic library. Since we're conservative, we should only consider methods having a return-type/ signature consistent with the "base" method. This means we look at the signature of PrintStream.print(Object arg) and only consider methods consistent with that. I think that would work fine, and be both conservative & reliable. Would that work for you? I did see that Brian suggested some kind of explicit enhancement of existing libraries.. not sure if that's a "print(any T)" method, or if so, how it would actually implement/ or despatch efficiently. Regards, Thomas From mikeb01 at gmail.com Thu Jan 8 00:33:01 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Thu, 8 Jan 2015 13:33:01 +1300 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: Yes, but with a slight step further. As you point out specialising everything will lead to bloating the number of classes. I was thinking about Hotspot specialising some combinations of generic classes and specific reference types based on some heuristic/profiling information. On 8 January 2015 at 11:32, Vitaly Davidovich wrote: > Ah, you're talking about specialized classes as a whole (I was referring > to just the arrays aspect). Yes, if it were to specialize every single > type, then you'd get better type information. Downside is you now explode > the number of method definitions in the runtime. In .NET, for example, > generic methods are not specialized for reference types, in part for this > reason I believe. Generally speaking, the downside to creating distinct > structures per type is the explosion in the number of types at runtime. I > encourage you to read this oldish blog post by Joe Duffy (MSFT engineer): > http://joeduffyblog.com/2011/10/23/on-generics-and-some-of-the-associated-overheads/ > > On Wed, Jan 7, 2015 at 5:23 PM, Michael Barker wrote: > >> My understand is that it does type profiling at the callsite and >> something like HashMap.hash() will encounter such wide variety of classes >> that it will rarely be anything other than fully mega-morphic. My guess >> was that if there was specialised class for a specific reference type then >> this could become mono-morphic. >> >> On 8 January 2015 at 11:12, Vitaly Davidovich wrote: >> >>> Right, but the reason I'm doubtful that this will have any impact is >>> because the JIT already does type profiling, and the runtime types it sees >>> (and the statistics around that) won't change due to erasure. My "make its >>> life easier" comment was a guess that perhaps some code paths in the >>> optimizer don't need to be taken (e.g. don't look at profiling info if it >>> now knows statically that an array is composed of final classes). >>> >>> On Wed, Jan 7, 2015 at 5:09 PM, Michael Barker >>> wrote: >>> >>>> The current functionality continues with the erasure plan. However, I >>>>>> wouldn't mind doing better! >>>>> >>>>> >>>>> Yeah, I can't immediately think of a critical reason why it can't stay >>>>> erased. For JIT optimizer, having a narrower upper bound on the type may >>>>> make its life easier, although I don't know if it'll have any material >>>>> difference. The one question is what reflection will do (and any code >>>>> based on reflection, such as custom serialization, code generation, etc): >>>>> >>>> >>>> (Caveat, I'm not a compiler expert so this is a bit of a guess.) >>>> >>>> One possible place where this could be used with within the optimiser. >>>> E.g. if Hotspot could see a specialised HashMap instead of >>>> an erased one, then it could determine that calls to hashCode and equals >>>> would be mono-morphic and apply more aggressive in-lining. This could lead >>>> to jump in performance across a broad ranges of apps (hands up who uses >>>> Strings and HashMaps :-). My understand is that the mega-morhpic dispatch >>>> (of hashCode/equals) is one of the more significant costs within HashMap. >>>> >>>> If that was possible then it would be pretty cool! >>>> >>>> Mike. >>>> >>> >>> >> > From vitalyd at gmail.com Thu Jan 8 00:33:44 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 19:33:44 -0500 Subject: valhalla-dev Digest, Vol 7, Issue 26 In-Reply-To: References: Message-ID: But if you're taking the conservative approach then you can only be calling that one Object overload (you can't overload by return value only) and then where's the specialization? Sent from my phone On Jan 7, 2015 7:24 PM, "Thomas W" wrote: > Hi Vitaly, good questions. > > Luckily there are good answers :) > > > So say I'm authoring a generic class and call System.out.println() - > how do I know which method is going to be called? > > The most specific one, matching the specialized type of your 'any T' > parameter. > > > Suppose println returned different types depending on overload - what's > my return value type? I can't target type it because that implies I know > what the generic type will be and I don't, I'm just shipping a generic > library. > > Since we're conservative, we should only consider methods having a > return-type/ signature consistent with the "base" method. This means we > look at the signature of PrintStream.print(Object arg) and only consider > methods consistent with that. I think that would work fine, and be both > conservative & reliable. > > Would that work for you? > > I did see that Brian suggested some kind of explicit enhancement of > existing libraries.. not sure if that's a "print(any T)" method, or if > so, how it would actually implement/ or despatch efficiently. > > > Regards, > Thomas > From vitalyd at gmail.com Thu Jan 8 00:35:25 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 19:35:25 -0500 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: Problem is specialization is javac-time, VM not involved. Sent from my phone On Jan 7, 2015 7:33 PM, "Michael Barker" wrote: > Yes, but with a slight step further. As you point out specialising > everything will lead to bloating the number of classes. I was thinking > about Hotspot specialising some combinations of generic classes and > specific reference types based on some heuristic/profiling information. > > On 8 January 2015 at 11:32, Vitaly Davidovich wrote: > >> Ah, you're talking about specialized classes as a whole (I was referring >> to just the arrays aspect). Yes, if it were to specialize every single >> type, then you'd get better type information. Downside is you now explode >> the number of method definitions in the runtime. In .NET, for example, >> generic methods are not specialized for reference types, in part for this >> reason I believe. Generally speaking, the downside to creating distinct >> structures per type is the explosion in the number of types at runtime. I >> encourage you to read this oldish blog post by Joe Duffy (MSFT engineer): >> http://joeduffyblog.com/2011/10/23/on-generics-and-some-of-the-associated-overheads/ >> >> On Wed, Jan 7, 2015 at 5:23 PM, Michael Barker wrote: >> >>> My understand is that it does type profiling at the callsite and >>> something like HashMap.hash() will encounter such wide variety of classes >>> that it will rarely be anything other than fully mega-morphic. My guess >>> was that if there was specialised class for a specific reference type then >>> this could become mono-morphic. >>> >>> On 8 January 2015 at 11:12, Vitaly Davidovich wrote: >>> >>>> Right, but the reason I'm doubtful that this will have any impact is >>>> because the JIT already does type profiling, and the runtime types it sees >>>> (and the statistics around that) won't change due to erasure. My "make its >>>> life easier" comment was a guess that perhaps some code paths in the >>>> optimizer don't need to be taken (e.g. don't look at profiling info if it >>>> now knows statically that an array is composed of final classes). >>>> >>>> On Wed, Jan 7, 2015 at 5:09 PM, Michael Barker >>>> wrote: >>>> >>>>> The current functionality continues with the erasure plan. However, I >>>>>>> wouldn't mind doing better! >>>>>> >>>>>> >>>>>> Yeah, I can't immediately think of a critical reason why it can't >>>>>> stay erased. For JIT optimizer, having a narrower upper bound on the type >>>>>> may make its life easier, although I don't know if it'll have any material >>>>>> difference. The one question is what reflection will do (and any code >>>>>> based on reflection, such as custom serialization, code generation, etc): >>>>>> >>>>> >>>>> (Caveat, I'm not a compiler expert so this is a bit of a guess.) >>>>> >>>>> One possible place where this could be used with within the >>>>> optimiser. E.g. if Hotspot could see a specialised HashMap >>>>> instead of an erased one, then it could determine that calls to hashCode >>>>> and equals would be mono-morphic and apply more aggressive in-lining. This >>>>> could lead to jump in performance across a broad ranges of apps (hands up >>>>> who uses Strings and HashMaps :-). My understand is that the mega-morhpic >>>>> dispatch (of hashCode/equals) is one of the more significant costs within >>>>> HashMap. >>>>> >>>>> If that was possible then it would be pretty cool! >>>>> >>>>> Mike. >>>>> >>>> >>>> >>> >> > From twhitmore.nz at gmail.com Thu Jan 8 01:17:37 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Thu, 8 Jan 2015 14:17:37 +1300 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: References: Message-ID: Hi Vitaly, people, > So i thought having equality externalized means you don't deal with it in the generic class - you call a black box passing a default and target item, and get an answer. That's fine as a possible *implementation mechanism*, but it doesn't help the language syntax. Nor does it expose so much to the Hotspot optimizer. It's a reasonable idea and I'm happy to consider it, though :) > Now yes the default for ref types is null but for better or worse null isn't going away in java so pretending it doesn't exist is just silly. Null absolutely exists in reftypes, but it absolutely does not for primitives or value-types. We should be recognizing the /logical meaning/ of eg Collections code that writes 'null', and provide a meaningful expression that applies across both reftypes and value-types. I did previously note that nulls are a huge pain-point for application developers in Java. So it seems very unlikely that I am pretending they don't exist! I want to address null at a language level for the Equals operator, so that we have a "universal equals" that works at a logical level -- across nullable/ and valuetypes equally. This would be hugely beneficial to application development, in a way that IEquatable/ IEqualityComparer would not. A quick outline of what 'null' is typically used for in Collections-type code may help illustrate my point: 1) elements[i] = null; // clearing or defaulting; --> T.empty aka T.default. 2) if (cand != null && cand.equals(item)) // 'eq' operator; --> cand eq item 3) if (node == null) return null; // not found returning sentinel; --> if (node == null) return T.sentinel // of course, we should suggest to use getOrDefault() or getOptional() if a sentinel is unsuitable -- for either reftypes or value-types! I can be quite clear, this very fairly recognizes & supports 'null' in reftypes. But very importantly, it avoids requiring or encouraging 'null' to be written in such code -- by providing meaningful alternatives, which are meaningful for primitives/ valuetypes as well as for reftypes. Animal.speak(), rather than Duck.woof(). Regards, Thomas From mikeb01 at gmail.com Thu Jan 8 01:20:37 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Thu, 8 Jan 2015 14:20:37 +1300 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: Aaah, okay. For some reason I'd convinced myself that VM would have the mechanics to actually do the specialisation. On 8 January 2015 at 13:35, Vitaly Davidovich wrote: > Problem is specialization is javac-time, VM not involved. > > Sent from my phone > On Jan 7, 2015 7:33 PM, "Michael Barker" wrote: > >> Yes, but with a slight step further. As you point out specialising >> everything will lead to bloating the number of classes. I was thinking >> about Hotspot specialising some combinations of generic classes and >> specific reference types based on some heuristic/profiling information. >> >> On 8 January 2015 at 11:32, Vitaly Davidovich wrote: >> >>> Ah, you're talking about specialized classes as a whole (I was referring >>> to just the arrays aspect). Yes, if it were to specialize every single >>> type, then you'd get better type information. Downside is you now explode >>> the number of method definitions in the runtime. In .NET, for example, >>> generic methods are not specialized for reference types, in part for this >>> reason I believe. Generally speaking, the downside to creating distinct >>> structures per type is the explosion in the number of types at runtime. I >>> encourage you to read this oldish blog post by Joe Duffy (MSFT engineer): >>> http://joeduffyblog.com/2011/10/23/on-generics-and-some-of-the-associated-overheads/ >>> >>> On Wed, Jan 7, 2015 at 5:23 PM, Michael Barker >>> wrote: >>> >>>> My understand is that it does type profiling at the callsite and >>>> something like HashMap.hash() will encounter such wide variety of classes >>>> that it will rarely be anything other than fully mega-morphic. My guess >>>> was that if there was specialised class for a specific reference type then >>>> this could become mono-morphic. >>>> >>>> On 8 January 2015 at 11:12, Vitaly Davidovich >>>> wrote: >>>> >>>>> Right, but the reason I'm doubtful that this will have any impact is >>>>> because the JIT already does type profiling, and the runtime types it sees >>>>> (and the statistics around that) won't change due to erasure. My "make its >>>>> life easier" comment was a guess that perhaps some code paths in the >>>>> optimizer don't need to be taken (e.g. don't look at profiling info if it >>>>> now knows statically that an array is composed of final classes). >>>>> >>>>> On Wed, Jan 7, 2015 at 5:09 PM, Michael Barker >>>>> wrote: >>>>> >>>>>> The current functionality continues with the erasure plan. However, >>>>>>>> I wouldn't mind doing better! >>>>>>> >>>>>>> >>>>>>> Yeah, I can't immediately think of a critical reason why it can't >>>>>>> stay erased. For JIT optimizer, having a narrower upper bound on the type >>>>>>> may make its life easier, although I don't know if it'll have any material >>>>>>> difference. The one question is what reflection will do (and any code >>>>>>> based on reflection, such as custom serialization, code generation, etc): >>>>>>> >>>>>> >>>>>> (Caveat, I'm not a compiler expert so this is a bit of a guess.) >>>>>> >>>>>> One possible place where this could be used with within the >>>>>> optimiser. E.g. if Hotspot could see a specialised HashMap >>>>>> instead of an erased one, then it could determine that calls to hashCode >>>>>> and equals would be mono-morphic and apply more aggressive in-lining. This >>>>>> could lead to jump in performance across a broad ranges of apps (hands up >>>>>> who uses Strings and HashMaps :-). My understand is that the mega-morhpic >>>>>> dispatch (of hashCode/equals) is one of the more significant costs within >>>>>> HashMap. >>>>>> >>>>>> If that was possible then it would be pretty cool! >>>>>> >>>>>> Mike. >>>>>> >>>>> >>>>> >>>> >>> >> From richard.warburton at gmail.com Thu Jan 8 01:21:33 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Thu, 8 Jan 2015 01:21:33 +0000 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: Hi, > The current functionality continues with the erasure plan. However, I > > wouldn't mind doing better! > > > Yeah, I can't immediately think of a critical reason why it can't stay > erased. For JIT optimizer, having a narrower upper bound on the type may > make its life easier, although I don't know if it'll have any material > difference. > At the moment a lot of collection classes have checkcast overhead on reads. For example List.get() or Map.get() both end up checkcasting their return value IIRC. Specialising the ref implementation gives you the opportunity to remove this checkcast. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From vitalyd at gmail.com Thu Jan 8 01:39:01 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 20:39:01 -0500 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: I don't think we need specialization for this, just reification. Specialization would help what Mike was talking about (at the cost of bloat if not controlled somehow). Sent from my phone On Jan 7, 2015 8:21 PM, "Richard Warburton" wrote: > Hi, > > > The current functionality continues with the erasure plan. However, I >> > wouldn't mind doing better! >> >> >> Yeah, I can't immediately think of a critical reason why it can't stay >> erased. For JIT optimizer, having a narrower upper bound on the type may >> make its life easier, although I don't know if it'll have any material >> difference. >> > > At the moment a lot of collection classes have checkcast overhead on > reads. For example List.get() or Map.get() both end up checkcasting their > return value IIRC. Specialising the ref implementation gives you the > opportunity to remove this checkcast. > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > From vitalyd at gmail.com Thu Jan 8 01:48:17 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 20:48:17 -0500 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: References: Message-ID: Why do you need language feature for #2? What's wrong with calling a lib method that wraps this up for you? Changing the language is a high barrier to entry and personally, it seems like eq is just shorthand for a generic method that can deal with this just as well. Now, one language change I'd appreciate is the elvis operator since you can't emulate that via lib. But, let me stop here as we're going off on a tangent. Broadly speaking, I don't think value type and specialization work should also be used to tackle any perceived issues with null. Sent from my phone On Jan 7, 2015 8:17 PM, "Thomas W" wrote: > Hi Vitaly, people, > > > So i thought having equality externalized means you don't deal with it > in the generic class - you call a black box passing a default and target > item, and get an answer. > > That's fine as a possible *implementation mechanism*, but it doesn't help > the language syntax. Nor does it expose so much to the Hotspot optimizer. > It's a reasonable idea and I'm happy to consider it, though :) > > > Now yes the default for ref types is null but for better or worse null > isn't going away in java so pretending it doesn't exist is just silly. > > Null absolutely exists in reftypes, but it absolutely does not for > primitives or value-types. We should be recognizing the /logical meaning/ > of eg Collections code that writes 'null', and provide a meaningful > expression that applies across both reftypes and value-types. > > I did previously note that nulls are a huge pain-point for application > developers in Java. So it seems very unlikely that I am pretending they > don't exist! I want to address null at a language level for the Equals > operator, so that we have a "universal equals" that works at a logical > level -- across nullable/ and valuetypes equally. This would be hugely > beneficial to application development, in a way that IEquatable/ > IEqualityComparer would not. > > A quick outline of what 'null' is typically used for in Collections-type > code may help illustrate my point: > > 1) elements[i] = null; > // clearing or defaulting; --> T.empty aka T.default. > > 2) if (cand != null && cand.equals(item)) > // 'eq' operator; --> cand eq item > > 3) if (node == null) return null; > // not found returning sentinel; --> if (node == null) return > T.sentinel > // of course, we should suggest to use getOrDefault() or > getOptional() if a sentinel is unsuitable -- for either reftypes or > value-types! > > I can be quite clear, this very fairly recognizes & supports 'null' in > reftypes. But very importantly, it avoids requiring or encouraging 'null' > to be written in such code -- by providing meaningful alternatives, which > are meaningful for primitives/ valuetypes as well as for reftypes. > > Animal.speak(), rather than Duck.woof(). > > Regards, > Thomas > From brian.goetz at oracle.com Thu Jan 8 02:12:02 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jan 2015 21:12:02 -0500 Subject: Casting reference array to any-T array. In-Reply-To: References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> Message-ID: <54ADE772.5020807@oracle.com> What makes you say that? Right now, specialization is class-load time. With some possible changes to the VM (to be revealed when more baked) we can push it later and get more sharing. On 1/7/2015 7:35 PM, Vitaly Davidovich wrote: > Problem is specialization is javac-time, VM not involved. > > Sent from my phone > > On Jan 7, 2015 7:33 PM, "Michael Barker" > wrote: > > Yes, but with a slight step further. As you point out specialising > everything will lead to bloating the number of classes. I was > thinking about Hotspot specialising some combinations of generic > classes and specific reference types based on some > heuristic/profiling information. > > On 8 January 2015 at 11:32, Vitaly Davidovich > wrote: > > Ah, you're talking about specialized classes as a whole (I was > referring to just the arrays aspect). Yes, if it were to > specialize every single type, then you'd get better type > information. Downside is you now explode the number of method > definitions in the runtime. In .NET, for example, generic > methods are not specialized for reference types, in part for > this reason I believe. Generally speaking, the downside to > creating distinct structures per type is the explosion in the > number of types at runtime. I encourage you to read this oldish > blog post by Joe Duffy (MSFT engineer): > http://joeduffyblog.com/2011/10/23/on-generics-and-some-of-the-associated-overheads/ > > On Wed, Jan 7, 2015 at 5:23 PM, Michael Barker > > wrote: > > My understand is that it does type profiling at the callsite > and something like HashMap.hash() will encounter such wide > variety of classes that it will rarely be anything other > than fully mega-morphic. My guess was that if there was > specialised class for a specific reference type then this > could become mono-morphic. > > On 8 January 2015 at 11:12, Vitaly Davidovich > > wrote: > > Right, but the reason I'm doubtful that this will have > any impact is because the JIT already does type > profiling, and the runtime types it sees (and the > statistics around that) won't change due to erasure. My > "make its life easier" comment was a guess that perhaps > some code paths in the optimizer don't need to be taken > (e.g. don't look at profiling info if it now knows > statically that an array is composed of final classes). > > On Wed, Jan 7, 2015 at 5:09 PM, Michael Barker > > wrote: > > The current functionality continues with the > erasure plan. However, I wouldn't mind > doing better! > > > Yeah, I can't immediately think of a critical > reason why it can't stay erased. For JIT > optimizer, having a narrower upper bound on the > type may make its life easier, although I don't > know if it'll have any material difference. The > one question is what reflection will do (and any > code based on reflection, such as custom > serialization, code generation, etc): > > > (Caveat, I'm not a compiler expert so this is a bit > of a guess.) > > One possible place where this could be used with > within the optimiser. E.g. if Hotspot could see a > specialised HashMap instead of an > erased one, then it could determine that calls to > hashCode and equals would be mono-morphic and apply > more aggressive in-lining. This could lead to jump > in performance across a broad ranges of apps (hands > up who uses Strings and HashMaps :-). My understand > is that the mega-morhpic dispatch (of > hashCode/equals) is one of the more significant > costs within HashMap. > > If that was possible then it would be pretty cool! > > Mike. > > > > > From vitalyd at gmail.com Thu Jan 8 02:25:40 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 7 Jan 2015 21:25:40 -0500 Subject: Casting reference array to any-T array. In-Reply-To: <54ADE772.5020807@oracle.com> References: <54ADA3F2.6050102@oracle.com> <54ADA9AA.6080802@oracle.com> <54ADE772.5020807@oracle.com> Message-ID: Sorry, that's my fault - misspoke. However, main point still stands in that optimizer isn't involved in this. Now, if you're saying you guys are considering moving this further into the VM then ok, I'll wait and see what you have in mind. Not immediately clear to me how you'd use profiling info to selectively specialize and then reload types or some other heuristic, but that's an interesting topic. I guess one thing that's still unclear to me and makes having some of these conversations a bit more difficult is exactly how much of this will be pushed into the JVM (and how deep in there). I'm sure you guys don't quite know yet, but I guess we'll discover through these threads :). Sent from my phone On Jan 7, 2015 9:12 PM, "Brian Goetz" wrote: > What makes you say that? Right now, specialization is class-load time. > With some possible changes to the VM (to be revealed when more baked) we > can push it later and get more sharing. > > On 1/7/2015 7:35 PM, Vitaly Davidovich wrote: > >> Problem is specialization is javac-time, VM not involved. >> >> Sent from my phone >> >> On Jan 7, 2015 7:33 PM, "Michael Barker" > > wrote: >> >> Yes, but with a slight step further. As you point out specialising >> everything will lead to bloating the number of classes. I was >> thinking about Hotspot specialising some combinations of generic >> classes and specific reference types based on some >> heuristic/profiling information. >> >> On 8 January 2015 at 11:32, Vitaly Davidovich > > wrote: >> >> Ah, you're talking about specialized classes as a whole (I was >> referring to just the arrays aspect). Yes, if it were to >> specialize every single type, then you'd get better type >> information. Downside is you now explode the number of method >> definitions in the runtime. In .NET, for example, generic >> methods are not specialized for reference types, in part for >> this reason I believe. Generally speaking, the downside to >> creating distinct structures per type is the explosion in the >> number of types at runtime. I encourage you to read this oldish >> blog post by Joe Duffy (MSFT engineer): >> http://joeduffyblog.com/2011/10/23/on-generics-and-some-of- >> the-associated-overheads/ >> >> On Wed, Jan 7, 2015 at 5:23 PM, Michael Barker >> > wrote: >> >> My understand is that it does type profiling at the callsite >> and something like HashMap.hash() will encounter such wide >> variety of classes that it will rarely be anything other >> than fully mega-morphic. My guess was that if there was >> specialised class for a specific reference type then this >> could become mono-morphic. >> >> On 8 January 2015 at 11:12, Vitaly Davidovich >> > wrote: >> >> Right, but the reason I'm doubtful that this will have >> any impact is because the JIT already does type >> profiling, and the runtime types it sees (and the >> statistics around that) won't change due to erasure. My >> "make its life easier" comment was a guess that perhaps >> some code paths in the optimizer don't need to be taken >> (e.g. don't look at profiling info if it now knows >> statically that an array is composed of final classes). >> >> On Wed, Jan 7, 2015 at 5:09 PM, Michael Barker >> > wrote: >> >> The current functionality continues with the >> erasure plan. However, I wouldn't mind >> doing better! >> >> >> Yeah, I can't immediately think of a critical >> reason why it can't stay erased. For JIT >> optimizer, having a narrower upper bound on the >> type may make its life easier, although I don't >> know if it'll have any material difference. The >> one question is what reflection will do (and any >> code based on reflection, such as custom >> serialization, code generation, etc): >> >> >> (Caveat, I'm not a compiler expert so this is a bit >> of a guess.) >> >> One possible place where this could be used with >> within the optimiser. E.g. if Hotspot could see a >> specialised HashMap instead of an >> erased one, then it could determine that calls to >> hashCode and equals would be mono-morphic and apply >> more aggressive in-lining. This could lead to jump >> in performance across a broad ranges of apps (hands >> up who uses Strings and HashMaps :-). My understand >> is that the mega-morhpic dispatch (of >> hashCode/equals) is one of the more significant >> costs within HashMap. >> >> If that was possible then it would be pretty cool! >> >> Mike. >> >> >> >> >> >> From twhitmore.nz at gmail.com Thu Jan 8 04:58:05 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Thu, 8 Jan 2015 17:58:05 +1300 Subject: 'Equals' in any-typed code: VERY BIG PROPOSAL Message-ID: Hi Vitaly, Thanks for your interest in this area. > Why do you need language feature for #2? What's wrong with calling a lib > method that wraps this up for you? Collections are built on 'eq' and 'hashCode'. It's a fundamental basic operator on values, and we're now moving to support values. So, change is here & we have an opportunity to get it right. You are correct, a static helper/ or EqualsStrategy can also achieve this for collections. But for the language more widely & application programming in general, an 'eq' operator would be a great help. As well as being a very simple & clean solution for specializable Collections. Here's a demonstration of length: if (car.getColor() != null && car.getColor().equals( someColor))) { if (ObjectUtils.equals( car.getColor(), someColor)) { if (car.getColor() eq someColor)) { There is a massive difference in the amount of typing, and in clarity. > Changing the language is a high barrier to entry Language changes are necessary to have any kind of syntax for specialization, that is not a rabidly horrible bunch of hacks. So I think we are over that barrier already. Millions will cheer & thanks you! > and personally, it seems like eq is just shorthand for a generic method that can deal with this > just as well. Logically, 'equals' is an _operator_ not a _method_. Now is the time to get this right. Alongside specialization, the Java 10 release will be marked with fireworks, champagne & celebration. > one language change I'd appreciate is the elvis operator since you can't emulate that via lib. Really, we can only consider the most concise & conservative set of operators -- that will stand on their own, solve the very biggest bugbear, and not be subject to "scope creep". Having a clear boundary for features is very important. I like the Elvis operator too -- and case-insensitive equals for strings -- but I'm putting forward a bounded proposal, prompted by the need for 'equals' in Collections. I want to start with that; it's also the most common form in Java application development. Regards, Thomas From peter.levart at gmail.com Thu Jan 8 08:09:32 2015 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 08 Jan 2015 09:09:32 +0100 Subject: runtime failure when "any T" generic method calls another "any T" generic method Message-ID: <54AE3B3C.5080006@gmail.com> Hi, Here's another problem I found: public class ValhallaTest { static void test2() { Class clazz = ValhallaTest.class; // line 7 System.out.println("test2 >>> " + clazz); } static void test1() { Class clazz = ValhallaTest.class; System.out.println("test1 >>> " + clazz); ValhallaTest.test2(); } public static void main(String[] args) { ValhallaTest.test1(); ValhallaTest.test1(); ValhallaTest.test1(); ValhallaTest.test1(); ValhallaTest.test1(); ValhallaTest.test1(); ValhallaTest.test1(); ValhallaTest.test1(); ValhallaTest.test1(); ValhallaTest.test1(); } } Which prints: Specializing method ValhallaTest$test1${0=Z}.test1()V with class=[] and method=[Z] Specializing ValhallaTest${0=Z}; searching for ValhallaTest.class (not found) Specializing ValhallaTest${0=Z}; searching for ValhallaTest.class (found) test1 >>> class ValhallaTest${0=Z} Specializing method ValhallaTest$test2${0=Z}.test2()V with class=[] and method=[Z] test2 >>> class ValhallaTest${0=Z} Specializing method ValhallaTest$test1${0=B}.test1()V with class=[] and method=[B] Specializing ValhallaTest${0=B}; searching for ValhallaTest.class (not found) Specializing ValhallaTest${0=B}; searching for ValhallaTest.class (found) test1 >>> class ValhallaTest${0=B} Specializing method ValhallaTest$test2${0=B}.test2()V with class=[] and method=[B] test2 >>> class ValhallaTest${0=B} Specializing method ValhallaTest$test1${0=C}.test1()V with class=[] and method=[C] Specializing ValhallaTest${0=C}; searching for ValhallaTest.class (not found) Specializing ValhallaTest${0=C}; searching for ValhallaTest.class (found) test1 >>> class ValhallaTest${0=C} Specializing method ValhallaTest$test2${0=C}.test2()V with class=[] and method=[C] test2 >>> class ValhallaTest${0=C} Specializing method ValhallaTest$test1${0=S}.test1()V with class=[] and method=[S] Specializing ValhallaTest${0=S}; searching for ValhallaTest.class (not found) Specializing ValhallaTest${0=S}; searching for ValhallaTest.class (found) test1 >>> class ValhallaTest${0=S} Specializing method ValhallaTest$test2${0=S}.test2()V with class=[] and method=[S] test2 >>> class ValhallaTest${0=S} Specializing method ValhallaTest$test1${0=I}.test1()V with class=[] and method=[I] Specializing ValhallaTest${0=I}; searching for ValhallaTest.class (not found) Specializing ValhallaTest${0=I}; searching for ValhallaTest.class (found) test1 >>> class ValhallaTest${0=I} Specializing method ValhallaTest$test2${0=I}.test2()V with class=[] and method=[I] test2 >>> class ValhallaTest${0=I} Specializing method ValhallaTest$test1${0=J}.test1()V with class=[] and method=[J] Specializing ValhallaTest${0=J}; searching for ValhallaTest.class (not found) Specializing ValhallaTest${0=J}; searching for ValhallaTest.class (found) test1 >>> class ValhallaTest${0=J} Specializing method ValhallaTest$test2${0=J}.test2()V with class=[] and method=[J] test2 >>> class ValhallaTest${0=J} Specializing method ValhallaTest$test1${0=F}.test1()V with class=[] and method=[F] Specializing ValhallaTest${0=F}; searching for ValhallaTest.class (not found) Specializing ValhallaTest${0=F}; searching for ValhallaTest.class (found) test1 >>> class ValhallaTest${0=F} Specializing method ValhallaTest$test2${0=F}.test2()V with class=[] and method=[F] test2 >>> class ValhallaTest${0=F} Specializing method ValhallaTest$test1${0=D}.test1()V with class=[] and method=[D] Specializing ValhallaTest${0=D}; searching for ValhallaTest.class (not found) Specializing ValhallaTest${0=D}; searching for ValhallaTest.class (found) test1 >>> class ValhallaTest${0=D} Specializing method ValhallaTest$test2${0=D}.test2()V with class=[] and method=[D] test2 >>> class ValhallaTest${0=D} test1 >>> class ValhallaTest Specializing method ValhallaTest$test2${}.test2()V with class=[] and method=[TT;] Exception in thread "main" java.lang.NoClassDefFoundError: ValhallaTest.class is evaluated, but only for reference type T. Note that there's no specialization for test1() being triggered for T=reference (which is understandable), but there is a specialization being triggered for cascaded call to test2() in this case. In my superficial understanding a call to: ValhallaTest.test1(); ...is implemented as invokestatic, but a call from test1() to: ValhallaTest.test2(); ...is implemented as invokedynamic. So in case such call comes from non-specialized test1(), it should be wired directly to the nonspecialized test2() method, right? Or, couldn't the call from nonspecialized test1() to .test2() also be implemented as invokestatic and be converted to invokedynamic only by specialization? Regards, Peter From ali.ebrahimi1781 at gmail.com Thu Jan 8 08:54:33 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Thu, 8 Jan 2015 12:24:33 +0330 Subject: Why JVM can not type infer type params but javac can do? Message-ID: Hi, maybe this is stupid question, but want to ask and what is requirements for this? Array instantiation sample: public T[] newArray(int){ return new T[0]; } even for non-any T: public T[] newArray(int){ return new T[0]; } With thousands number of optimization and inlining mechanics equipped in jvm, What is the reasoning behind not allowing to jvm to be aware of generics info emitted in classfilles? Why jvm should not able do following inlining? String[] strs = newArray(int) => String[] strs = new String[0]; I mean even in jitted code. I think some time ago I read some papers in similar area. -- Best Regards, Ali Ebrahimi From palo.marton at gmail.com Thu Jan 8 09:47:39 2015 From: palo.marton at gmail.com (Palo Marton) Date: Thu, 8 Jan 2015 10:47:39 +0100 Subject: =?UTF-8?Q?Value_types_=2D_compatibility_with_existing_=E2=80=9Cvalue_o?= =?UTF-8?Q?bjects=E2=80=9D?= Message-ID: Hi. I have a question about value types: Will it be possible (within valhalla) to rewrite java String as value type in JDK and use it in all places in JDK where it makes sense? (eg. Class.forName(string), Object.toString(), Integer.parseInt(string). ?) Or more general question: There are already plenty of ?value objects? in existing java code. In JDK we have, String, File, Point2D,... Also many people have already declared their own class Point { final x,y; } and similar objects. Once we move to value types in java, will it be possible to rewrite these to value types without breaking compatibility with old pre-value-types code? E.g. if I change my Point object to value type in my library, can this new library (jar) still be used in other projects that were written before this change? And without need to recompile those other projects? From maurizio.cimadamore at oracle.com Thu Jan 8 11:02:23 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 08 Jan 2015 11:02:23 +0000 Subject: runtime failure when "any T" generic method calls another "any T" generic method In-Reply-To: <54AE3B3C.5080006@gmail.com> References: <54AE3B3C.5080006@gmail.com> Message-ID: <54AE63BF.5030301@oracle.com> Hi Peter, this is an issue we know about - the generated bytecode is correct - but it's likely we'll need to enhance it to have the bootstrap saying something like: TT;=T (i.e. tvar = type) This will help the specializer figuring out that there's really nothing to do, since T is itself not specialized, so the erased version of the method should be used. Maurizio On 08/01/15 08:09, Peter Levart wrote: > Hi, > > Here's another problem I found: > > > public class ValhallaTest { > > static void test2() { > Class clazz = ValhallaTest.class; // line 7 > System.out.println("test2 >>> " + clazz); > } > > static void test1() { > Class clazz = ValhallaTest.class; > System.out.println("test1 >>> " + clazz); > > ValhallaTest.test2(); > } > > public static void main(String[] args) { > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > } > } > > > Which prints: > > > Specializing method ValhallaTest$test1${0=Z}.test1()V with class=[] > and method=[Z] > Specializing ValhallaTest${0=Z}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=Z}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=Z} > Specializing method ValhallaTest$test2${0=Z}.test2()V with class=[] > and method=[Z] > test2 >>> class ValhallaTest${0=Z} > Specializing method ValhallaTest$test1${0=B}.test1()V with class=[] > and method=[B] > Specializing ValhallaTest${0=B}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=B}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=B} > Specializing method ValhallaTest$test2${0=B}.test2()V with class=[] > and method=[B] > test2 >>> class ValhallaTest${0=B} > Specializing method ValhallaTest$test1${0=C}.test1()V with class=[] > and method=[C] > Specializing ValhallaTest${0=C}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=C}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=C} > Specializing method ValhallaTest$test2${0=C}.test2()V with class=[] > and method=[C] > test2 >>> class ValhallaTest${0=C} > Specializing method ValhallaTest$test1${0=S}.test1()V with class=[] > and method=[S] > Specializing ValhallaTest${0=S}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=S}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=S} > Specializing method ValhallaTest$test2${0=S}.test2()V with class=[] > and method=[S] > test2 >>> class ValhallaTest${0=S} > Specializing method ValhallaTest$test1${0=I}.test1()V with class=[] > and method=[I] > Specializing ValhallaTest${0=I}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=I}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=I} > Specializing method ValhallaTest$test2${0=I}.test2()V with class=[] > and method=[I] > test2 >>> class ValhallaTest${0=I} > Specializing method ValhallaTest$test1${0=J}.test1()V with class=[] > and method=[J] > Specializing ValhallaTest${0=J}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=J}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=J} > Specializing method ValhallaTest$test2${0=J}.test2()V with class=[] > and method=[J] > test2 >>> class ValhallaTest${0=J} > Specializing method ValhallaTest$test1${0=F}.test1()V with class=[] > and method=[F] > Specializing ValhallaTest${0=F}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=F}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=F} > Specializing method ValhallaTest$test2${0=F}.test2()V with class=[] > and method=[F] > test2 >>> class ValhallaTest${0=F} > Specializing method ValhallaTest$test1${0=D}.test1()V with class=[] > and method=[D] > Specializing ValhallaTest${0=D}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=D}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=D} > Specializing method ValhallaTest$test2${0=D}.test2()V with class=[] > and method=[D] > test2 >>> class ValhallaTest${0=D} > test1 >>> class ValhallaTest > Specializing method ValhallaTest$test2${}.test2()V with class=[] and > method=[TT;] > Exception in thread "main" java.lang.NoClassDefFoundError: > ValhallaTest at ValhallaTest$test2${}/401625763.test2(ValhallaTest.java:7) > at ValhallaTest.test1(ValhallaTest.java:15) > at ValhallaTest.main(ValhallaTest.java:27) > Caused by: java.lang.ClassNotFoundException: ValhallaTest at java.net.URLClassLoader.findClass(URLClassLoader.java:451) > at java.lang.ClassLoader.loadClass(ClassLoader.java:426) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317) > at java.lang.ClassLoader.loadClass(ClassLoader.java:359) > ... 3 more > > > Within tes1() method, everything works fine, but cascading call to > test2() fails in line 7 where ValhallaTest.class is evaluated, but > only for reference type T. > > Note that there's no specialization for test1() being triggered for > T=reference (which is understandable), but there is a specialization > being triggered for cascaded call to test2() in this case. In my > superficial understanding a call to: > > ValhallaTest.test1(); > > ...is implemented as invokestatic, but a call from test1() to: > > ValhallaTest.test2(); > > ...is implemented as invokedynamic. So in case such call comes from > non-specialized test1(), it should be wired directly to the > nonspecialized test2() method, right? > > Or, couldn't the call from nonspecialized test1() to .test2() also > be implemented as invokestatic and be converted to invokedynamic only > by specialization? > > Regards, Peter > From peter.levart at gmail.com Thu Jan 8 11:23:38 2015 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 08 Jan 2015 12:23:38 +0100 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: References: Message-ID: <54AE68BA.3000403@gmail.com> On 01/08/2015 12:33 AM, Vitaly Davidovich wrote: >> For value-types: >> x eq y --> (x == y) > > You mean for primitives? Custom value types won't work with == as there's > no operator overloading. > > Getting back to the null aspect though, here's my take ... > > I think we should not introduce a new operator such as eq when we already > have a "universal" equals. What you really want to say is something like > this: > > void foo(T t) { > if (T.default.equals(t)) > System.out.println("got default/null"); > else > System.out.println("got non-default/null"); > } > > Now, the obvious problem here is that this NPEs on reference types since > T.default yields a null and you can't reverse the comparison because t may > be null. So, what you'd like here is similar to what .NET does. Introduce > an IEqualityComparer-like JDK interface, and provide a way to obtain a > "default" comparer for a given type. In .NET, there's a generic class > called EqualityComparer, which has a static property called Default (e.g. > EqualityComparer.Default). That internally (in static constructor) > figures out, based on inspecting T's capabilities and type, which is the > default equality comparer and then hands out that sole instance on each > Default access. Ultimately, what you want here is instead of doing > T.default.equals(t), you want some other class to define an " > boolean equals(T x, T y)" which can internally handle one side being null > (for ref types). This sounds similar to the following: http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/util/AnyHelper.java This can be used as a temporary work-around for lack of such features in current prototype. Peter > > tldr; well behaved classes already should be handling null inputs in their > equals() impl, so we should find a way to reuse that instead of introducing > a new operator/keyword. > > > On Wed, Jan 7, 2015 at 6:07 PM, Thomas W wrote: > >> Hi Simon, Vitaly, people, >> >> Lots of people have been talking about adapting Collections for 'any' >> type. Going forwards, I do not believe we should be writing code that >> explicitly uses/ or checks for nulls (though we may be backwards-compatible >> for a while) any more than we should be writing or calling a Duck.woof() >> method. >> >> To answering a few questions: >> >>> a) What are the actual semantics? >> For ref-types: >> x eq y --> (x != null && x.equals(y)) >> For value-types: >> x eq y --> (x == y) >> >>> b) How is this different to the approaches already outlines earlier? >> It provides a basic logical building-block (equality check) at the >> language level, in a way that works both for primitives/valuetypes and for >> nullable references. It addresses a very big long-standing pain point in >> Java application development. It avoids writing/ or encouraging >> "Duck.woof()" style-hacks -- null checks -- into a domain that does not >> uniformly support them. >> >>> Why do we need a new special operator? What's wrong with using equals >> () and having the specializer rewrite that for primitives? >>> What's wrong with using equals() and having the specializer rewrite >> that for primitives? For other types >>> (refs and custom value types), it should just delegate to the real >> equals. >> >> To have a proper logical equals in one operator, which is independent of >> implementation & null-safe for reftypes. >> >> What's wrong is writing null-checks & nulls, in a domain where nulls may >> not exist. >> We are not dealing with Dog, we are dealing with Animal and there should >> be no 'woof()' method. >> >>> What's wrong with using equals() and having the specializer rewrite >> that for primitives? For other types >>> (refs and custom value types), it should just delegate to the real >> equals. >> >> What's wrong with an operator that logically has exactly the right >> meaning, without exposing details that don't exist in the domain? >> It also provides a vast benefit for application programmers, as this kind >> of requirement is ridiculously common in hundreds of millions of lines of >> application code. >> >> >> Regards, >> Thomas Whitmore >> From maurizio.cimadamore at oracle.com Thu Jan 8 11:35:40 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 08 Jan 2015 11:35:40 +0000 Subject: Why JVM can not type infer type params but javac can do? In-Reply-To: References: Message-ID: <54AE6B8C.708@oracle.com> On 08/01/15 08:54, Ali Ebrahimi wrote: > Hi, maybe this is stupid question, but want to ask and what is requirements > for this? > > Array instantiation sample: > public T[] newArray(int){ > return new T[0]; > } > > even for non-any T: > public T[] newArray(int){ > return new T[0]; > } > > With thousands number of optimization and inlining mechanics equipped in > jvm, What is the reasoning behind not allowing to jvm to be aware of > generics info emitted in classfilles? > > Why jvm should not able do following inlining? > > String[] strs = newArray(int) => String[] strs = new String[0]; > I mean even in jitted code. > > I think some time ago I read some papers in similar area. Well, you probably don't want the types of your arrays to change depending on mere optimizations; the type of the array is important - if it's Object[] then you can stuff anything you like into it and the VM will not bark; but if it's String[] then the VM will be much more paranoid about it. And you can't have a program that runs correctly or fails depending on which kind of optimizations have been applied. So, the question maybe is: why doesn't the VM do all the type lifting that is required in order to figure out that T means String? Well, one good reason is that answering such question is not simple and it involves a lot of type-system machinery - which the compiler already has. If we were to replicate this logic in the VM, it will turn out to be a maintenaince pain - as we will have then to worry to keep javac and the JVM type-system routines in sync. Since javac has all the info (as your email subject suggests), it is way more straightforward to just add an 'hint' that tells the VM how to reconstruct the type of T. Also, other languages might have different ideas of how inference should be performed and find the java-like behavior too strict/or simply insufficient. Maurizio From paul.sandoz at oracle.com Thu Jan 8 11:46:51 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 8 Jan 2015 12:46:51 +0100 Subject: Why JVM can not type infer type params but javac can do? In-Reply-To: References: Message-ID: On Jan 8, 2015, at 9:54 AM, Ali Ebrahimi wrote: > Hi, maybe this is stupid question, but want to ask and what is requirements > for this? > > Array instantiation sample: > public T[] newArray(int){ > return new T[0]; > } > > even for non-any T: > public T[] newArray(int){ > return new T[0]; > } > > With thousands number of optimization and inlining mechanics equipped in > jvm, What is the reasoning behind not allowing to jvm to be aware of > generics info emitted in classfilles? > > Why jvm should not able do following inlining? > > String[] strs = newArray(int) => String[] strs = new String[0]; > I mean even in jitted code. > > I think some time ago I read some papers in similar area. Try compiling the following: public class A { static T[] array(int s) { return new T[s]; } static int[] intArray(int s) { return array(s); } public static void main(String[] args) throws Throwable { int sum = 0; for (int i = 0; i < 1000000; i++) { int[] ia = intArray(0); sum += ia.length; } System.out.println(sum); } } Then run with: java -XX:-TieredCompilation -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining -Dvalhalla.dumpProxyClasses=. A You should see an inline trace such as: @ 11 A::intArray (7 bytes) inline (hot) @ 1 java.lang.invoke.LambdaForm$MH/1389133897::linkToTargetMethod (9 bytes) force inline by annotation @ 5 java.lang.invoke.LambdaForm$DMH/321001045::invokeStatic_I_L (14 bytes) force inline by annotation @ 1 java.lang.invoke.DirectMethodHandle::internalMemberName (8 bytes) force inline by annotation @ 10 A$array${0=I}/1922154895::array (4 bytes) inline (hot) Look at the specialized class holding the specialized the method: javap -p -v A\$array\$\{0\=I\}.1922154895.class and you will see: static int[] array(int); descriptor: (I)[I flags: ACC_STATIC Code: stack=1, locals=1, args_size=1 0: iload_0 1: newarray int 3: areturn It all inlines rather nicely. At the moment you will get a ClassCastException if you use a ref, such as String, since javac currently compiles the array method to: static T[] array(int); descriptor: (I)[Ljava/lang/Object; flags: ACC_STATIC Code: stack=1, locals=1, args_size=1 0: iload_0 1: anewarray #2 // class java/lang/Object 4: areturn LineNumberTable: line 4: 0 BytecodeMapping: Code_idx Signature 1: [TT; Signature: #18 // (I)[TT; TypeVariablesMap: LA;::array(I)[Ljava/lang/Object;: Tvar Flags Erased bound T [ANY] Ljava/lang/Object; That method is not specalized for refs (nor is T reified in such cases). There are some nasty edge cases w.r.t. arrays that need to be solved at some point. Paul. From palo.marton at gmail.com Thu Jan 8 12:03:25 2015 From: palo.marton at gmail.com (Palo Marton) Date: Thu, 8 Jan 2015 13:03:25 +0100 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: References: Message-ID: Thank you for response. I have read that thread. But apart from first few messages it seems to deal with something else - mutable types. My idea with rewriting String as value is not about changing meaning of java.lang.String. That identifier should still mean heap object with identity, null, etc... The idea is to give a new name for string value, e.g. java.lang.string (lowercase s), or just "string". So "String" will be boxed version of "string" and all old pre-value-types code will use only boxed objects as before and there will be no semantic change. In new code you will use "string" in most cases, but you can use also "String" in places where you need it. >From what I have read there are no plans to support something like that. Am I right? Reason why I am asking this is that some time ago I was thinking about value types in java and how they should be implemented in JVM. My idea about how to implement it in JVM was somehow different, but it can support this old-code / new-code compatibility. On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < richard.warburton at gmail.com> wrote: > Hi, > > Will it be possible (within valhalla) to rewrite java String as value type >> in JDK and use it in all places in JDK where it makes sense? (eg. >> Class.forName(string), Object.toString(), Integer.parseInt(string). ?) >> >> Or more general question: >> >> There are already plenty of ?value objects? in existing java code. In JDK >> we have, String, File, Point2D,... Also many people have already declared >> their own class Point { final x,y; } and similar objects. Once we move to >> value types in java, will it be possible to rewrite these to value types >> without breaking compatibility with old pre-value-types code? E.g. if I >> change my Point object to value type in my library, can this new library >> (jar) still be used in other projects that were written before this >> change? >> And without need to recompile those other projects? > > > See this thread: > http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-January/000546.html > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > > From forax at univ-mlv.fr Thu Jan 8 12:19:09 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 08 Jan 2015 13:19:09 +0100 Subject: Why JVM can not type infer type params but javac can do? In-Reply-To: References: Message-ID: <54AE75BD.1080107@univ-mlv.fr> Do you really want to enhance a simple VM checkcast to check subtyping relationship between wildcards at runtime ? R?mi On 01/08/2015 09:54 AM, Ali Ebrahimi wrote: > Hi, maybe this is stupid question, but want to ask and what is requirements > for this? > > Array instantiation sample: > public T[] newArray(int){ > return new T[0]; > } > > even for non-any T: > public T[] newArray(int){ > return new T[0]; > } > > With thousands number of optimization and inlining mechanics equipped in > jvm, What is the reasoning behind not allowing to jvm to be aware of > generics info emitted in classfilles? > > Why jvm should not able do following inlining? > > String[] strs = newArray(int) => String[] strs = new String[0]; > I mean even in jitted code. > > I think some time ago I read some papers in similar area. From ali.ebrahimi1781 at gmail.com Thu Jan 8 12:32:14 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Thu, 8 Jan 2015 16:02:14 +0330 Subject: Why JVM can not type infer type params but javac can do? In-Reply-To: <54AE75BD.1080107@univ-mlv.fr> References: <54AE75BD.1080107@univ-mlv.fr> Message-ID: I mean we can do better job than today by some hints and already unused info by VM. One first step: disallowing raw types by java9/10. And some form of http://www.google.com/patents/US7810077 On Thu, Jan 8, 2015 at 3:49 PM, Remi Forax wrote: > Do you really want to enhance a simple VM checkcast to check subtyping > relationship between wildcards at runtime ? > > R?mi > > > On 01/08/2015 09:54 AM, Ali Ebrahimi wrote: > >> Hi, maybe this is stupid question, but want to ask and what is >> requirements >> for this? >> >> Array instantiation sample: >> public T[] newArray(int){ >> return new T[0]; >> } >> >> even for non-any T: >> public T[] newArray(int){ >> return new T[0]; >> } >> >> With thousands number of optimization and inlining mechanics equipped in >> jvm, What is the reasoning behind not allowing to jvm to be aware of >> generics info emitted in classfilles? >> >> Why jvm should not able do following inlining? >> >> String[] strs = newArray(int) => String[] strs = new String[0]; >> I mean even in jitted code. >> >> I think some time ago I read some papers in similar area. >> > > -- Best Regards, Ali Ebrahimi From vitalyd at gmail.com Thu Jan 8 13:50:28 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 08:50:28 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: References: Message-ID: Why do you want string to be value type? What problem (s) will that address? Sent from my phone On Jan 8, 2015 7:04 AM, "Palo Marton" wrote: > Thank you for response. I have read that thread. But apart from first few > messages it seems to deal with something else - mutable types. > > My idea with rewriting String as value is not about changing meaning of > java.lang.String. That identifier should still mean heap object with > identity, null, etc... The idea is to give a new name for string value, > e.g. java.lang.string (lowercase s), or just "string". So "String" will be > boxed version of "string" and all old pre-value-types code will use only > boxed objects as before and there will be no semantic change. In new code > you will use "string" in most cases, but you can use also "String" in > places where you need it. > > From what I have read there are no plans to support something like that. Am > I right? > > Reason why I am asking this is that some time ago I was thinking about > value types in java and how they should be implemented in JVM. My idea > about how to implement it in JVM was somehow different, but it can support > this old-code / new-code compatibility. > > On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < > richard.warburton at gmail.com> wrote: > > > Hi, > > > > Will it be possible (within valhalla) to rewrite java String as value > type > >> in JDK and use it in all places in JDK where it makes sense? (eg. > >> Class.forName(string), Object.toString(), Integer.parseInt(string). ?) > >> > >> Or more general question: > >> > >> There are already plenty of ?value objects? in existing java code. In > JDK > >> we have, String, File, Point2D,... Also many people have already > declared > >> their own class Point { final x,y; } and similar objects. Once we move > to > >> value types in java, will it be possible to rewrite these to value types > >> without breaking compatibility with old pre-value-types code? E.g. if I > >> change my Point object to value type in my library, can this new library > >> (jar) still be used in other projects that were written before this > >> change? > >> And without need to recompile those other projects? > > > > > > See this thread: > > > http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-January/000546.html > > > > regards, > > > > Richard Warburton > > > > http://insightfullogic.com > > @RichardWarburto > > > > > From peter.levart at gmail.com Thu Jan 8 13:54:47 2015 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 08 Jan 2015 14:54:47 +0100 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: References: Message-ID: <54AE8C27.50400@gmail.com> On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: > Why do you want string to be value type? What problem (s) will that > address? String as a value type would eliminate one indirection. If would effectively become an immutable char[] with a bunch of operations. Peter > > Sent from my phone > On Jan 8, 2015 7:04 AM, "Palo Marton" wrote: > >> Thank you for response. I have read that thread. But apart from first few >> messages it seems to deal with something else - mutable types. >> >> My idea with rewriting String as value is not about changing meaning of >> java.lang.String. That identifier should still mean heap object with >> identity, null, etc... The idea is to give a new name for string value, >> e.g. java.lang.string (lowercase s), or just "string". So "String" will be >> boxed version of "string" and all old pre-value-types code will use only >> boxed objects as before and there will be no semantic change. In new code >> you will use "string" in most cases, but you can use also "String" in >> places where you need it. >> >> From what I have read there are no plans to support something like that. Am >> I right? >> >> Reason why I am asking this is that some time ago I was thinking about >> value types in java and how they should be implemented in JVM. My idea >> about how to implement it in JVM was somehow different, but it can support >> this old-code / new-code compatibility. >> >> On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < >> richard.warburton at gmail.com> wrote: >> >>> Hi, >>> >>> Will it be possible (within valhalla) to rewrite java String as value >> type >>>> in JDK and use it in all places in JDK where it makes sense? (eg. >>>> Class.forName(string), Object.toString(), Integer.parseInt(string). ?) >>>> >>>> Or more general question: >>>> >>>> There are already plenty of ?value objects? in existing java code. In >> JDK >>>> we have, String, File, Point2D,... Also many people have already >> declared >>>> their own class Point { final x,y; } and similar objects. Once we move >> to >>>> value types in java, will it be possible to rewrite these to value types >>>> without breaking compatibility with old pre-value-types code? E.g. if I >>>> change my Point object to value type in my library, can this new library >>>> (jar) still be used in other projects that were written before this >>>> change? >>>> And without need to recompile those other projects? >>> >>> See this thread: >>> >> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-January/000546.html >>> regards, >>> >>> Richard Warburton >>> >>> http://insightfullogic.com >>> @RichardWarburto >>> >>> From peter.levart at gmail.com Thu Jan 8 13:57:32 2015 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 08 Jan 2015 14:57:32 +0100 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: <54AE8C27.50400@gmail.com> References: <54AE8C27.50400@gmail.com> Message-ID: <54AE8CCC.9050100@gmail.com> On 01/08/2015 02:54 PM, Peter Levart wrote: > On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: >> Why do you want string to be value type? What problem (s) will that >> address? > > String as a value type would eliminate one indirection. If would > effectively become an immutable char[] with a bunch of operations. > The only problem being the 'hash' field, which is mutable (cache). But that could be encoded in char[] as 1st two slots, or computed uprfont in constructor. > Peter > >> >> Sent from my phone >> On Jan 8, 2015 7:04 AM, "Palo Marton" wrote: >> >>> Thank you for response. I have read that thread. But apart from >>> first few >>> messages it seems to deal with something else - mutable types. >>> >>> My idea with rewriting String as value is not about changing meaning of >>> java.lang.String. That identifier should still mean heap object with >>> identity, null, etc... The idea is to give a new name for string value, >>> e.g. java.lang.string (lowercase s), or just "string". So "String" >>> will be >>> boxed version of "string" and all old pre-value-types code will use >>> only >>> boxed objects as before and there will be no semantic change. In new >>> code >>> you will use "string" in most cases, but you can use also "String" in >>> places where you need it. >>> >>> From what I have read there are no plans to support something like >>> that. Am >>> I right? >>> >>> Reason why I am asking this is that some time ago I was thinking about >>> value types in java and how they should be implemented in JVM. My idea >>> about how to implement it in JVM was somehow different, but it can >>> support >>> this old-code / new-code compatibility. >>> >>> On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < >>> richard.warburton at gmail.com> wrote: >>> >>>> Hi, >>>> >>>> Will it be possible (within valhalla) to rewrite java String as value >>> type >>>>> in JDK and use it in all places in JDK where it makes sense? (eg. >>>>> Class.forName(string), Object.toString(), >>>>> Integer.parseInt(string). ?) >>>>> >>>>> Or more general question: >>>>> >>>>> There are already plenty of ?value objects? in existing java code. In >>> JDK >>>>> we have, String, File, Point2D,... Also many people have already >>> declared >>>>> their own class Point { final x,y; } and similar objects. Once we >>>>> move >>> to >>>>> value types in java, will it be possible to rewrite these to value >>>>> types >>>>> without breaking compatibility with old pre-value-types code? E.g. >>>>> if I >>>>> change my Point object to value type in my library, can this new >>>>> library >>>>> (jar) still be used in other projects that were written before this >>>>> change? >>>>> And without need to recompile those other projects? >>>> >>>> See this thread: >>>> >>> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-January/000546.html >>> >>>> regards, >>>> >>>> Richard Warburton >>>> >>>> http://insightfullogic.com >>>> @RichardWarburto >>>> >>>> > From vitalyd at gmail.com Thu Jan 8 14:02:57 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 09:02:57 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AE8C27.50400@gmail.com> References: <54AE8C27.50400@gmail.com> Message-ID: I had a suspicion this would be the gist. However, I think better solution is to simply change string to have inline storage of the data, so no indirection to the char[]. Ultimately, the string data will have to live on the heap one way or another, but it'd be nice if String had that data inlined into its storage so when you load address of string you get the data at a small fixed offset from that address. Sent from my phone On Jan 8, 2015 8:54 AM, "Peter Levart" wrote: > On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: > >> Why do you want string to be value type? What problem (s) will that >> address? >> > > String as a value type would eliminate one indirection. If would > effectively become an immutable char[] with a bunch of operations. > > Peter > > >> Sent from my phone >> On Jan 8, 2015 7:04 AM, "Palo Marton" wrote: >> >> Thank you for response. I have read that thread. But apart from first few >>> messages it seems to deal with something else - mutable types. >>> >>> My idea with rewriting String as value is not about changing meaning of >>> java.lang.String. That identifier should still mean heap object with >>> identity, null, etc... The idea is to give a new name for string value, >>> e.g. java.lang.string (lowercase s), or just "string". So "String" will >>> be >>> boxed version of "string" and all old pre-value-types code will use only >>> boxed objects as before and there will be no semantic change. In new code >>> you will use "string" in most cases, but you can use also "String" in >>> places where you need it. >>> >>> From what I have read there are no plans to support something like >>> that. Am >>> I right? >>> >>> Reason why I am asking this is that some time ago I was thinking about >>> value types in java and how they should be implemented in JVM. My idea >>> about how to implement it in JVM was somehow different, but it can >>> support >>> this old-code / new-code compatibility. >>> >>> On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < >>> richard.warburton at gmail.com> wrote: >>> >>> Hi, >>>> >>>> Will it be possible (within valhalla) to rewrite java String as value >>>> >>> type >>> >>>> in JDK and use it in all places in JDK where it makes sense? (eg. >>>>> Class.forName(string), Object.toString(), Integer.parseInt(string). ?) >>>>> >>>>> Or more general question: >>>>> >>>>> There are already plenty of ?value objects? in existing java code. In >>>>> >>>> JDK >>> >>>> we have, String, File, Point2D,... Also many people have already >>>>> >>>> declared >>> >>>> their own class Point { final x,y; } and similar objects. Once we move >>>>> >>>> to >>> >>>> value types in java, will it be possible to rewrite these to value types >>>>> without breaking compatibility with old pre-value-types code? E.g. if I >>>>> change my Point object to value type in my library, can this new >>>>> library >>>>> (jar) still be used in other projects that were written before this >>>>> change? >>>>> And without need to recompile those other projects? >>>>> >>>> >>>> See this thread: >>>> >>>> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- >>> January/000546.html >>> >>>> regards, >>>> >>>> Richard Warburton >>>> >>>> http://insightfullogic.com >>>> @RichardWarburto >>>> >>>> >>>> > From palo.marton at gmail.com Thu Jan 8 14:02:42 2015 From: palo.marton at gmail.com (Palo Marton) Date: Thu, 8 Jan 2015 15:02:42 +0100 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AE8CCC.9050100@gmail.com> References: <54AE8C27.50400@gmail.com> <54AE8CCC.9050100@gmail.com> Message-ID: Yes, it could be stored at the beginning of the array (there are 2 hash codes in current string, so it will need 4 slots). Benefits of having value string are: - faster access because of removed indirection - save memory, as it removes String box object from heap Drawbacks: - adds one level of indirection for accessing hash code when using boxed String On Thu, Jan 8, 2015 at 2:57 PM, Peter Levart wrote: > On 01/08/2015 02:54 PM, Peter Levart wrote: > >> On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: >> >>> Why do you want string to be value type? What problem (s) will that >>> address? >>> >> >> String as a value type would eliminate one indirection. If would >> effectively become an immutable char[] with a bunch of operations. >> >> > The only problem being the 'hash' field, which is mutable (cache). But > that could be encoded in char[] as 1st two slots, or computed uprfont in > constructor. > > > Peter >> >> >>> Sent from my phone >>> On Jan 8, 2015 7:04 AM, "Palo Marton" wrote: >>> >>> Thank you for response. I have read that thread. But apart from first >>>> few >>>> messages it seems to deal with something else - mutable types. >>>> >>>> My idea with rewriting String as value is not about changing meaning of >>>> java.lang.String. That identifier should still mean heap object with >>>> identity, null, etc... The idea is to give a new name for string value, >>>> e.g. java.lang.string (lowercase s), or just "string". So "String" will >>>> be >>>> boxed version of "string" and all old pre-value-types code will use only >>>> boxed objects as before and there will be no semantic change. In new >>>> code >>>> you will use "string" in most cases, but you can use also "String" in >>>> places where you need it. >>>> >>>> From what I have read there are no plans to support something like >>>> that. Am >>>> I right? >>>> >>>> Reason why I am asking this is that some time ago I was thinking about >>>> value types in java and how they should be implemented in JVM. My idea >>>> about how to implement it in JVM was somehow different, but it can >>>> support >>>> this old-code / new-code compatibility. >>>> >>>> On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < >>>> richard.warburton at gmail.com> wrote: >>>> >>>> Hi, >>>>> >>>>> Will it be possible (within valhalla) to rewrite java String as value >>>>> >>>> type >>>> >>>>> in JDK and use it in all places in JDK where it makes sense? (eg. >>>>>> Class.forName(string), Object.toString(), Integer.parseInt(string). ?) >>>>>> >>>>>> Or more general question: >>>>>> >>>>>> There are already plenty of ?value objects? in existing java code. In >>>>>> >>>>> JDK >>>> >>>>> we have, String, File, Point2D,... Also many people have already >>>>>> >>>>> declared >>>> >>>>> their own class Point { final x,y; } and similar objects. Once we move >>>>>> >>>>> to >>>> >>>>> value types in java, will it be possible to rewrite these to value >>>>>> types >>>>>> without breaking compatibility with old pre-value-types code? E.g. if >>>>>> I >>>>>> change my Point object to value type in my library, can this new >>>>>> library >>>>>> (jar) still be used in other projects that were written before this >>>>>> change? >>>>>> And without need to recompile those other projects? >>>>>> >>>>> >>>>> See this thread: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- >>>> January/000546.html >>>> >>>>> regards, >>>>> >>>>> Richard Warburton >>>>> >>>>> http://insightfullogic.com >>>>> @RichardWarburto >>>>> >>>>> >>>>> >> > From vitalyd at gmail.com Thu Jan 8 14:16:35 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 09:16:35 -0500 Subject: 'Equals' in any-typed code: VERY BIG PROPOSAL In-Reply-To: References: Message-ID: So regarding terseness/shortness, you could use java.lang.Objects.equal () static method; static import it and you have: equals (car.getColor (), someColor). If an version is added, that seems pretty good to me. Sent from my phone On Jan 7, 2015 11:58 PM, "Thomas W" wrote: > Hi Vitaly, > > Thanks for your interest in this area. > > > Why do you need language feature for #2? What's wrong with calling a lib > > method that wraps this up for you? > > Collections are built on 'eq' and 'hashCode'. It's a fundamental basic > operator on values, and we're now moving to support values. So, change is > here & we have an opportunity to get it right. > > You are correct, a static helper/ or EqualsStrategy can also achieve this > for collections. > > But for the language more widely & application programming in general, an > 'eq' operator would be a great help. As well as being a very simple & clean > solution for specializable Collections. > > Here's a demonstration of length: > > if (car.getColor() != null && car.getColor().equals( someColor))) { > if (ObjectUtils.equals( car.getColor(), someColor)) { > if (car.getColor() eq someColor)) { > > There is a massive difference in the amount of typing, and in clarity. > > > Changing the language is a high barrier to entry > > Language changes are necessary to have any kind of syntax for > specialization, that is not a rabidly horrible bunch of hacks. So I think > we are over that barrier already. > > Millions will cheer & thanks you! > > > and personally, it seems like eq is just shorthand for a generic method > that can deal with this > > just as well. > > Logically, 'equals' is an _operator_ not a _method_. Now is the time to > get this right. Alongside specialization, the Java 10 release will be > marked with fireworks, champagne & celebration. > > > one language change I'd appreciate is the elvis operator since you can't > emulate that via lib. > > Really, we can only consider the most concise & conservative set of > operators -- that will stand on their own, solve the very biggest bugbear, > and not be subject to "scope creep". Having a clear boundary for features > is very important. > > I like the Elvis operator too -- and case-insensitive equals for strings > -- but I'm putting forward a bounded proposal, prompted by the need for > 'equals' in Collections. I want to start with that; it's also the most > common form in Java application development. > > > Regards, > Thomas > From palo.marton at gmail.com Thu Jan 8 14:17:32 2015 From: palo.marton at gmail.com (Palo Marton) Date: Thu, 8 Jan 2015 15:17:32 +0100 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: References: <54AE8C27.50400@gmail.com> Message-ID: My question is not just about String but about all existing value types. There are many of them in current java code (both JDK and user code). Current proposal seems to ignore them. I think that the original idea of VT: "Codes like a class, works like an int!" can be changed to: "Codes like a class, works like class, but can be stored and passed as int!" as the problem with current value objects (like String) is not with the fact that they are objects, but in how they are stored in memory and passed between functions. On Thu, Jan 8, 2015 at 3:02 PM, Vitaly Davidovich wrote: > I had a suspicion this would be the gist. However, I think better > solution is to simply change string to have inline storage of the data, so > no indirection to the char[]. Ultimately, the string data will have to > live on the heap one way or another, but it'd be nice if String had that > data inlined into its storage so when you load address of string you get > the data at a small fixed offset from that address. > > Sent from my phone > On Jan 8, 2015 8:54 AM, "Peter Levart" wrote: > >> On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: >> >>> Why do you want string to be value type? What problem (s) will that >>> address? >>> >> >> String as a value type would eliminate one indirection. If would >> effectively become an immutable char[] with a bunch of operations. >> >> Peter >> >> >>> Sent from my phone >>> On Jan 8, 2015 7:04 AM, "Palo Marton" wrote: >>> >>> Thank you for response. I have read that thread. But apart from first >>>> few >>>> messages it seems to deal with something else - mutable types. >>>> >>>> My idea with rewriting String as value is not about changing meaning of >>>> java.lang.String. That identifier should still mean heap object with >>>> identity, null, etc... The idea is to give a new name for string value, >>>> e.g. java.lang.string (lowercase s), or just "string". So "String" will >>>> be >>>> boxed version of "string" and all old pre-value-types code will use only >>>> boxed objects as before and there will be no semantic change. In new >>>> code >>>> you will use "string" in most cases, but you can use also "String" in >>>> places where you need it. >>>> >>>> From what I have read there are no plans to support something like >>>> that. Am >>>> I right? >>>> >>>> Reason why I am asking this is that some time ago I was thinking about >>>> value types in java and how they should be implemented in JVM. My idea >>>> about how to implement it in JVM was somehow different, but it can >>>> support >>>> this old-code / new-code compatibility. >>>> >>>> On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < >>>> richard.warburton at gmail.com> wrote: >>>> >>>> Hi, >>>>> >>>>> Will it be possible (within valhalla) to rewrite java String as value >>>>> >>>> type >>>> >>>>> in JDK and use it in all places in JDK where it makes sense? (eg. >>>>>> Class.forName(string), Object.toString(), Integer.parseInt(string). ?) >>>>>> >>>>>> Or more general question: >>>>>> >>>>>> There are already plenty of ?value objects? in existing java code. In >>>>>> >>>>> JDK >>>> >>>>> we have, String, File, Point2D,... Also many people have already >>>>>> >>>>> declared >>>> >>>>> their own class Point { final x,y; } and similar objects. Once we move >>>>>> >>>>> to >>>> >>>>> value types in java, will it be possible to rewrite these to value >>>>>> types >>>>>> without breaking compatibility with old pre-value-types code? E.g. if >>>>>> I >>>>>> change my Point object to value type in my library, can this new >>>>>> library >>>>>> (jar) still be used in other projects that were written before this >>>>>> change? >>>>>> And without need to recompile those other projects? >>>>>> >>>>> >>>>> See this thread: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- >>>> January/000546.html >>>> >>>>> regards, >>>>> >>>>> Richard Warburton >>>>> >>>>> http://insightfullogic.com >>>>> @RichardWarburto >>>>> >>>>> >>>>> >> From vitalyd at gmail.com Thu Jan 8 14:24:38 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 09:24:38 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: References: <54AE8C27.50400@gmail.com> Message-ID: I don't think this has been ignored, certainly not on the recent traffic on this mailing list. I think the short answer to the "can existing classes that are value objects be migrated to value types with no client change" is probably no. The reason, or one of them, is that there's no way to know (without analyzing all usages) whether client code uses object features of said object (e.g. uses reference identity comparison, locks on an instance, etc). In your rephrasing, "works like a class" implies identity of the object and can be synchronized on. That's not what VT is of course. Sent from my phone On Jan 8, 2015 9:17 AM, "Palo Marton" wrote: > My question is not just about String but about all existing value types. > There are many of them in current java code (both JDK and user code). > Current proposal seems to ignore them. > > I think that the original idea of VT: > > "Codes like a class, works like an int!" > > can be changed to: > > "Codes like a class, works like class, but can be stored and passed as > int!" > > as the problem with current value objects (like String) is not with the > fact that they are objects, but in how they are stored in memory and passed > between functions. > > > > On Thu, Jan 8, 2015 at 3:02 PM, Vitaly Davidovich > wrote: > >> I had a suspicion this would be the gist. However, I think better >> solution is to simply change string to have inline storage of the data, so >> no indirection to the char[]. Ultimately, the string data will have to >> live on the heap one way or another, but it'd be nice if String had that >> data inlined into its storage so when you load address of string you get >> the data at a small fixed offset from that address. >> >> Sent from my phone >> On Jan 8, 2015 8:54 AM, "Peter Levart" wrote: >> >>> On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: >>> >>>> Why do you want string to be value type? What problem (s) will that >>>> address? >>>> >>> >>> String as a value type would eliminate one indirection. If would >>> effectively become an immutable char[] with a bunch of operations. >>> >>> Peter >>> >>> >>>> Sent from my phone >>>> On Jan 8, 2015 7:04 AM, "Palo Marton" wrote: >>>> >>>> Thank you for response. I have read that thread. But apart from first >>>>> few >>>>> messages it seems to deal with something else - mutable types. >>>>> >>>>> My idea with rewriting String as value is not about changing meaning of >>>>> java.lang.String. That identifier should still mean heap object with >>>>> identity, null, etc... The idea is to give a new name for string value, >>>>> e.g. java.lang.string (lowercase s), or just "string". So "String" >>>>> will be >>>>> boxed version of "string" and all old pre-value-types code will use >>>>> only >>>>> boxed objects as before and there will be no semantic change. In new >>>>> code >>>>> you will use "string" in most cases, but you can use also "String" in >>>>> places where you need it. >>>>> >>>>> From what I have read there are no plans to support something like >>>>> that. Am >>>>> I right? >>>>> >>>>> Reason why I am asking this is that some time ago I was thinking about >>>>> value types in java and how they should be implemented in JVM. My idea >>>>> about how to implement it in JVM was somehow different, but it can >>>>> support >>>>> this old-code / new-code compatibility. >>>>> >>>>> On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < >>>>> richard.warburton at gmail.com> wrote: >>>>> >>>>> Hi, >>>>>> >>>>>> Will it be possible (within valhalla) to rewrite java String as value >>>>>> >>>>> type >>>>> >>>>>> in JDK and use it in all places in JDK where it makes sense? (eg. >>>>>>> Class.forName(string), Object.toString(), Integer.parseInt(string). >>>>>>> ?) >>>>>>> >>>>>>> Or more general question: >>>>>>> >>>>>>> There are already plenty of ?value objects? in existing java code. In >>>>>>> >>>>>> JDK >>>>> >>>>>> we have, String, File, Point2D,... Also many people have already >>>>>>> >>>>>> declared >>>>> >>>>>> their own class Point { final x,y; } and similar objects. Once we move >>>>>>> >>>>>> to >>>>> >>>>>> value types in java, will it be possible to rewrite these to value >>>>>>> types >>>>>>> without breaking compatibility with old pre-value-types code? E.g. >>>>>>> if I >>>>>>> change my Point object to value type in my library, can this new >>>>>>> library >>>>>>> (jar) still be used in other projects that were written before this >>>>>>> change? >>>>>>> And without need to recompile those other projects? >>>>>>> >>>>>> >>>>>> See this thread: >>>>>> >>>>>> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- >>>>> January/000546.html >>>>> >>>>>> regards, >>>>>> >>>>>> Richard Warburton >>>>>> >>>>>> http://insightfullogic.com >>>>>> @RichardWarburto >>>>>> >>>>>> >>>>>> >>> > From palo.marton at gmail.com Thu Jan 8 14:51:48 2015 From: palo.marton at gmail.com (Palo Marton) Date: Thu, 8 Jan 2015 15:51:48 +0100 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: References: <54AE8C27.50400@gmail.com> Message-ID: Yes, I know that you can not simply replace existing value objects with value types when executing old pre-values-types code. Such code must be executed with objects. My idea is something along these lines: Declare value types as normal class (final with final fields), but add secondary "value name" for it, e.g. like this: public final class *String *value *string *{ ... } When in your code you declare variable as "String" than it will work the same as it works now (with identity, lock, null, etc..) But when you declare variable as "string", you basically say that: - I don't care about identity and locking and don't plan to use it - this will never be null. Which gives JVM option to store and pass it "by value" instead of "by reference". On Thu, Jan 8, 2015 at 3:24 PM, Vitaly Davidovich wrote: > I don't think this has been ignored, certainly not on the recent traffic > on this mailing list. > > I think the short answer to the "can existing classes that are value > objects be migrated to value types with no client change" is probably no. > The reason, or one of them, is that there's no way to know (without > analyzing all usages) whether client code uses object features of said > object (e.g. uses reference identity comparison, locks on an instance, etc). > > In your rephrasing, "works like a class" implies identity of the object > and can be synchronized on. That's not what VT is of course. > > Sent from my phone > On Jan 8, 2015 9:17 AM, "Palo Marton" wrote: > >> My question is not just about String but about all existing value types. >> There are many of them in current java code (both JDK and user code). >> Current proposal seems to ignore them. >> >> I think that the original idea of VT: >> >> "Codes like a class, works like an int!" >> >> can be changed to: >> >> "Codes like a class, works like class, but can be stored and passed as >> int!" >> >> as the problem with current value objects (like String) is not with the >> fact that they are objects, but in how they are stored in memory and passed >> between functions. >> >> >> >> On Thu, Jan 8, 2015 at 3:02 PM, Vitaly Davidovich >> wrote: >> >>> I had a suspicion this would be the gist. However, I think better >>> solution is to simply change string to have inline storage of the data, so >>> no indirection to the char[]. Ultimately, the string data will have to >>> live on the heap one way or another, but it'd be nice if String had that >>> data inlined into its storage so when you load address of string you get >>> the data at a small fixed offset from that address. >>> >>> Sent from my phone >>> On Jan 8, 2015 8:54 AM, "Peter Levart" wrote: >>> >>>> On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: >>>> >>>>> Why do you want string to be value type? What problem (s) will that >>>>> address? >>>>> >>>> >>>> String as a value type would eliminate one indirection. If would >>>> effectively become an immutable char[] with a bunch of operations. >>>> >>>> Peter >>>> >>>> >>>>> Sent from my phone >>>>> On Jan 8, 2015 7:04 AM, "Palo Marton" wrote: >>>>> >>>>> Thank you for response. I have read that thread. But apart from first >>>>>> few >>>>>> messages it seems to deal with something else - mutable types. >>>>>> >>>>>> My idea with rewriting String as value is not about changing meaning >>>>>> of >>>>>> java.lang.String. That identifier should still mean heap object with >>>>>> identity, null, etc... The idea is to give a new name for string >>>>>> value, >>>>>> e.g. java.lang.string (lowercase s), or just "string". So "String" >>>>>> will be >>>>>> boxed version of "string" and all old pre-value-types code will use >>>>>> only >>>>>> boxed objects as before and there will be no semantic change. In new >>>>>> code >>>>>> you will use "string" in most cases, but you can use also "String" in >>>>>> places where you need it. >>>>>> >>>>>> From what I have read there are no plans to support something like >>>>>> that. Am >>>>>> I right? >>>>>> >>>>>> Reason why I am asking this is that some time ago I was thinking about >>>>>> value types in java and how they should be implemented in JVM. My idea >>>>>> about how to implement it in JVM was somehow different, but it can >>>>>> support >>>>>> this old-code / new-code compatibility. >>>>>> >>>>>> On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < >>>>>> richard.warburton at gmail.com> wrote: >>>>>> >>>>>> Hi, >>>>>>> >>>>>>> Will it be possible (within valhalla) to rewrite java String as value >>>>>>> >>>>>> type >>>>>> >>>>>>> in JDK and use it in all places in JDK where it makes sense? (eg. >>>>>>>> Class.forName(string), Object.toString(), Integer.parseInt(string). >>>>>>>> ?) >>>>>>>> >>>>>>>> Or more general question: >>>>>>>> >>>>>>>> There are already plenty of ?value objects? in existing java code. >>>>>>>> In >>>>>>>> >>>>>>> JDK >>>>>> >>>>>>> we have, String, File, Point2D,... Also many people have already >>>>>>>> >>>>>>> declared >>>>>> >>>>>>> their own class Point { final x,y; } and similar objects. Once we >>>>>>>> move >>>>>>>> >>>>>>> to >>>>>> >>>>>>> value types in java, will it be possible to rewrite these to value >>>>>>>> types >>>>>>>> without breaking compatibility with old pre-value-types code? E.g. >>>>>>>> if I >>>>>>>> change my Point object to value type in my library, can this new >>>>>>>> library >>>>>>>> (jar) still be used in other projects that were written before this >>>>>>>> change? >>>>>>>> And without need to recompile those other projects? >>>>>>>> >>>>>>> >>>>>>> See this thread: >>>>>>> >>>>>>> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- >>>>>> January/000546.html >>>>>> >>>>>>> regards, >>>>>>> >>>>>>> Richard Warburton >>>>>>> >>>>>>> http://insightfullogic.com >>>>>>> @RichardWarburto >>>>>>> >>>>>>> >>>>>>> >>>> >> From brian.goetz at oracle.com Thu Jan 8 15:45:16 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jan 2015 10:45:16 -0500 Subject: runtime failure when "any T" generic method calls another "any T" generic method In-Reply-To: <54AE3B3C.5080006@gmail.com> References: <54AE3B3C.5080006@gmail.com> Message-ID: <54AEA60C.3000509@oracle.com> Looks like a simple bug where some code path in the specializer is forgetting to check for "is this uninstantiated? if so, ignore it and fall back to erased." We've had a few of these before. Thanks for the bug report. On 1/8/2015 3:09 AM, Peter Levart wrote: > Hi, > > Here's another problem I found: > > > public class ValhallaTest { > > static void test2() { > Class clazz = ValhallaTest.class; // line 7 > System.out.println("test2 >>> " + clazz); > } > > static void test1() { > Class clazz = ValhallaTest.class; > System.out.println("test1 >>> " + clazz); > > ValhallaTest.test2(); > } > > public static void main(String[] args) { > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > } > } > > > Which prints: > > > Specializing method ValhallaTest$test1${0=Z}.test1()V with class=[] and > method=[Z] > Specializing ValhallaTest${0=Z}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=Z}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=Z} > Specializing method ValhallaTest$test2${0=Z}.test2()V with class=[] and > method=[Z] > test2 >>> class ValhallaTest${0=Z} > Specializing method ValhallaTest$test1${0=B}.test1()V with class=[] and > method=[B] > Specializing ValhallaTest${0=B}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=B}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=B} > Specializing method ValhallaTest$test2${0=B}.test2()V with class=[] and > method=[B] > test2 >>> class ValhallaTest${0=B} > Specializing method ValhallaTest$test1${0=C}.test1()V with class=[] and > method=[C] > Specializing ValhallaTest${0=C}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=C}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=C} > Specializing method ValhallaTest$test2${0=C}.test2()V with class=[] and > method=[C] > test2 >>> class ValhallaTest${0=C} > Specializing method ValhallaTest$test1${0=S}.test1()V with class=[] and > method=[S] > Specializing ValhallaTest${0=S}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=S}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=S} > Specializing method ValhallaTest$test2${0=S}.test2()V with class=[] and > method=[S] > test2 >>> class ValhallaTest${0=S} > Specializing method ValhallaTest$test1${0=I}.test1()V with class=[] and > method=[I] > Specializing ValhallaTest${0=I}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=I}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=I} > Specializing method ValhallaTest$test2${0=I}.test2()V with class=[] and > method=[I] > test2 >>> class ValhallaTest${0=I} > Specializing method ValhallaTest$test1${0=J}.test1()V with class=[] and > method=[J] > Specializing ValhallaTest${0=J}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=J}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=J} > Specializing method ValhallaTest$test2${0=J}.test2()V with class=[] and > method=[J] > test2 >>> class ValhallaTest${0=J} > Specializing method ValhallaTest$test1${0=F}.test1()V with class=[] and > method=[F] > Specializing ValhallaTest${0=F}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=F}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=F} > Specializing method ValhallaTest$test2${0=F}.test2()V with class=[] and > method=[F] > test2 >>> class ValhallaTest${0=F} > Specializing method ValhallaTest$test1${0=D}.test1()V with class=[] and > method=[D] > Specializing ValhallaTest${0=D}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=D}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=D} > Specializing method ValhallaTest$test2${0=D}.test2()V with class=[] and > method=[D] > test2 >>> class ValhallaTest${0=D} > test1 >>> class ValhallaTest > Specializing method ValhallaTest$test2${}.test2()V with class=[] and > method=[TT;] > Exception in thread "main" java.lang.NoClassDefFoundError: ValhallaTest at ValhallaTest$test2${}/401625763.test2(ValhallaTest.java:7) > at ValhallaTest.test1(ValhallaTest.java:15) > at ValhallaTest.main(ValhallaTest.java:27) > Caused by: java.lang.ClassNotFoundException: ValhallaTest at java.net.URLClassLoader.findClass(URLClassLoader.java:451) > at java.lang.ClassLoader.loadClass(ClassLoader.java:426) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317) > at java.lang.ClassLoader.loadClass(ClassLoader.java:359) > ... 3 more > > > Within tes1() method, everything works fine, but cascading call to > test2() fails in line 7 where ValhallaTest.class is evaluated, but > only for reference type T. > > Note that there's no specialization for test1() being triggered for > T=reference (which is understandable), but there is a specialization > being triggered for cascaded call to test2() in this case. In my > superficial understanding a call to: > > ValhallaTest.test1(); > > ...is implemented as invokestatic, but a call from test1() to: > > ValhallaTest.test2(); > > ...is implemented as invokedynamic. So in case such call comes from > non-specialized test1(), it should be wired directly to the > nonspecialized test2() method, right? > > Or, couldn't the call from nonspecialized test1() to .test2() also be > implemented as invokestatic and be converted to invokedynamic only by > specialization? > > Regards, Peter > From brian.goetz at oracle.com Thu Jan 8 15:48:46 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jan 2015 10:48:46 -0500 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: References: Message-ID: <54AEA6DE.4080701@oracle.com> If we were rewriting Java from scratch, obviously yes. In practice, we're not sure yet, but we sure would like a story for this. The challenge lies in whether we can come up with a migration compatibility strategy that allows existing clients that say "reference" to freely interoperate with a class which has been migrated to value both with and without recompilation. The tradeoff is one you've seen illustrated quite recently; much of the complexity of the language comes from trying to provide smooth migration compatibility like this. It is of course possible to do; the question is whether it is worth the complexity it would impose -- which is TBD. On 1/8/2015 4:47 AM, Palo Marton wrote: > Hi. I have a question about value types: > > Will it be possible (within valhalla) to rewrite java String as value type > in JDK and use it in all places in JDK where it makes sense? (eg. > Class.forName(string), Object.toString(), Integer.parseInt(string). ?) > > Or more general question: > > There are already plenty of ?value objects? in existing java code. In JDK > we have, String, File, Point2D,... Also many people have already declared > their own class Point { final x,y; } and similar objects. Once we move to > value types in java, will it be possible to rewrite these to value types > without breaking compatibility with old pre-value-types code? E.g. if I > change my Point object to value type in my library, can this new library > (jar) still be used in other projects that were written before this change? > And without need to recompile those other projects? > From brian.goetz at oracle.com Thu Jan 8 15:53:00 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jan 2015 10:53:00 -0500 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: References: Message-ID: <54AEA7DC.7070608@oracle.com> > From what I have read there are no plans to support something like that. Am > I right? Correction: there are no plans *at this time*. Because, we're still working out the basics. From brian.goetz at oracle.com Thu Jan 8 16:02:30 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jan 2015 11:02:30 -0500 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: References: <54AE8C27.50400@gmail.com> Message-ID: <54AEAA16.2010306@oracle.com> Like many questions we get here, this is a fine question which has basically arrived at the wrong time, so we're going to ignore it for a while. Not because it's not important -- it is! But because you have to pour the foundation before you can paint. And the foundational work is occupying all of our attention right now. Of course migrating existing value-based classes is a highly desirable thing and it will definitely be on our mind as we work through the more foundational aspects. String may in the end be a lost cause -- I guarantee you there is plenty of mission-critical code out there that synchronizes on strings -- but there are plenty of value-based classes in the JDK that could benefit from such a treatment and might be less problematic to convert. This is already on the "to think about next year" list, but since the "to think about this year" list is already several years long, we probably won't get to it for a while. On 1/8/2015 9:51 AM, Palo Marton wrote: > Yes, I know that you can not simply replace existing value objects with > value types when executing old pre-values-types code. Such code must be > executed with objects. > > My idea is something along these lines: > > Declare value types as normal class (final with final fields), but add > secondary "value name" for it, e.g. like this: > > public final class *String *value *string *{ > ... > } > > When in your code you declare variable as "String" than it will work the > same as it works now (with identity, lock, null, etc..) > > But when you declare variable as "string", you basically say that: > > - I don't care about identity and locking and don't plan to use it > - this will never be null. > > Which gives JVM option to store and pass it "by value" instead of "by > reference". > > > > > On Thu, Jan 8, 2015 at 3:24 PM, Vitaly Davidovich wrote: > >> I don't think this has been ignored, certainly not on the recent traffic >> on this mailing list. >> >> I think the short answer to the "can existing classes that are value >> objects be migrated to value types with no client change" is probably no. >> The reason, or one of them, is that there's no way to know (without >> analyzing all usages) whether client code uses object features of said >> object (e.g. uses reference identity comparison, locks on an instance, etc). >> >> In your rephrasing, "works like a class" implies identity of the object >> and can be synchronized on. That's not what VT is of course. >> >> Sent from my phone >> On Jan 8, 2015 9:17 AM, "Palo Marton" wrote: >> >>> My question is not just about String but about all existing value types. >>> There are many of them in current java code (both JDK and user code). >>> Current proposal seems to ignore them. >>> >>> I think that the original idea of VT: >>> >>> "Codes like a class, works like an int!" >>> >>> can be changed to: >>> >>> "Codes like a class, works like class, but can be stored and passed as >>> int!" >>> >>> as the problem with current value objects (like String) is not with the >>> fact that they are objects, but in how they are stored in memory and passed >>> between functions. >>> >>> >>> >>> On Thu, Jan 8, 2015 at 3:02 PM, Vitaly Davidovich >>> wrote: >>> >>>> I had a suspicion this would be the gist. However, I think better >>>> solution is to simply change string to have inline storage of the data, so >>>> no indirection to the char[]. Ultimately, the string data will have to >>>> live on the heap one way or another, but it'd be nice if String had that >>>> data inlined into its storage so when you load address of string you get >>>> the data at a small fixed offset from that address. >>>> >>>> Sent from my phone >>>> On Jan 8, 2015 8:54 AM, "Peter Levart" wrote: >>>> >>>>> On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: >>>>> >>>>>> Why do you want string to be value type? What problem (s) will that >>>>>> address? >>>>>> >>>>> >>>>> String as a value type would eliminate one indirection. If would >>>>> effectively become an immutable char[] with a bunch of operations. >>>>> >>>>> Peter >>>>> >>>>> >>>>>> Sent from my phone >>>>>> On Jan 8, 2015 7:04 AM, "Palo Marton" wrote: >>>>>> >>>>>> Thank you for response. I have read that thread. But apart from first >>>>>>> few >>>>>>> messages it seems to deal with something else - mutable types. >>>>>>> >>>>>>> My idea with rewriting String as value is not about changing meaning >>>>>>> of >>>>>>> java.lang.String. That identifier should still mean heap object with >>>>>>> identity, null, etc... The idea is to give a new name for string >>>>>>> value, >>>>>>> e.g. java.lang.string (lowercase s), or just "string". So "String" >>>>>>> will be >>>>>>> boxed version of "string" and all old pre-value-types code will use >>>>>>> only >>>>>>> boxed objects as before and there will be no semantic change. In new >>>>>>> code >>>>>>> you will use "string" in most cases, but you can use also "String" in >>>>>>> places where you need it. >>>>>>> >>>>>>> From what I have read there are no plans to support something like >>>>>>> that. Am >>>>>>> I right? >>>>>>> >>>>>>> Reason why I am asking this is that some time ago I was thinking about >>>>>>> value types in java and how they should be implemented in JVM. My idea >>>>>>> about how to implement it in JVM was somehow different, but it can >>>>>>> support >>>>>>> this old-code / new-code compatibility. >>>>>>> >>>>>>> On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < >>>>>>> richard.warburton at gmail.com> wrote: >>>>>>> >>>>>>> Hi, >>>>>>>> >>>>>>>> Will it be possible (within valhalla) to rewrite java String as value >>>>>>>> >>>>>>> type >>>>>>> >>>>>>>> in JDK and use it in all places in JDK where it makes sense? (eg. >>>>>>>>> Class.forName(string), Object.toString(), Integer.parseInt(string). >>>>>>>>> ?) >>>>>>>>> >>>>>>>>> Or more general question: >>>>>>>>> >>>>>>>>> There are already plenty of ?value objects? in existing java code. >>>>>>>>> In >>>>>>>>> >>>>>>>> JDK >>>>>>> >>>>>>>> we have, String, File, Point2D,... Also many people have already >>>>>>>>> >>>>>>>> declared >>>>>>> >>>>>>>> their own class Point { final x,y; } and similar objects. Once we >>>>>>>>> move >>>>>>>>> >>>>>>>> to >>>>>>> >>>>>>>> value types in java, will it be possible to rewrite these to value >>>>>>>>> types >>>>>>>>> without breaking compatibility with old pre-value-types code? E.g. >>>>>>>>> if I >>>>>>>>> change my Point object to value type in my library, can this new >>>>>>>>> library >>>>>>>>> (jar) still be used in other projects that were written before this >>>>>>>>> change? >>>>>>>>> And without need to recompile those other projects? >>>>>>>>> >>>>>>>> >>>>>>>> See this thread: >>>>>>>> >>>>>>>> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- >>>>>>> January/000546.html >>>>>>> >>>>>>>> regards, >>>>>>>> >>>>>>>> Richard Warburton >>>>>>>> >>>>>>>> http://insightfullogic.com >>>>>>>> @RichardWarburto >>>>>>>> >>>>>>>> >>>>>>>> >>>>> >>> From palo.marton at gmail.com Thu Jan 8 16:17:45 2015 From: palo.marton at gmail.com (Palo Marton) Date: Thu, 8 Jan 2015 17:17:45 +0100 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AEAA16.2010306@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> Message-ID: I see your point and may be I'm too late with what I wanted to suggest. I know that you need to have good foundation for this, but this foundation will probably greatly influence possible future solution to value object/type compatibility. I will try to write down some general outline of my idea on how to implement value types and achieve this compatibility. It is very close to current proposal in its syntax, memory layout, value array implementation and it will probably work also with generic specialization. Where it differs is how is value type defined in class file and in method/field descriptors containing value type. (and it will also cause some change to vftable). It also addresses this (I think serious) problem in current proposal: - ?Value types? seem to assume that you will use it just for small types and you will implement bigger types as objects (with different syntax). In practice this might be a problem. E.g. in computer graphics/vision you have vectors of various dimensions (2,3,4), matrices of various dimensions (2x2,2x3,3x3,4x3,4x4) and other similar immutable types. Having different syntax for these based on the size (2x2 matrix of floats is value type, but 4x4 matrix of float or 2x2 matrix of doubles should be object) will be really cumbersome. On Thu, Jan 8, 2015 at 5:02 PM, Brian Goetz wrote: > Like many questions we get here, this is a fine question which has > basically arrived at the wrong time, so we're going to ignore it for a > while. Not because it's not important -- it is! But because you have to > pour the foundation before you can paint. And the foundational work is > occupying all of our attention right now. > > Of course migrating existing value-based classes is a highly desirable > thing and it will definitely be on our mind as we work through the more > foundational aspects. String may in the end be a lost cause -- I guarantee > you there is plenty of mission-critical code out there that synchronizes on > strings -- but there are plenty of value-based classes in the JDK that > could benefit from such a treatment and might be less problematic to > convert. > > This is already on the "to think about next year" list, but since the "to > think about this year" list is already several years long, we probably > won't get to it for a while. > > On 1/8/2015 9:51 AM, Palo Marton wrote: > >> Yes, I know that you can not simply replace existing value objects with >> value types when executing old pre-values-types code. Such code must be >> executed with objects. >> >> My idea is something along these lines: >> >> Declare value types as normal class (final with final fields), but add >> secondary "value name" for it, e.g. like this: >> >> public final class *String *value *string *{ >> >> ... >> } >> >> When in your code you declare variable as "String" than it will work the >> same as it works now (with identity, lock, null, etc..) >> >> But when you declare variable as "string", you basically say that: >> >> - I don't care about identity and locking and don't plan to use it >> - this will never be null. >> >> Which gives JVM option to store and pass it "by value" instead of "by >> reference". >> >> >> >> >> On Thu, Jan 8, 2015 at 3:24 PM, Vitaly Davidovich >> wrote: >> >> I don't think this has been ignored, certainly not on the recent traffic >>> on this mailing list. >>> >>> I think the short answer to the "can existing classes that are value >>> objects be migrated to value types with no client change" is probably no. >>> The reason, or one of them, is that there's no way to know (without >>> analyzing all usages) whether client code uses object features of said >>> object (e.g. uses reference identity comparison, locks on an instance, >>> etc). >>> >>> In your rephrasing, "works like a class" implies identity of the object >>> and can be synchronized on. That's not what VT is of course. >>> >>> Sent from my phone >>> On Jan 8, 2015 9:17 AM, "Palo Marton" wrote: >>> >>> My question is not just about String but about all existing value types. >>>> There are many of them in current java code (both JDK and user code). >>>> Current proposal seems to ignore them. >>>> >>>> I think that the original idea of VT: >>>> >>>> "Codes like a class, works like an int!" >>>> >>>> can be changed to: >>>> >>>> "Codes like a class, works like class, but can be stored and passed as >>>> int!" >>>> >>>> as the problem with current value objects (like String) is not with the >>>> fact that they are objects, but in how they are stored in memory and >>>> passed >>>> between functions. >>>> >>>> >>>> >>>> On Thu, Jan 8, 2015 at 3:02 PM, Vitaly Davidovich >>>> wrote: >>>> >>>> I had a suspicion this would be the gist. However, I think better >>>>> solution is to simply change string to have inline storage of the >>>>> data, so >>>>> no indirection to the char[]. Ultimately, the string data will have to >>>>> live on the heap one way or another, but it'd be nice if String had >>>>> that >>>>> data inlined into its storage so when you load address of string you >>>>> get >>>>> the data at a small fixed offset from that address. >>>>> >>>>> Sent from my phone >>>>> On Jan 8, 2015 8:54 AM, "Peter Levart" wrote: >>>>> >>>>> On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: >>>>>> >>>>>> Why do you want string to be value type? What problem (s) will that >>>>>>> address? >>>>>>> >>>>>>> >>>>>> String as a value type would eliminate one indirection. If would >>>>>> effectively become an immutable char[] with a bunch of operations. >>>>>> >>>>>> Peter >>>>>> >>>>>> >>>>>> Sent from my phone >>>>>>> On Jan 8, 2015 7:04 AM, "Palo Marton" wrote: >>>>>>> >>>>>>> Thank you for response. I have read that thread. But apart from >>>>>>> first >>>>>>> >>>>>>>> few >>>>>>>> messages it seems to deal with something else - mutable types. >>>>>>>> >>>>>>>> My idea with rewriting String as value is not about changing meaning >>>>>>>> of >>>>>>>> java.lang.String. That identifier should still mean heap object with >>>>>>>> identity, null, etc... The idea is to give a new name for string >>>>>>>> value, >>>>>>>> e.g. java.lang.string (lowercase s), or just "string". So "String" >>>>>>>> will be >>>>>>>> boxed version of "string" and all old pre-value-types code will use >>>>>>>> only >>>>>>>> boxed objects as before and there will be no semantic change. In new >>>>>>>> code >>>>>>>> you will use "string" in most cases, but you can use also "String" >>>>>>>> in >>>>>>>> places where you need it. >>>>>>>> >>>>>>>> From what I have read there are no plans to support something like >>>>>>>> that. Am >>>>>>>> I right? >>>>>>>> >>>>>>>> Reason why I am asking this is that some time ago I was thinking >>>>>>>> about >>>>>>>> value types in java and how they should be implemented in JVM. My >>>>>>>> idea >>>>>>>> about how to implement it in JVM was somehow different, but it can >>>>>>>> support >>>>>>>> this old-code / new-code compatibility. >>>>>>>> >>>>>>>> On Thu, Jan 8, 2015 at 11:42 AM, Richard Warburton < >>>>>>>> richard.warburton at gmail.com> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>>> >>>>>>>>> Will it be possible (within valhalla) to rewrite java String as >>>>>>>>> value >>>>>>>>> >>>>>>>>> type >>>>>>>> >>>>>>>> in JDK and use it in all places in JDK where it makes sense? (eg. >>>>>>>>> >>>>>>>>>> Class.forName(string), Object.toString(), >>>>>>>>>> Integer.parseInt(string). >>>>>>>>>> ?) >>>>>>>>>> >>>>>>>>>> Or more general question: >>>>>>>>>> >>>>>>>>>> There are already plenty of ?value objects? in existing java code. >>>>>>>>>> In >>>>>>>>>> >>>>>>>>>> JDK >>>>>>>>> >>>>>>>> >>>>>>>> we have, String, File, Point2D,... Also many people have already >>>>>>>>> >>>>>>>>>> >>>>>>>>>> declared >>>>>>>>> >>>>>>>> >>>>>>>> their own class Point { final x,y; } and similar objects. Once we >>>>>>>>> >>>>>>>>>> move >>>>>>>>>> >>>>>>>>>> to >>>>>>>>> >>>>>>>> >>>>>>>> value types in java, will it be possible to rewrite these to value >>>>>>>>> >>>>>>>>>> types >>>>>>>>>> without breaking compatibility with old pre-value-types code? E.g. >>>>>>>>>> if I >>>>>>>>>> change my Point object to value type in my library, can this new >>>>>>>>>> library >>>>>>>>>> (jar) still be used in other projects that were written before >>>>>>>>>> this >>>>>>>>>> change? >>>>>>>>>> And without need to recompile those other projects? >>>>>>>>>> >>>>>>>>>> >>>>>>>>> See this thread: >>>>>>>>> >>>>>>>>> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- >>>>>>>>> >>>>>>>> January/000546.html >>>>>>>> >>>>>>>> regards, >>>>>>>>> >>>>>>>>> Richard Warburton >>>>>>>>> >>>>>>>>> http://insightfullogic.com >>>>>>>>> @RichardWarburto >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>> >>>> From richard.warburton at gmail.com Thu Jan 8 16:33:41 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Thu, 8 Jan 2015 16:33:41 +0000 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> Message-ID: Hi, I think its also worth noting that your request to make String faster by removing the pointer indirection on its char array is a problem which is ideally suited to the Object Layout proposal (http://objectlayout.org/). This proposal doesn't add value types to the JVM, but simply gives more control over the member layout of existing Objects and is thus significantly narrower in scope. It consequently doesn't have the migration compatibility issues outlined on this list. Object Layout and value types aren't mutually exclusive. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From palo.marton at gmail.com Thu Jan 8 16:37:49 2015 From: palo.marton at gmail.com (Palo Marton) Date: Thu, 8 Jan 2015 17:37:49 +0100 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> Message-ID: Yes, I know about that proposal. But my main idea is not to fix string class, but to improve compatibility between post-VT and pre-VT code. String was used just as an example of existing value class. On Thu, Jan 8, 2015 at 5:33 PM, Richard Warburton < richard.warburton at gmail.com> wrote: > Hi, > > I think its also worth noting that your request to make String faster by > removing the pointer indirection on its char array is a problem which is > ideally suited to the Object Layout proposal (http://objectlayout.org/). > This proposal doesn't add value types to the JVM, but simply gives more > control over the member layout of existing Objects and is thus > significantly narrower in scope. It consequently doesn't have the migration > compatibility issues outlined on this list. Object Layout and value types > aren't mutually exclusive. > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > From brian.goetz at oracle.com Thu Jan 8 16:45:00 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jan 2015 11:45:00 -0500 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> Message-ID: <54AEB40C.7030702@oracle.com> > It also addresses this (I think serious) problem in current proposal: You say "problem", I say "tradeoff". > ?Value types? seem to assume that you will use it just for small > types and you will implement bigger types as objects (with different > syntax). Essentially, yes. There's no hard cutoff of "value types must be less than X size", but the VM reserves the right to transparently box values that are too big (this is better than rejecting them either statically or dynamically). You can still code them as values and get the value semantics and the VM will try its best to give you the performance, but you don't get guaranteed flattening. (Note that the benefits of flattening decrease dramatically once they get big enough anyway.) If we could make objects just magically work like ints when they were small enough, there'd be no need to introduce a new construct in the first place. > In practice this might be a problem. E.g. in computer > graphics/vision you have vectors of various dimensions (2,3,4), > matrices of various dimensions (2x2,2x3,3x3,4x3,4x4) and other > similar immutable types. Having different syntax for these based on > the size (2x2 matrix of floats is value type, but 4x4 matrix of > float or 2x2 matrix of doubles should be object) will be really > cumbersome. I am not sure where you got this idea that somehow "too big" values would require a different syntax. However, its possible you're running on a machine with a very restricted register set, and a 4x4 matrix of longs is too big to pass by value, so it gets silently boxed, and you don't get all the performance you get with a 2x2 matrix. But this is no different from any other hardware effect (size or number of cache lines, etc.) (And yes, we understand that transform matrices is a desirable thing to be able to fit into the sweet spot.) Will value types solve all indirection-related performance woes? Clearly not. There are some people who would prefer something more like mutable structs, which would address a different subset of indirection-related performance woes (and introduce a whole new set of complexities.) IBM's Packed Objects proposal had a whole different set of tradeoffs. > > > > On Thu, Jan 8, 2015 at 5:02 PM, Brian Goetz > wrote: > > Like many questions we get here, this is a fine question which has > basically arrived at the wrong time, so we're going to ignore it for > a while. Not because it's not important -- it is! But because you > have to pour the foundation before you can paint. And the > foundational work is occupying all of our attention right now. > > Of course migrating existing value-based classes is a highly > desirable thing and it will definitely be on our mind as we work > through the more foundational aspects. String may in the end be a > lost cause -- I guarantee you there is plenty of mission-critical > code out there that synchronizes on strings -- but there are plenty > of value-based classes in the JDK that could benefit from such a > treatment and might be less problematic to convert. > > This is already on the "to think about next year" list, but since > the "to think about this year" list is already several years long, > we probably won't get to it for a while. > > On 1/8/2015 9:51 AM, Palo Marton wrote: > > Yes, I know that you can not simply replace existing value > objects with > value types when executing old pre-values-types code. Such code > must be > executed with objects. > > My idea is something along these lines: > > Declare value types as normal class (final with final fields), > but add > secondary "value name" for it, e.g. like this: > > public final class *String *value *string *{ > > ... > } > > When in your code you declare variable as "String" than it will > work the > same as it works now (with identity, lock, null, etc..) > > But when you declare variable as "string", you basically say that: > > - I don't care about identity and locking and don't plan to use it > - this will never be null. > > Which gives JVM option to store and pass it "by value" instead > of "by > reference". > > > > > On Thu, Jan 8, 2015 at 3:24 PM, Vitaly Davidovich > > wrote: > > I don't think this has been ignored, certainly not on the > recent traffic > on this mailing list. > > I think the short answer to the "can existing classes that > are value > objects be migrated to value types with no client change" is > probably no. > The reason, or one of them, is that there's no way to know > (without > analyzing all usages) whether client code uses object > features of said > object (e.g. uses reference identity comparison, locks on an > instance, etc). > > In your rephrasing, "works like a class" implies identity of > the object > and can be synchronized on. That's not what VT is of course. > > Sent from my phone > On Jan 8, 2015 9:17 AM, "Palo Marton" > wrote: > > My question is not just about String but about all > existing value types. > There are many of them in current java code (both JDK > and user code). > Current proposal seems to ignore them. > > I think that the original idea of VT: > > "Codes like a class, works like an int!" > > can be changed to: > > "Codes like a class, works like class, but can be stored > and passed as > int!" > > as the problem with current value objects (like String) > is not with the > fact that they are objects, but in how they are stored > in memory and passed > between functions. > > > > On Thu, Jan 8, 2015 at 3:02 PM, Vitaly Davidovich > > > wrote: > > I had a suspicion this would be the gist. However, > I think better > solution is to simply change string to have inline > storage of the data, so > no indirection to the char[]. Ultimately, the > string data will have to > live on the heap one way or another, but it'd be > nice if String had that > data inlined into its storage so when you load > address of string you get > the data at a small fixed offset from that address. > > Sent from my phone > On Jan 8, 2015 8:54 AM, "Peter Levart" > > wrote: > > On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: > > Why do you want string to be value type? > What problem (s) will that > address? > > > String as a value type would eliminate one > indirection. If would > effectively become an immutable char[] with a > bunch of operations. > > Peter > > > Sent from my phone > On Jan 8, 2015 7:04 AM, "Palo Marton" > > wrote: > > Thank you for response. I have read that > thread. But apart from first > > few > messages it seems to deal with something > else - mutable types. > > My idea with rewriting String as value > is not about changing meaning > of > java.lang.String. That identifier should > still mean heap object with > identity, null, etc... The idea is to > give a new name for string > value, > e.g. java.lang.string (lowercase s), or > just "string". So "String" > will be > boxed version of "string" and all old > pre-value-types code will use > only > boxed objects as before and there will > be no semantic change. In new > code > you will use "string" in most cases, but > you can use also "String" in > places where you need it. > > From what I have read there are no > plans to support something like > that. Am > I right? > > Reason why I am asking this is that some > time ago I was thinking about > value types in java and how they should > be implemented in JVM. My idea > about how to implement it in JVM was > somehow different, but it can > support > this old-code / new-code compatibility. > > On Thu, Jan 8, 2015 at 11:42 AM, Richard > Warburton < > richard.warburton at gmail.com > > wrote: > > Hi, > > > Will it be possible (within > valhalla) to rewrite java String as > value > > type > > in JDK and use it in all places in > JDK where it makes sense? (eg. > > Class.forName(string), > Object.toString(), > Integer.parseInt(string). > ?) > > Or more general question: > > There are already plenty of > ?value objects? in existing java > code. > In > > JDK > > > we have, String, File, Point2D,... > Also many people have already > > > declared > > > their own class Point { final x,y; } > and similar objects. Once we > > move > > to > > > value types in java, will it be > possible to rewrite these to value > > types > without breaking compatibility > with old pre-value-types code? E.g. > if I > change my Point object to value > type in my library, can this new > library > (jar) still be used in other > projects that were written > before this > change? > And without need to recompile > those other projects? > > > See this thread: > > http://mail.openjdk.java.net/__pipermail/valhalla-dev/2015- > > > January/000546.html > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > > > > > > > > From vitalyd at gmail.com Thu Jan 8 16:59:56 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 11:59:56 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AEB40C.7030702@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> Message-ID: > > I am not sure where you got this idea that somehow "too big" values would > require a different syntax. However, its possible you're running on a > machine with a very restricted register set, and a 4x4 matrix of longs is > too big to pass by value, so it gets silently boxed, and you don't get all > the performance you get with a 2x2 matrix. But this is no different from > any other hardware effect (size or number of cache lines, etc.) Why is # of registers a factor here? I'd expect some # of registers to be used for passing some of the values (depending on calling convention), and the rest passed on the stack. This is also the first I hear of VM reserving the right to transparently box depending on some size threshold. My preference would be to simply use the stack, and if you run out of stack, handle that the same way as today. Clearly someone passing a giant value type around is not doing themselves any good, but the automagic boxing behind the scenes doesn't sit well. What if boxing that giant value type causes an OOM? I don't think that would be expected. On Thu, Jan 8, 2015 at 11:45 AM, Brian Goetz wrote: > It also addresses this (I think serious) problem in current proposal: >> > > You say "problem", I say "tradeoff". > > ?Value types? seem to assume that you will use it just for small >> types and you will implement bigger types as objects (with different >> syntax). >> > > Essentially, yes. There's no hard cutoff of "value types must be less > than X size", but the VM reserves the right to transparently box values > that are too big (this is better than rejecting them either statically or > dynamically). You can still code them as values and get the value > semantics and the VM will try its best to give you the performance, but you > don't get guaranteed flattening. (Note that the benefits of flattening > decrease dramatically once they get big enough anyway.) > > If we could make objects just magically work like ints when they were > small enough, there'd be no need to introduce a new construct in the first > place. > > In practice this might be a problem. E.g. in computer >> graphics/vision you have vectors of various dimensions (2,3,4), >> matrices of various dimensions (2x2,2x3,3x3,4x3,4x4) and other >> similar immutable types. Having different syntax for these based on >> the size (2x2 matrix of floats is value type, but 4x4 matrix of >> float or 2x2 matrix of doubles should be object) will be really >> cumbersome. >> > > I am not sure where you got this idea that somehow "too big" values would > require a different syntax. However, its possible you're running on a > machine with a very restricted register set, and a 4x4 matrix of longs is > too big to pass by value, so it gets silently boxed, and you don't get all > the performance you get with a 2x2 matrix. But this is no different from > any other hardware effect (size or number of cache lines, etc.) > > (And yes, we understand that transform matrices is a desirable thing to be > able to fit into the sweet spot.) > > Will value types solve all indirection-related performance woes? Clearly > not. There are some people who would prefer something more like mutable > structs, which would address a different subset of indirection-related > performance woes (and introduce a whole new set of complexities.) IBM's > Packed Objects proposal had a whole different set of tradeoffs. > > > > > > >> >> >> On Thu, Jan 8, 2015 at 5:02 PM, Brian Goetz > > wrote: >> >> Like many questions we get here, this is a fine question which has >> basically arrived at the wrong time, so we're going to ignore it for >> a while. Not because it's not important -- it is! But because you >> have to pour the foundation before you can paint. And the >> foundational work is occupying all of our attention right now. >> >> Of course migrating existing value-based classes is a highly >> desirable thing and it will definitely be on our mind as we work >> through the more foundational aspects. String may in the end be a >> lost cause -- I guarantee you there is plenty of mission-critical >> code out there that synchronizes on strings -- but there are plenty >> of value-based classes in the JDK that could benefit from such a >> treatment and might be less problematic to convert. >> >> This is already on the "to think about next year" list, but since >> the "to think about this year" list is already several years long, >> we probably won't get to it for a while. >> >> On 1/8/2015 9:51 AM, Palo Marton wrote: >> >> Yes, I know that you can not simply replace existing value >> objects with >> value types when executing old pre-values-types code. Such code >> must be >> executed with objects. >> >> My idea is something along these lines: >> >> Declare value types as normal class (final with final fields), >> but add >> secondary "value name" for it, e.g. like this: >> >> public final class *String *value *string *{ >> >> ... >> } >> >> When in your code you declare variable as "String" than it will >> work the >> same as it works now (with identity, lock, null, etc..) >> >> But when you declare variable as "string", you basically say that: >> >> - I don't care about identity and locking and don't plan to use it >> - this will never be null. >> >> Which gives JVM option to store and pass it "by value" instead >> of "by >> reference". >> >> >> >> >> On Thu, Jan 8, 2015 at 3:24 PM, Vitaly Davidovich >> > wrote: >> >> I don't think this has been ignored, certainly not on the >> recent traffic >> on this mailing list. >> >> I think the short answer to the "can existing classes that >> are value >> objects be migrated to value types with no client change" is >> probably no. >> The reason, or one of them, is that there's no way to know >> (without >> analyzing all usages) whether client code uses object >> features of said >> object (e.g. uses reference identity comparison, locks on an >> instance, etc). >> >> In your rephrasing, "works like a class" implies identity of >> the object >> and can be synchronized on. That's not what VT is of course. >> >> Sent from my phone >> On Jan 8, 2015 9:17 AM, "Palo Marton" > > wrote: >> >> My question is not just about String but about all >> existing value types. >> There are many of them in current java code (both JDK >> and user code). >> Current proposal seems to ignore them. >> >> I think that the original idea of VT: >> >> "Codes like a class, works like an int!" >> >> can be changed to: >> >> "Codes like a class, works like class, but can be stored >> and passed as >> int!" >> >> as the problem with current value objects (like String) >> is not with the >> fact that they are objects, but in how they are stored >> in memory and passed >> between functions. >> >> >> >> On Thu, Jan 8, 2015 at 3:02 PM, Vitaly Davidovich >> > >> wrote: >> >> I had a suspicion this would be the gist. However, >> I think better >> solution is to simply change string to have inline >> storage of the data, so >> no indirection to the char[]. Ultimately, the >> string data will have to >> live on the heap one way or another, but it'd be >> nice if String had that >> data inlined into its storage so when you load >> address of string you get >> the data at a small fixed offset from that address. >> >> Sent from my phone >> On Jan 8, 2015 8:54 AM, "Peter Levart" >> > > wrote: >> >> On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: >> >> Why do you want string to be value type? >> What problem (s) will that >> address? >> >> >> String as a value type would eliminate one >> indirection. If would >> effectively become an immutable char[] with a >> bunch of operations. >> >> Peter >> >> >> Sent from my phone >> On Jan 8, 2015 7:04 AM, "Palo Marton" >> > > wrote: >> >> Thank you for response. I have read that >> thread. But apart from first >> >> few >> messages it seems to deal with something >> else - mutable types. >> >> My idea with rewriting String as value >> is not about changing meaning >> of >> java.lang.String. That identifier should >> still mean heap object with >> identity, null, etc... The idea is to >> give a new name for string >> value, >> e.g. java.lang.string (lowercase s), or >> just "string". So "String" >> will be >> boxed version of "string" and all old >> pre-value-types code will use >> only >> boxed objects as before and there will >> be no semantic change. In new >> code >> you will use "string" in most cases, but >> you can use also "String" in >> places where you need it. >> >> From what I have read there are no >> plans to support something like >> that. Am >> I right? >> >> Reason why I am asking this is that some >> time ago I was thinking about >> value types in java and how they should >> be implemented in JVM. My idea >> about how to implement it in JVM was >> somehow different, but it can >> support >> this old-code / new-code compatibility. >> >> On Thu, Jan 8, 2015 at 11:42 AM, Richard >> Warburton < >> richard.warburton at gmail.com >> > >> wrote: >> >> Hi, >> >> >> Will it be possible (within >> valhalla) to rewrite java String as >> value >> >> type >> >> in JDK and use it in all places in >> JDK where it makes sense? (eg. >> >> Class.forName(string), >> Object.toString(), >> Integer.parseInt(string). >> ?) >> >> Or more general question: >> >> There are already plenty of >> ?value objects? in existing java >> code. >> In >> >> JDK >> >> >> we have, String, File, Point2D,... >> Also many people have already >> >> >> declared >> >> >> their own class Point { final x,y; } >> and similar objects. Once we >> >> move >> >> to >> >> >> value types in java, will it be >> possible to rewrite these to value >> >> types >> without breaking compatibility >> with old pre-value-types code? >> E.g. >> if I >> change my Point object to value >> type in my library, can this new >> library >> (jar) still be used in other >> projects that were written >> before this >> change? >> And without need to recompile >> those other projects? >> >> >> See this thread: >> >> http://mail.openjdk.java.net/_ >> _pipermail/valhalla-dev/2015- >> > pipermail/valhalla-dev/2015-> >> >> January/000546.html >> >> regards, >> >> Richard Warburton >> >> http://insightfullogic.com >> @RichardWarburto >> > > >> >> >> >> >> >> >> From pbenedict at apache.org Thu Jan 8 18:09:47 2015 From: pbenedict at apache.org (Paul Benedict) Date: Thu, 8 Jan 2015 12:09:47 -0600 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> Message-ID: Regarding ?Value types? seem to assume that you will use it just for small types and you will implement bigger types as objects (with different syntax)."... Value types are "final" which means it won't be possible to subclass. However, I am disappointed with this restriction because I would like to use inheritance as an easy way of composing value types. Brian, what are your thoughts about this? Cheers, Paul On Thu, Jan 8, 2015 at 10:17 AM, Palo Marton wrote: > I see your point and may be I'm too late with what I wanted to suggest. > I know that you need to have good foundation for this, but this foundation > will probably greatly influence possible future solution to value > object/type compatibility. > > I will try to write down some general outline of my idea on how to > implement value types and achieve this compatibility. It is very close to > current proposal in its syntax, memory layout, value array implementation > and it will probably work also with generic specialization. Where it > differs is how is value type defined in class file and in method/field > descriptors containing value type. (and it will also cause some change to > vftable). > > It also addresses this (I think serious) problem in current proposal: > > > - > > ?Value types? seem to assume that you will use it just for small types > and you will implement bigger types as objects (with different syntax). > In > practice this might be a problem. E.g. in computer graphics/vision you > have > vectors of various dimensions (2,3,4), matrices of various dimensions > (2x2,2x3,3x3,4x3,4x4) and other similar immutable types. Having > different > syntax for these based on the size (2x2 matrix of floats is value type, > but > 4x4 matrix of float or 2x2 matrix of doubles should be object) will be > really cumbersome. > > > > From brian.goetz at oracle.com Thu Jan 8 18:27:25 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jan 2015 13:27:25 -0500 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> Message-ID: <54AECC0D.8070703@oracle.com> > Why is # of registers a factor here? I'd expect some # of registers to > be used for passing some of the values (depending on calling > convention), and the rest passed on the stack. > > This is also the first I hear of VM reserving the right to transparently > box depending on some size threshold. My preference would be to simply > use the stack, and if you run out of stack, handle that the same way as > today. Clearly someone passing a giant value type around is not doing > themselves any good, but the automagic boxing behind the scenes doesn't > sit well. What if boxing that giant value type causes an OOM? I don't > think that would be expected. That would be taking away a valuable degree of freedom for the VM to optimize. Suppose you have a big value type VVV with 1000 fields, a1 .. a1000. Now you have: class Foo { VVV biggie; } and a method: void getA33(VVV biggie) { return biggie.a33; } and a call getA33(foo.biggie); Do you want to push 1000 elements on the stack just to pick #33? The "value" is already on the heap, in the "biggie" field of Foo. Why not let the VM pass it by reference, saying "that's the big value you want, over there", and let the VM make the tradeoff based on the relative cost of copying N elements to the stack vs the indirection to the heap? The bytecodes define the semantics; the VM should be free to pass via registers or stack or heap or smoke signals based on what it decides is the best. The size of register files and stack segments will likely be one input into these calculations. Another consideration is atomicity, for values that need to be guaranteed tear-free. If they fit into a cache line, there's probably already a free atomic update mechanism available to the VM; if the value is big, the VM might prefer to box the value and CAS on the reference. (Or maybe there's hardware TM available.) We let the VM make these choices. From brian.goetz at oracle.com Thu Jan 8 18:35:46 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jan 2015 13:35:46 -0500 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> Message-ID: <54AECE02.9050908@oracle.com> > Regarding ?Value types? seem to assume that you will use it just for > small types and you will implement bigger types as objects (with > different syntax)."... There seem to be a lot of mis-assumptions on this topic. There is no arbitrary limit on the size of a value type (other than existing classfile limitations such as constant pool size). However, many of the performance benefits of value types drop off when you get beyond a handful or two of components. This is why we say the "sweet spot" is things like numerics, small-ish tuples, or algebraic data types. But there's no "it stops working beyond" point that you are forced to switch to, (and where there are visible performance differences will be a function of hardware and time, just like cache effects are today.) > Value types are "final" which means it won't be possible to subclass. > However, I am disappointed with this restriction because I would like to > use inheritance as an easy way of composing value types. Brian, what are > your thoughts about this? Nothing is free. Here's one example of where the flexibility you want (and I understand why you want it, restrictions usually feel restrictive) interferes with the performance you want. Suppose I could do this: value class A { int x; int y; } value class B extends A { int z; } A[] array; OK, how do I lay this out now? Part of the point was that I want a packed layout without object headers. But if I don't know how big a B is, I can't have this. (I don't even know whether there are other subclasses of A out there that haven't even been loaded yet, and might be even bigger than B. So I can't do the simplistic thing of "take the biggest subclass layout I've seen at time of array instantiation".) From vitalyd at gmail.com Thu Jan 8 18:49:02 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 13:49:02 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AECC0D.8070703@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <54AECC0D.8070703@oracle.com> Message-ID: Let me preface that for your simple example, I'd hope the JVM can optimize this where it boils down to simply reading biggie.a33 without any copying and method calls. However, I take the point where this may not be the case. How would the box be created in this case? Would it be a "view" over the memory range covered by biggie inside the Foo instance? I'm guessing this box is not an actual heap allocated box with the data copied into it, because now you're back to copying. So, is the idea that the VM can "box" in different ways, sometimes making a full heap copy (because perhaps someone then uses the box as an object, e.g. synchronizing on it) and at other times creating a slice/view over some bit of memory? I guess my point is that if I authored a value type with 1000/"very large # of" fields, then I'm clearly saying I'm ok with having this passed around and copied. The problem with boxing operations that are invisible and also auto-tuned by the JVM is that it may not always make the most optimal choice (e.g. older JVM running on newer hardware where the # of registers, cache line size, and/or any other parameter have changed). Are you then going to expose more JVM tunables/flags to allow users to control this? Seems like a lot of complexity to handle an edge case (i.e. I don't expect developers to create ridiculously large value types; that certainly hasn't been my experience on the .NET side of things). Also, perhaps I don't care about wasting stack space and incurring the copy cost, but want to avoid heap allocations at all cost. Another consideration is atomicity, for values that need to be guaranteed > tear-free. If they fit into a cache line, there's probably already a free > atomic update mechanism available to the VM; if the value is big, the VM > might prefer to box the value and CAS on the reference. (Or maybe there's > hardware TM available.) We let the VM make these choices. Not sure how boxing the value and CAS'ing on the reference would work -- how would the new object be copied back into the "inlined" value storage of the owning object? Using the Foo + VVV example above, how would that work exactly? I'm not even sure it's a good idea (at least for v1) to make any promises around tearing/atomicity of value types. Outside of HTM, modern hardware does not support arbitrary width atomic operations. I would make the user use locks/synchronization to update the value types. For cases where you have a small struct (e.g. wraps an int32) and want to treat like AtomicInteger/etc, then they could always store the value internally as an int, but work with it using the value type. On Thu, Jan 8, 2015 at 1:27 PM, Brian Goetz wrote: > Why is # of registers a factor here? I'd expect some # of registers to >> be used for passing some of the values (depending on calling >> convention), and the rest passed on the stack. >> >> This is also the first I hear of VM reserving the right to transparently >> box depending on some size threshold. My preference would be to simply >> use the stack, and if you run out of stack, handle that the same way as >> today. Clearly someone passing a giant value type around is not doing >> themselves any good, but the automagic boxing behind the scenes doesn't >> sit well. What if boxing that giant value type causes an OOM? I don't >> think that would be expected. >> > > That would be taking away a valuable degree of freedom for the VM to > optimize. > > Suppose you have a big value type VVV with 1000 fields, a1 .. a1000. Now > you have: > > class Foo { > VVV biggie; > } > > and a method: > > void getA33(VVV biggie) { return biggie.a33; } > > and a call > > getA33(foo.biggie); > > Do you want to push 1000 elements on the stack just to pick #33? The > "value" is already on the heap, in the "biggie" field of Foo. Why not let > the VM pass it by reference, saying "that's the big value you want, over > there", and let the VM make the tradeoff based on the relative cost of > copying N elements to the stack vs the indirection to the heap? > > The bytecodes define the semantics; the VM should be free to pass via > registers or stack or heap or smoke signals based on what it decides is the > best. > > The size of register files and stack segments will likely be one input > into these calculations. > > > > Another consideration is atomicity, for values that need to be guaranteed > tear-free. If they fit into a cache line, there's probably already a free > atomic update mechanism available to the VM; if the value is big, the VM > might prefer to box the value and CAS on the reference. (Or maybe there's > hardware TM available.) We let the VM make these choices. > > From vitalyd at gmail.com Thu Jan 8 18:50:24 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 13:50:24 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AECE02.9050908@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AECE02.9050908@oracle.com> Message-ID: My question would be why not compose value types using ... well, composition? Is there an example where inheritance is workable but composition isn't? On Thu, Jan 8, 2015 at 1:35 PM, Brian Goetz wrote: > Regarding ?Value types? seem to assume that you will use it just for >> small types and you will implement bigger types as objects (with >> different syntax)."... >> > > There seem to be a lot of mis-assumptions on this topic. There is no > arbitrary limit on the size of a value type (other than existing classfile > limitations such as constant pool size). However, many of the performance > benefits of value types drop off when you get beyond a handful or two of > components. This is why we say the "sweet spot" is things like numerics, > small-ish tuples, or algebraic data types. But there's no "it stops > working beyond" point that you are forced to switch to, (and where there > are visible performance differences will be a function of hardware and > time, just like cache effects are today.) > > Value types are "final" which means it won't be possible to subclass. >> However, I am disappointed with this restriction because I would like to >> use inheritance as an easy way of composing value types. Brian, what are >> your thoughts about this? >> > > Nothing is free. > > Here's one example of where the flexibility you want (and I understand why > you want it, restrictions usually feel restrictive) interferes with the > performance you want. Suppose I could do this: > > value class A { > int x; > int y; > } > > value class B extends A { > int z; > } > > A[] array; > > OK, how do I lay this out now? Part of the point was that I want a packed > layout without object headers. But if I don't know how big a B is, I can't > have this. (I don't even know whether there are other subclasses of A out > there that haven't even been loaded yet, and might be even bigger than B. > So I can't do the simplistic thing of "take the biggest subclass layout > I've seen at time of array instantiation".) > > > From peter.levart at gmail.com Thu Jan 8 18:52:40 2015 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 08 Jan 2015 19:52:40 +0100 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: <54AECE02.9050908@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AECE02.9050908@oracle.com> Message-ID: <54AED1F8.9050004@gmail.com> On 01/08/2015 07:35 PM, Brian Goetz wrote: >> Regarding ?Value types? seem to assume that you will use it just for >> small types and you will implement bigger types as objects (with >> different syntax)."... > > There seem to be a lot of mis-assumptions on this topic. There is no > arbitrary limit on the size of a value type (other than existing > classfile limitations such as constant pool size). However, many of > the performance benefits of value types drop off when you get beyond a > handful or two of components. This is why we say the "sweet spot" is > things like numerics, small-ish tuples, or algebraic data types. But > there's no "it stops working beyond" point that you are forced to > switch to, (and where there are visible performance differences will > be a function of hardware and time, just like cache effects are today.) > >> Value types are "final" which means it won't be possible to subclass. >> However, I am disappointed with this restriction because I would like to >> use inheritance as an easy way of composing value types. Brian, what are >> your thoughts about this? > > Nothing is free. > > Here's one example of where the flexibility you want (and I understand > why you want it, restrictions usually feel restrictive) interferes > with the performance you want. Suppose I could do this: > > value class A { > int x; > int y; > } > > value class B extends A { > int z; > } > > A[] array; > > OK, how do I lay this out now? Part of the point was that I want a > packed layout without object headers. But if I don't know how big a B > is, I can't have this. (I don't even know whether there are other > subclasses of A out there that haven't even been loaded yet, and might > be even bigger than B. So I can't do the simplistic thing of "take > the biggest subclass layout I've seen at time of array instantiation".) > > What if A and B as value types were unrelated? Just their boxed counterparts would be related. Peter From brian.goetz at oracle.com Thu Jan 8 19:17:40 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jan 2015 14:17:40 -0500 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <54AECC0D.8070703@oracle.com> Message-ID: <54AED7D4.8030101@oracle.com> > How would the box be created in this case? Would it be a "view" over > the memory range covered by biggie inside the Foo instance? These would be invisible to you, just like 99% of the representational decisions the VM makes on your behalf with all sorts of other things (e.g., field layout). For some extreme cases, we expose tuning flags, but that is definitely not our default. > I'm not even sure it's a good idea (at least for v1) to make any > promises around tearing/atomicity of value types. Not an option. There are security implications around structure tearing that we can't brush under the rug. From vitalyd at gmail.com Thu Jan 8 23:48:53 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 18:48:53 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AED7D4.8030101@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <54AECC0D.8070703@oracle.com> <54AED7D4.8030101@oracle.com> Message-ID: Yes I fully understand that this would be invisible, but that's part of the "issue". It's funny you mention layout; see @Contended and ObjectLayout discussions for examples of where people care :). The problem is that people who are sensitive to performance (and that group is likely to care more about this feature than others) care about these "invisible" details. When you guys engineer the VM, do you care about how the OS implements certain things or brush it off as invisible? If you don't have or want to share the details now, that's fine too. Sent from my phone On Jan 8, 2015 2:17 PM, "Brian Goetz" wrote: > How would the box be created in this case? Would it be a "view" over >> the memory range covered by biggie inside the Foo instance? >> > > These would be invisible to you, just like 99% of the representational > decisions the VM makes on your behalf with all sorts of other things (e.g., > field layout). For some extreme cases, we expose tuning flags, but that is > definitely not our default. > > I'm not even sure it's a good idea (at least for v1) to make any >> promises around tearing/atomicity of value types. >> > > Not an option. There are security implications around structure tearing > that we can't brush under the rug. > > From john.r.rose at oracle.com Fri Jan 9 00:07:08 2015 From: john.r.rose at oracle.com (John Rose) Date: Thu, 8 Jan 2015 16:07:08 -0800 Subject: =?utf-8?Q?Re=3A_Value_types_-_compatibility_with_existing_?= =?utf-8?Q?=E2=80=9Cvalue_objects=E2=80=9D?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> Message-ID: <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> On Jan 8, 2015, at 8:59 AM, Vitaly Davidovich wrote: > >> I am not sure where you got this idea that somehow "too big" values would >> require a different syntax. However, its possible you're running on a >> machine with a very restricted register set, and a 4x4 matrix of longs is >> too big to pass by value, so it gets silently boxed, and you don't get all >> the performance you get with a 2x2 matrix. But this is no different from >> any other hardware effect (size or number of cache lines, etc.) > > > Why is # of registers a factor here? I'd expect some # of registers to be > used for passing some of the values (depending on calling convention), and > the rest passed on the stack. The values proposal touches on these points; see the paragraph beginning like this: > A "soft" restriction on values is that they are "not too large". http://cr.openjdk.java.net/~jrose/values/values-0.html > This is also the first I hear of VM reserving the right to transparently > box depending on some size threshold. Most of us have moments of surprise at what the VMs does! As Brian says, they have virtual instruction semantics to implement on the users behalf, and lots of freedom how to make it work. > My preference would be to simply use > the stack, and if you run out of stack, handle that the same way as today. > Clearly someone passing a giant value type around is not doing themselves > any good, but the automagic boxing behind the scenes doesn't sit well. There was a time before Java was widespread, when automatic storage management didn't "sit well" with most C/C++ programmers. (Perhaps it still doesn't, though now there is Boehm GC, etc.) In our present world, VMs manage storage automatically. A necessary cost of this is occasional errors when resources are exhausted. > What if boxing that giant value type causes an OOM? I don't think that > would be expected. It depends on what you mean by "expected", but part of the agreement between users and VMs is that the JVM is allowed to throw certain errors (subtype of VMError) when its automatic tactics fail. Such errors are rarely expected, except in the sense that you can expect the VM to throw such an error where other kinds of programs will crash or execute unpredictable paths through hand-written recovery code. See JVMS 6.3, 2.5.[23456], and 5.4 for a discussion of VM errors. We know there are users who work very hard to tweak their code to exclude such errors. It is a difficult task, since they are tweaking for behaviors that are not guaranteed by the spec., but are provided by the good qualities of a particular JVM implementation. JVM implementation and optimization techniques change over time, and you can expect behaviors to change along with them (staying in-spec), including subtle shifts in where the "red line" of resource exhaustion is for any given JVM workload. ? John From john.r.rose at oracle.com Fri Jan 9 00:18:02 2015 From: john.r.rose at oracle.com (John Rose) Date: Thu, 8 Jan 2015 16:18:02 -0800 Subject: =?utf-8?Q?Re=3A_Value_types_-_compatibility_with_existing_?= =?utf-8?Q?=E2=80=9Cvalue_objects=E2=80=9D?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> Message-ID: <3691095D-3DA5-42A3-8D1E-76BBCF0A4FF7@oracle.com> On Jan 8, 2015, at 10:09 AM, Paul Benedict wrote: > > Value types are "final" which means it won't be possible to subclass. > However, I am disappointed with this restriction because I would like to > use inheritance as an easy way of composing value types. Brian, what are > your thoughts about this? To me this sounds like disappointment with physics. Let me try to explain why. If we are designing data structures which can be efficiently packed into memory without pointers or headers, then you give stuff up, and pointer polymorphism is one of the things you give up. It's explicitly called out in the values doc: > The decision to limit or prohibit subclassing and subtyping of value > types is necessary to avoid pointer polymorphism. Abstract super > types (like the interface `Comparable`) are only considered because > they cannot be instantiated. For a discussion of pointer polymorphism (and several other things we are hashing out here) see the section titled "Life without pointers". It is an interesting design exercise to work out how to add back inheritance into value types, and it can be done (maybe later in Java 11 or 12?), but it is decidedly non-trivial given the design goals of values. Pointer polymorphism (what inheritance is built from) requires pointers and headers, and those are exactly what we are removing from values. As a design pattern, you can create fixed-sized values with variable-sized extensions referred to via pointers to normally polymorphic objects (with normal headers). This doesn't need JVM support, and may never. I think the most interesting answer to "what will I do with values that don't inherit" is to create design patterns in which fixed-sized values delegate variable behavior via their fields. Maybe the ultimate answer will be, as with inner classes, varargs, and lambdas, to watch how users hand-roll their patterns, and then add tasteful support for common patterns into the source-to-bytecode translation (javac, the JLS). ? John From vitalyd at gmail.com Fri Jan 9 00:29:39 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 19:29:39 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> Message-ID: Thanks John. I read that paragraph just now and do see mention of spilling to heap. However, the bulk of the paragraph talks about the intended use of value types, which I fully agree with. The register file is just an example of how one can best achieve performance by scalarizing the value type across registers - great, love it! However, I don't quite understand why you need to spill to heap and not restrict it to stack only. I know this is probably discussing a pathological case as I'd imagine the threshold you pick will not be hit by people actually writing performant code, so perhaps we don't need to discuss it at length. In terms of freedom of implementation, another thing I highly agree with. However, I'd like to have a bit more control in some cases. There are things the VM does that either I can't do reasonably or at all and I appreciate that (e.g. the various JIT optimizations around devirtualization as just one example). But, for some things I'd like to have more say :). Storage is one of them. I'm sure you guys know that there are people out there that either avoid the GC like a plague and/or take their data offheap. Using stack for temps is almost always going to be preferred over heap. Anything that we can do to facilitate that would be fantastic. Automatic storage is great when it's warranted, but it's a big hammer in some situations. P.S. I think GC still doesn't sit well with certain groups of people, thus some excitement about new languages like Rust and criticism of others (e.g. Go, D) that have it. Obviously I'm not saying java needs to abandon it, but there are folks building middleware/infra components where they'd like better facilities. Sent from my phone On Jan 8, 2015 7:07 PM, "John Rose" wrote: > On Jan 8, 2015, at 8:59 AM, Vitaly Davidovich wrote: > > > I am not sure where you got this idea that somehow "too big" values would > require a different syntax. However, its possible you're running on a > machine with a very restricted register set, and a 4x4 matrix of longs is > too big to pass by value, so it gets silently boxed, and you don't get all > the performance you get with a 2x2 matrix. But this is no different from > any other hardware effect (size or number of cache lines, etc.) > > > > Why is # of registers a factor here? I'd expect some # of registers to be > used for passing some of the values (depending on calling convention), and > the rest passed on the stack. > > > The values proposal touches on these points; see the paragraph beginning > like this: > > A "soft" restriction on values is that they are "not too large". > > http://cr.openjdk.java.net/~jrose/values/values-0.html > > This is also the first I hear of VM reserving the right to transparently > box depending on some size threshold. > > > Most of us have moments of surprise at what the VMs does! As Brian says, > they have virtual instruction semantics to implement on the users behalf, > and lots of freedom how to make it work. > > My preference would be to simply use > the stack, and if you run out of stack, handle that the same way as today. > Clearly someone passing a giant value type around is not doing themselves > any good, but the automagic boxing behind the scenes doesn't sit well. > > > There was a time before Java was widespread, when automatic storage > management didn't "sit well" with most C/C++ programmers. (Perhaps it > still doesn't, though now there is Boehm GC, etc.) In our present world, > VMs manage storage automatically. A necessary cost of this is occasional > errors when resources are exhausted. > > What if boxing that giant value type causes an OOM? I don't think that > would be expected. > > > It depends on what you mean by "expected", but part of the agreement > between users and VMs is that the JVM is allowed to throw certain errors > (subtype of VMError) when its automatic tactics fail. Such errors are > rarely expected, except in the sense that you can expect the VM to throw > such an error where other kinds of programs will crash or execute > unpredictable paths through hand-written recovery code. > > See JVMS 6.3, 2.5.[23456], and 5.4 for a discussion of VM errors. > > We know there are users who work very hard to tweak their code to exclude > such errors. It is a difficult task, since they are tweaking for behaviors > that are not guaranteed by the spec., but are provided by the good > qualities of a particular JVM implementation. JVM implementation and > optimization techniques change over time, and you can expect behaviors to > change along with them (staying in-spec), including subtle shifts in where > the "red line" of resource exhaustion is for any given JVM workload. > > ? John > From john.r.rose at oracle.com Fri Jan 9 00:37:57 2015 From: john.r.rose at oracle.com (John Rose) Date: Thu, 8 Jan 2015 16:37:57 -0800 Subject: =?utf-8?Q?Re=3A_Value_types_-_compatibility_with_existing_?= =?utf-8?Q?=E2=80=9Cvalue_objects=E2=80=9D?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> Message-ID: <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> On Jan 8, 2015, at 4:29 PM, Vitaly Davidovich wrote: > Thanks John. I read that paragraph just now and do see mention of spilling to heap. However, the bulk of the paragraph talks about the intended use of value types, which I fully agree with. The register file is just an example of how one can best achieve performance by scalarizing the value type across registers - great, love it! However, I don't quite understand why you need to spill to heap and not restrict it to stack only. I know this is probably discussing a pathological case as I'd imagine the threshold you pick will not be hit by people actually writing performant code, so perhaps we don't need to discuss it at length. > In terms of freedom of implementation, another thing I highly agree with. However, I'd like to have a bit more control in some cases. There are things the VM does that either I can't do reasonably or at all and I appreciate that (e.g. the various JIT optimizations around devirtualization as just one example). But, for some things I'd like to have more say :). Storage is one of them. I'm sure you guys know that there are people out there that either avoid the GC like a plague and/or take their data offheap. Using stack for temps is almost always going to be preferred over heap. Anything that we can do to facilitate that would be fantastic. Automatic storage is great when it's warranted, but it's a big hammer in some situations. > > P.S. I think GC still doesn't sit well with certain groups of people, thus some excitement about new languages like Rust and criticism of others (e.g. Go, D) that have it. Obviously I'm not saying java needs to abandon it, but there are folks building middleware/infra components where they'd like better facilities. > Thanks for the good comments, Vitaly. Yes, GC is a key value-add, and requires a whole team to keep fresh. Indeed we are looking at managing stack more cleverly also. A warning note: If you OOME by moving stuff onto stack, you increase the frequency of SOE! From vitalyd at gmail.com Fri Jan 9 00:59:30 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 19:59:30 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> Message-ID: Great, happy to hear you guys are exploring that. I can't tell you how much code I've seen and written that's just ugly because people are avoiding temp objects (e.g. pools, decomposing their would-be objects into primitives and then passing them around as arguments, not using collection iterators, using primitives and then adding a bunch of static methods to "make sense" of it, etc). There are lots of times I want to say "goddamit, I know what I'm doing - stop holding my hand and let me out of this sandbox!" :) Now, I know some of this comes with the territory but it just feels like we can do better. Value types have the potential to cure some of this substantially, and that's why it's a subject I'm particularly interested in (and I'm sure others as well). As for abusing stack space and getting SOE, no issues in my opinion - I can cure that without destroying the code. But that should be the preferred form of automatic storage management for ephemeral situations (and there are lots of them) and leave the heap for stuff that has unknown/long/shared/etc lifetime. So again, happy that you guys are exploring that. Sent from my phone On Jan 8, 2015 7:38 PM, "John Rose" wrote: > On Jan 8, 2015, at 4:29 PM, Vitaly Davidovich wrote: > > Thanks John. I read that paragraph just now and do see mention of > spilling to heap. However, the bulk of the paragraph talks about the > intended use of value types, which I fully agree with. The register file > is just an example of how one can best achieve performance by scalarizing > the value type across registers - great, love it! However, I don't quite > understand why you need to spill to heap and not restrict it to stack > only. I know this is probably discussing a pathological case as I'd > imagine the threshold you pick will not be hit by people actually writing > performant code, so perhaps we don't need to discuss it at length. > > In terms of freedom of implementation, another thing I highly agree > with. However, I'd like to have a bit more control in some cases. There > are things the VM does that either I can't do reasonably or at all and I > appreciate that (e.g. the various JIT optimizations around devirtualization > as just one example). But, for some things I'd like to have more say :). > Storage is one of them. I'm sure you guys know that there are people out > there that either avoid the GC like a plague and/or take their data > offheap. Using stack for temps is almost always going to be preferred over > heap. Anything that we can do to facilitate that would be fantastic. > Automatic storage is great when it's warranted, but it's a big hammer in > some situations. > > P.S. I think GC still doesn't sit well with certain groups of people, thus > some excitement about new languages like Rust and criticism of others (e.g. > Go, D) that have it. Obviously I'm not saying java needs to abandon it, > but there are folks building middleware/infra components where they'd like > better facilities. > > Thanks for the good comments, Vitaly. Yes, GC is a key value-add, and > requires a whole team to keep fresh. > > Indeed we are looking at managing stack more cleverly also. A warning > note: If you OOME by moving stuff onto stack, you increase the frequency > of SOE! > > From david.lloyd at redhat.com Fri Jan 9 01:57:19 2015 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 08 Jan 2015 19:57:19 -0600 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> Message-ID: <54AF357F.9060604@redhat.com> On 01/08/2015 06:37 PM, John Rose wrote: > On Jan 8, 2015, at 4:29 PM, Vitaly Davidovich wrote: >> Thanks John. I read that paragraph just now and do see mention of spilling to heap. However, the bulk of the paragraph talks about the intended use of value types, which I fully agree with. The register file is just an example of how one can best achieve performance by scalarizing the value type across registers - great, love it! However, I don't quite understand why you need to spill to heap and not restrict it to stack only. I know this is probably discussing a pathological case as I'd imagine the threshold you pick will not be hit by people actually writing performant code, so perhaps we don't need to discuss it at length. >> In terms of freedom of implementation, another thing I highly agree with. However, I'd like to have a bit more control in some cases. There are things the VM does that either I can't do reasonably or at all and I appreciate that (e.g. the various JIT optimizations around devirtualization as just one example). But, for some things I'd like to have more say :). Storage is one of them. I'm sure you guys know that there are people out there that either avoid the GC like a plague and/or take their data offheap. Using stack for temps is almost always going to be preferred over heap. Anything that we can do to facilitate that would be fantastic. Automatic storage is great when it's warranted, but it's a big hammer in some situations. >> >> P.S. I think GC still doesn't sit well with certain groups of people, thus some excitement about new languages like Rust and criticism of others (e.g. Go, D) that have it. Obviously I'm not saying java needs to abandon it, but there are folks building middleware/infra components where they'd like better facilities. >> > Thanks for the good comments, Vitaly. Yes, GC is a key value-add, and requires a whole team to keep fresh. > > Indeed we are looking at managing stack more cleverly also. A warning note: If you OOME by moving stuff onto stack, you increase the frequency of SOE! Worth mentioning that SOE is infinitely more recoverable than heap OOME. In the latter case (especially with complex systems, where the risk is ironically even higher of it happening) there's often not much you can do other than kill off the JVM and try again. But if you run out of stack, everything should unwind in a very predictable (and fast) manner. You could even automatically rebuild your thread pools with larger stack sizes fairly easily. -- - DML From vitalyd at gmail.com Fri Jan 9 02:02:37 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 21:02:37 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AF357F.9060604@redhat.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> Message-ID: I'm not sure that's true - what if right before the stack is expanded some mutation is performed that isn't undone upon SOE? I'd say both OOME and SOE are the types of "async" exceptions that are hard to recover from. I do, however, think it's easier to track down the cause of SOE than a heap OOME since you have one call stack to inspect whereas you may hit a heap OOME purely being a victim of someone else polluting it. Sent from my phone On Jan 8, 2015 8:57 PM, "David M. Lloyd" wrote: > On 01/08/2015 06:37 PM, John Rose wrote: > >> On Jan 8, 2015, at 4:29 PM, Vitaly Davidovich wrote: >> >>> Thanks John. I read that paragraph just now and do see mention of >>> spilling to heap. However, the bulk of the paragraph talks about the >>> intended use of value types, which I fully agree with. The register file >>> is just an example of how one can best achieve performance by scalarizing >>> the value type across registers - great, love it! However, I don't quite >>> understand why you need to spill to heap and not restrict it to stack >>> only. I know this is probably discussing a pathological case as I'd >>> imagine the threshold you pick will not be hit by people actually writing >>> performant code, so perhaps we don't need to discuss it at length. >>> In terms of freedom of implementation, another thing I highly agree >>> with. However, I'd like to have a bit more control in some cases. There >>> are things the VM does that either I can't do reasonably or at all and I >>> appreciate that (e.g. the various JIT optimizations around devirtualization >>> as just one example). But, for some things I'd like to have more say :). >>> Storage is one of them. I'm sure you guys know that there are people out >>> there that either avoid the GC like a plague and/or take their data >>> offheap. Using stack for temps is almost always going to be preferred over >>> heap. Anything that we can do to facilitate that would be fantastic. >>> Automatic storage is great when it's warranted, but it's a big hammer in >>> some situations. >>> >>> P.S. I think GC still doesn't sit well with certain groups of people, >>> thus some excitement about new languages like Rust and criticism of others >>> (e.g. Go, D) that have it. Obviously I'm not saying java needs to abandon >>> it, but there are folks building middleware/infra components where they'd >>> like better facilities. >>> >>> Thanks for the good comments, Vitaly. Yes, GC is a key value-add, and >> requires a whole team to keep fresh. >> >> Indeed we are looking at managing stack more cleverly also. A warning >> note: If you OOME by moving stuff onto stack, you increase the frequency >> of SOE! >> > > Worth mentioning that SOE is infinitely more recoverable than heap OOME. > In the latter case (especially with complex systems, where the risk is > ironically even higher of it happening) there's often not much you can do > other than kill off the JVM and try again. But if you run out of stack, > everything should unwind in a very predictable (and fast) manner. You > could even automatically rebuild your thread pools with larger stack sizes > fairly easily. > > -- > - DML > From twhitmore.nz at gmail.com Fri Jan 9 02:23:19 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Fri, 9 Jan 2015 15:23:19 +1300 Subject: 'Equals' in any-typed code: VERY BIG PROPOSAL In-Reply-To: References: Message-ID: Hi Vitaly, people, When we document how to write specializable classes, this should be language-level; not really relying on libraries. I feel 'Eq' deserves to be seriously considered as 1st class operator in the language. This can apply efficiently across whatever actual type, and well exposed for optimization. Static imports are OK as a workaround, but a majority of application developers out there will never use this technique. An 'eq' operator, on the other hand, they will use. My background is in commercial application development, as well as architecture, so I see these things. --------------- Separately, what's our story on static methods? How are calls to these going to be implemented without boxing? Regards, Thomas From david.lloyd at redhat.com Fri Jan 9 02:48:06 2015 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 08 Jan 2015 20:48:06 -0600 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> Message-ID: <54AF4166.5060700@redhat.com> I'm admittedly no expert on such things, but I'd hope that my method would never be entered unless there was stack space enough for its whole execution frame. On 01/08/2015 08:02 PM, Vitaly Davidovich wrote: > I'm not sure that's true - what if right before the stack is expanded > some mutation is performed that isn't undone upon SOE? I'd say both OOME > and SOE are the types of "async" exceptions that are hard to recover from. > > I do, however, think it's easier to track down the cause of SOE than a > heap OOME since you have one call stack to inspect whereas you may hit a > heap OOME purely being a victim of someone else polluting it. > > Sent from my phone > > On Jan 8, 2015 8:57 PM, "David M. Lloyd" > wrote: > > On 01/08/2015 06:37 PM, John Rose wrote: > > On Jan 8, 2015, at 4:29 PM, Vitaly Davidovich > wrote: > > Thanks John. I read that paragraph just now and do see > mention of spilling to heap. However, the bulk of the > paragraph talks about the intended use of value types, which > I fully agree with. The register file is just an example of > how one can best achieve performance by scalarizing the > value type across registers - great, love it! However, I > don't quite understand why you need to spill to heap and not > restrict it to stack only. I know this is probably > discussing a pathological case as I'd imagine the threshold > you pick will not be hit by people actually writing > performant code, so perhaps we don't need to discuss it at > length. > In terms of freedom of implementation, another thing I > highly agree with. However, I'd like to have a bit more > control in some cases. There are things the VM does that > either I can't do reasonably or at all and I appreciate that > (e.g. the various JIT optimizations around devirtualization > as just one example). But, for some things I'd like to have > more say :). Storage is one of them. I'm sure you guys > know that there are people out there that either avoid the > GC like a plague and/or take their data offheap. Using > stack for temps is almost always going to be preferred over > heap. Anything that we can do to facilitate that would be > fantastic. Automatic storage is great when it's warranted, > but it's a big hammer in some situations. > > P.S. I think GC still doesn't sit well with certain groups > of people, thus some excitement about new languages like > Rust and criticism of others (e.g. Go, D) that have it. > Obviously I'm not saying java needs to abandon it, but there > are folks building middleware/infra components where they'd > like better facilities. > > Thanks for the good comments, Vitaly. Yes, GC is a key > value-add, and requires a whole team to keep fresh. > > Indeed we are looking at managing stack more cleverly also. A > warning note: If you OOME by moving stuff onto stack, you > increase the frequency of SOE! > > > Worth mentioning that SOE is infinitely more recoverable than heap > OOME. In the latter case (especially with complex systems, where > the risk is ironically even higher of it happening) there's often > not much you can do other than kill off the JVM and try again. But > if you run out of stack, everything should unwind in a very > predictable (and fast) manner. You could even automatically rebuild > your thread pools with larger stack sizes fairly easily. > > -- > - DML > -- - DML From brian.goetz at oracle.com Fri Jan 9 02:49:12 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jan 2015 21:49:12 -0500 Subject: 'Equals' in any-typed code: VERY BIG PROPOSAL In-Reply-To: References: Message-ID: <54AF41A8.3060809@oracle.com> > Separately, what's our story on static methods? How are calls > to these going to be implemented without boxing? They're already implemented, so try compiling one and decompiling with javap -c. No boxing! From vitalyd at gmail.com Fri Jan 9 03:01:17 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 8 Jan 2015 22:01:17 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AF4166.5060700@redhat.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> <54AF4166.5060700@redhat.com> Message-ID: Silly but easy to see (I think) example: infinite recursion via method calling itself, no tail call optimization, and each invocation of this method mutates something that outlives this thread. JVM won't throw SOE until it bumps into the yellow zone (I think it's the yellow where the red is reserved for itself to have room to handle java SOE) and can't expand the stack any further. Sent from my phone On Jan 8, 2015 9:48 PM, "David M. Lloyd" wrote: > I'm admittedly no expert on such things, but I'd hope that my method would > never be entered unless there was stack space enough for its whole > execution frame. > > On 01/08/2015 08:02 PM, Vitaly Davidovich wrote: > >> I'm not sure that's true - what if right before the stack is expanded >> some mutation is performed that isn't undone upon SOE? I'd say both OOME >> and SOE are the types of "async" exceptions that are hard to recover from. >> >> I do, however, think it's easier to track down the cause of SOE than a >> heap OOME since you have one call stack to inspect whereas you may hit a >> heap OOME purely being a victim of someone else polluting it. >> >> Sent from my phone >> >> On Jan 8, 2015 8:57 PM, "David M. Lloyd" > > wrote: >> >> On 01/08/2015 06:37 PM, John Rose wrote: >> >> On Jan 8, 2015, at 4:29 PM, Vitaly Davidovich > > wrote: >> >> Thanks John. I read that paragraph just now and do see >> mention of spilling to heap. However, the bulk of the >> paragraph talks about the intended use of value types, which >> I fully agree with. The register file is just an example of >> how one can best achieve performance by scalarizing the >> value type across registers - great, love it! However, I >> don't quite understand why you need to spill to heap and not >> restrict it to stack only. I know this is probably >> discussing a pathological case as I'd imagine the threshold >> you pick will not be hit by people actually writing >> performant code, so perhaps we don't need to discuss it at >> length. >> In terms of freedom of implementation, another thing I >> highly agree with. However, I'd like to have a bit more >> control in some cases. There are things the VM does that >> either I can't do reasonably or at all and I appreciate that >> (e.g. the various JIT optimizations around devirtualization >> as just one example). But, for some things I'd like to have >> more say :). Storage is one of them. I'm sure you guys >> know that there are people out there that either avoid the >> GC like a plague and/or take their data offheap. Using >> stack for temps is almost always going to be preferred over >> heap. Anything that we can do to facilitate that would be >> fantastic. Automatic storage is great when it's warranted, >> but it's a big hammer in some situations. >> >> P.S. I think GC still doesn't sit well with certain groups >> of people, thus some excitement about new languages like >> Rust and criticism of others (e.g. Go, D) that have it. >> Obviously I'm not saying java needs to abandon it, but there >> are folks building middleware/infra components where they'd >> like better facilities. >> >> Thanks for the good comments, Vitaly. Yes, GC is a key >> value-add, and requires a whole team to keep fresh. >> >> Indeed we are looking at managing stack more cleverly also. A >> warning note: If you OOME by moving stuff onto stack, you >> increase the frequency of SOE! >> >> >> Worth mentioning that SOE is infinitely more recoverable than heap >> OOME. In the latter case (especially with complex systems, where >> the risk is ironically even higher of it happening) there's often >> not much you can do other than kill off the JVM and try again. But >> if you run out of stack, everything should unwind in a very >> predictable (and fast) manner. You could even automatically rebuild >> your thread pools with larger stack sizes fairly easily. >> >> -- >> - DML >> >> > -- > - DML > From david.holmes at oracle.com Fri Jan 9 05:52:31 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 09 Jan 2015 15:52:31 +1000 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: <54AF357F.9060604@redhat.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> Message-ID: <54AF6C9F.8020706@oracle.com> On 9/01/2015 11:57 AM, David M. Lloyd wrote: > On 01/08/2015 06:37 PM, John Rose wrote: >> On Jan 8, 2015, at 4:29 PM, Vitaly Davidovich wrote: >>> Thanks John. I read that paragraph just now and do see mention of >>> spilling to heap. However, the bulk of the paragraph talks about the >>> intended use of value types, which I fully agree with. The register >>> file is just an example of how one can best achieve performance by >>> scalarizing the value type across registers - great, love it! >>> However, I don't quite understand why you need to spill to heap and >>> not restrict it to stack only. I know this is probably discussing a >>> pathological case as I'd imagine the threshold you pick will not be >>> hit by people actually writing performant code, so perhaps we don't >>> need to discuss it at length. >>> In terms of freedom of implementation, another thing I highly agree >>> with. However, I'd like to have a bit more control in some cases. >>> There are things the VM does that either I can't do reasonably or at >>> all and I appreciate that (e.g. the various JIT optimizations around >>> devirtualization as just one example). But, for some things I'd like >>> to have more say :). Storage is one of them. I'm sure you guys know >>> that there are people out there that either avoid the GC like a >>> plague and/or take their data offheap. Using stack for temps is >>> almost always going to be preferred over heap. Anything that we can >>> do to facilitate that would be fantastic. Automatic storage is great >>> when it's warranted, but it's a big hammer in some situations. >>> >>> P.S. I think GC still doesn't sit well with certain groups of people, >>> thus some excitement about new languages like Rust and criticism of >>> others (e.g. Go, D) that have it. Obviously I'm not saying java >>> needs to abandon it, but there are folks building middleware/infra >>> components where they'd like better facilities. >>> >> Thanks for the good comments, Vitaly. Yes, GC is a key value-add, and >> requires a whole team to keep fresh. >> >> Indeed we are looking at managing stack more cleverly also. A warning >> note: If you OOME by moving stuff onto stack, you increase the >> frequency of SOE! > > Worth mentioning that SOE is infinitely more recoverable than heap OOME. > In the latter case (especially with complex systems, where the risk is > ironically even higher of it happening) there's often not much you can > do other than kill off the JVM and try again. But if you run out of > stack, everything should unwind in a very predictable (and fast) manner. > You could even automatically rebuild your thread pools with larger > stack sizes fairly easily. Going OT but I strongly disagree. At least people try to write code that accounts for potential OOME for explicit allocation sites (eg allocate first then modify local state that affects invariants). I've never seen any code that anticipates that any call might trigger a SOE. David H. From peter.levart at gmail.com Fri Jan 9 07:14:45 2015 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 09 Jan 2015 08:14:45 +0100 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: <54AF6C9F.8020706@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> <54AF6C9F.8020706@oracle.com> Message-ID: <54AF7FE5.2070306@gmail.com> On 01/09/2015 06:52 AM, David Holmes wrote: > On 9/01/2015 11:57 AM, David M. Lloyd wrote: >> On 01/08/2015 06:37 PM, John Rose wrote: >>> On Jan 8, 2015, at 4:29 PM, Vitaly Davidovich >>> wrote: >>>> Thanks John. I read that paragraph just now and do see mention of >>>> spilling to heap. However, the bulk of the paragraph talks about the >>>> intended use of value types, which I fully agree with. The register >>>> file is just an example of how one can best achieve performance by >>>> scalarizing the value type across registers - great, love it! >>>> However, I don't quite understand why you need to spill to heap and >>>> not restrict it to stack only. I know this is probably discussing a >>>> pathological case as I'd imagine the threshold you pick will not be >>>> hit by people actually writing performant code, so perhaps we don't >>>> need to discuss it at length. >>>> In terms of freedom of implementation, another thing I highly agree >>>> with. However, I'd like to have a bit more control in some cases. >>>> There are things the VM does that either I can't do reasonably or at >>>> all and I appreciate that (e.g. the various JIT optimizations around >>>> devirtualization as just one example). But, for some things I'd like >>>> to have more say :). Storage is one of them. I'm sure you guys know >>>> that there are people out there that either avoid the GC like a >>>> plague and/or take their data offheap. Using stack for temps is >>>> almost always going to be preferred over heap. Anything that we can >>>> do to facilitate that would be fantastic. Automatic storage is great >>>> when it's warranted, but it's a big hammer in some situations. >>>> >>>> P.S. I think GC still doesn't sit well with certain groups of people, >>>> thus some excitement about new languages like Rust and criticism of >>>> others (e.g. Go, D) that have it. Obviously I'm not saying java >>>> needs to abandon it, but there are folks building middleware/infra >>>> components where they'd like better facilities. >>>> >>> Thanks for the good comments, Vitaly. Yes, GC is a key value-add, and >>> requires a whole team to keep fresh. >>> >>> Indeed we are looking at managing stack more cleverly also. A warning >>> note: If you OOME by moving stuff onto stack, you increase the >>> frequency of SOE! >> >> Worth mentioning that SOE is infinitely more recoverable than heap OOME. >> In the latter case (especially with complex systems, where the risk is >> ironically even higher of it happening) there's often not much you can >> do other than kill off the JVM and try again. But if you run out of >> stack, everything should unwind in a very predictable (and fast) manner. >> You could even automatically rebuild your thread pools with larger >> stack sizes fairly easily. > > Going OT but I strongly disagree. At least people try to write code > that accounts for potential OOME for explicit allocation sites (eg > allocate first then modify local state that affects invariants). I've > never seen any code that anticipates that any call might trigger a SOE. > > David H. > Another point to consider is that heap is shared among all threads and stack space is allocated per-thread. If peak stack utilization increases with value types in some threads and as a consequence user increases max. stack size for the whole JVM, a lot of space is wasted in many threads that don't need that much stack space. There are various trade-offs here. Peter From forax at univ-mlv.fr Fri Jan 9 07:55:18 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 09 Jan 2015 08:55:18 +0100 Subject: Why JVM can not type infer type params but javac can do? In-Reply-To: References: <54AE75BD.1080107@univ-mlv.fr> Message-ID: <54AF8966.1030202@univ-mlv.fr> On 01/08/2015 01:32 PM, Ali Ebrahimi wrote: > I mean we can do better job than today by some hints and already > unused info by VM. > One first step: disallowing raw types by java9/10. you can not do that easily because the actual JLS introduces raw type even if the user code doesn't explicitly write one. List list = ... list.getClass() is inferred as Class :( > And some form of http://www.google.com/patents/US7810077 while the idea of trying to infer the type argument from the bytecode is interesting, I've trouble to see how runtime information can be used for finding type arguments given that if you have found more than one runtime class for a type argument it can be either an error or the type argument is a wildcard. regards, R?mi > > On Thu, Jan 8, 2015 at 3:49 PM, Remi Forax > wrote: > > Do you really want to enhance a simple VM checkcast to check > subtyping relationship between wildcards at runtime ? > > R?mi > > > On 01/08/2015 09:54 AM, Ali Ebrahimi wrote: > > Hi, maybe this is stupid question, but want to ask and what is > requirements > for this? > > Array instantiation sample: > public T[] newArray(int){ > return new T[0]; > } > > even for non-any T: > public T[] newArray(int){ > return new T[0]; > } > > With thousands number of optimization and inlining mechanics > equipped in > jvm, What is the reasoning behind not allowing to jvm to be > aware of > generics info emitted in classfilles? > > Why jvm should not able do following inlining? > > String[] strs = newArray(int) => String[] strs = new String[0]; > I mean even in jitted code. > > I think some time ago I read some papers in similar area. > > > > > > -- > > Best Regards, > Ali Ebrahimi From vitalyd at gmail.com Fri Jan 9 12:10:07 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 9 Jan 2015 07:10:07 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AF7FE5.2070306@gmail.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> <54AF6C9F.8020706@oracle.com> <54AF7FE5.2070306@gmail.com> Message-ID: Stack space is usually reserved but not committed; this means your virtual memory goes up but your resident/working set doesn't really (assuming not every thread hits the same peak); on 64bit systems this is not an issue. C/C++ programs use stack heavily, including allocating large temp buffers, and I've yet to see anyone it's a problem (in fact, quite the opposite - easy use of stack is one of the touted strengths of these languages when these debates come up on various websites). At any rate, nobody writing sane value types (even largish ones) will blow their stack space. You're more likely to do that via recursion, I think. Sent from my phone On Jan 9, 2015 2:15 AM, "Peter Levart" wrote: > On 01/09/2015 06:52 AM, David Holmes wrote: > >> On 9/01/2015 11:57 AM, David M. Lloyd wrote: >> >>> On 01/08/2015 06:37 PM, John Rose wrote: >>> >>>> On Jan 8, 2015, at 4:29 PM, Vitaly Davidovich >>>> wrote: >>>> >>>>> Thanks John. I read that paragraph just now and do see mention of >>>>> spilling to heap. However, the bulk of the paragraph talks about the >>>>> intended use of value types, which I fully agree with. The register >>>>> file is just an example of how one can best achieve performance by >>>>> scalarizing the value type across registers - great, love it! >>>>> However, I don't quite understand why you need to spill to heap and >>>>> not restrict it to stack only. I know this is probably discussing a >>>>> pathological case as I'd imagine the threshold you pick will not be >>>>> hit by people actually writing performant code, so perhaps we don't >>>>> need to discuss it at length. >>>>> In terms of freedom of implementation, another thing I highly agree >>>>> with. However, I'd like to have a bit more control in some cases. >>>>> There are things the VM does that either I can't do reasonably or at >>>>> all and I appreciate that (e.g. the various JIT optimizations around >>>>> devirtualization as just one example). But, for some things I'd like >>>>> to have more say :). Storage is one of them. I'm sure you guys know >>>>> that there are people out there that either avoid the GC like a >>>>> plague and/or take their data offheap. Using stack for temps is >>>>> almost always going to be preferred over heap. Anything that we can >>>>> do to facilitate that would be fantastic. Automatic storage is great >>>>> when it's warranted, but it's a big hammer in some situations. >>>>> >>>>> P.S. I think GC still doesn't sit well with certain groups of people, >>>>> thus some excitement about new languages like Rust and criticism of >>>>> others (e.g. Go, D) that have it. Obviously I'm not saying java >>>>> needs to abandon it, but there are folks building middleware/infra >>>>> components where they'd like better facilities. >>>>> >>>>> Thanks for the good comments, Vitaly. Yes, GC is a key value-add, and >>>> requires a whole team to keep fresh. >>>> >>>> Indeed we are looking at managing stack more cleverly also. A warning >>>> note: If you OOME by moving stuff onto stack, you increase the >>>> frequency of SOE! >>>> >>> >>> Worth mentioning that SOE is infinitely more recoverable than heap OOME. >>> In the latter case (especially with complex systems, where the risk is >>> ironically even higher of it happening) there's often not much you can >>> do other than kill off the JVM and try again. But if you run out of >>> stack, everything should unwind in a very predictable (and fast) manner. >>> You could even automatically rebuild your thread pools with larger >>> stack sizes fairly easily. >>> >> >> Going OT but I strongly disagree. At least people try to write code that >> accounts for potential OOME for explicit allocation sites (eg allocate >> first then modify local state that affects invariants). I've never seen any >> code that anticipates that any call might trigger a SOE. >> >> David H. >> >> > Another point to consider is that heap is shared among all threads and > stack space is allocated per-thread. If peak stack utilization increases > with value types in some threads and as a consequence user increases max. > stack size for the whole JVM, a lot of space is wasted in many threads that > don't need that much stack space. There are various trade-offs here. > > Peter > > From vitalyd at gmail.com Fri Jan 9 12:13:07 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 9 Jan 2015 07:13:07 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AF6C9F.8020706@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> <54AF6C9F.8020706@oracle.com> Message-ID: I've seen this too, but only a handful of times. For all practical purposes, it's very difficult to write a java program that's hardened against memory exhaustion (this is arguably where something like C has it easier), heap or stack. Sent from my phone On Jan 9, 2015 12:52 AM, "David Holmes" wrote: > On 9/01/2015 11:57 AM, David M. Lloyd wrote: > >> On 01/08/2015 06:37 PM, John Rose wrote: >> >>> On Jan 8, 2015, at 4:29 PM, Vitaly Davidovich wrote: >>> >>>> Thanks John. I read that paragraph just now and do see mention of >>>> spilling to heap. However, the bulk of the paragraph talks about the >>>> intended use of value types, which I fully agree with. The register >>>> file is just an example of how one can best achieve performance by >>>> scalarizing the value type across registers - great, love it! >>>> However, I don't quite understand why you need to spill to heap and >>>> not restrict it to stack only. I know this is probably discussing a >>>> pathological case as I'd imagine the threshold you pick will not be >>>> hit by people actually writing performant code, so perhaps we don't >>>> need to discuss it at length. >>>> In terms of freedom of implementation, another thing I highly agree >>>> with. However, I'd like to have a bit more control in some cases. >>>> There are things the VM does that either I can't do reasonably or at >>>> all and I appreciate that (e.g. the various JIT optimizations around >>>> devirtualization as just one example). But, for some things I'd like >>>> to have more say :). Storage is one of them. I'm sure you guys know >>>> that there are people out there that either avoid the GC like a >>>> plague and/or take their data offheap. Using stack for temps is >>>> almost always going to be preferred over heap. Anything that we can >>>> do to facilitate that would be fantastic. Automatic storage is great >>>> when it's warranted, but it's a big hammer in some situations. >>>> >>>> P.S. I think GC still doesn't sit well with certain groups of people, >>>> thus some excitement about new languages like Rust and criticism of >>>> others (e.g. Go, D) that have it. Obviously I'm not saying java >>>> needs to abandon it, but there are folks building middleware/infra >>>> components where they'd like better facilities. >>>> >>>> Thanks for the good comments, Vitaly. Yes, GC is a key value-add, and >>> requires a whole team to keep fresh. >>> >>> Indeed we are looking at managing stack more cleverly also. A warning >>> note: If you OOME by moving stuff onto stack, you increase the >>> frequency of SOE! >>> >> >> Worth mentioning that SOE is infinitely more recoverable than heap OOME. >> In the latter case (especially with complex systems, where the risk is >> ironically even higher of it happening) there's often not much you can >> do other than kill off the JVM and try again. But if you run out of >> stack, everything should unwind in a very predictable (and fast) manner. >> You could even automatically rebuild your thread pools with larger >> stack sizes fairly easily. >> > > Going OT but I strongly disagree. At least people try to write code that > accounts for potential OOME for explicit allocation sites (eg allocate > first then modify local state that affects invariants). I've never seen any > code that anticipates that any call might trigger a SOE. > > David H. > > From simon at ochsenreither.de Fri Jan 9 13:40:43 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Fri, 09 Jan 2015 14:40:43 +0100 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: References: Message-ID: <54AFDA5B.7040804@ochsenreither.de> > To answering a few questions: > >> a) What are the actual semantics? > > For ref-types: > x eq y --> (x != null && x.equals(y)) > For value-types: > x eq y --> (x == y) What's wrong with giving == and != a sensible implementation for value types as mentioned earlier? This would avoid introducing a new concept and would not require rewriting every equals/contains/... method in existence. >> b) How is this different to the approaches already outlines earlier? > > It provides a basic logical building-block (equality check) at the > language level, in a way that works both for primitives/valuetypes and > for nullable references. It addresses a very big long-standing pain > point in Java application development. It avoids writing/ or encouraging > "Duck.woof()" style-hacks -- null checks -- into a domain that does not > uniformly support them. Isn't this what was proposed earlier, just with == instead of introducing a new operator? From simon at ochsenreither.de Fri Jan 9 14:01:59 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Fri, 09 Jan 2015 15:01:59 +0100 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: <54AF357F.9060604@redhat.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> Message-ID: <54AFDF57.9000003@ochsenreither.de> > Worth mentioning that SOE is infinitely more recoverable than heap OOME. > In the latter case (especially with complex systems, where the risk is > ironically even higher of it happening) there's often not much you can > do other than kill off the JVM and try again. But if you run out of > stack, everything should unwind in a very predictable (and fast) manner. > You could even automatically rebuild your thread pools with larger > stack sizes fairly easily. Your VM might be broken after both. SOE is definitely not recoverable. Yes, it might work, but no guarantees it does. Money quote: "You will find occurrences of StackOverflowError leaving locks locked, monitors acquired, and "finally" block not invoked, even if JVM doesn't crash." Maybe things have changed since then? From david.lloyd at redhat.com Fri Jan 9 14:08:08 2015 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 09 Jan 2015 08:08:08 -0600 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: <54AF6C9F.8020706@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> <54AF6C9F.8020706@oracle.com> Message-ID: <54AFE0C8.5000807@redhat.com> On 01/08/2015 11:52 PM, David Holmes wrote: > On 9/01/2015 11:57 AM, David M. Lloyd wrote: >> Worth mentioning that SOE is infinitely more recoverable than heap OOME. >> In the latter case (especially with complex systems, where the risk is >> ironically even higher of it happening) there's often not much you can >> do other than kill off the JVM and try again. But if you run out of >> stack, everything should unwind in a very predictable (and fast) manner. >> You could even automatically rebuild your thread pools with larger >> stack sizes fairly easily. > > Going OT but I strongly disagree. At least people try to write code that > accounts for potential OOME for explicit allocation sites (eg allocate > first then modify local state that affects invariants). I've never seen > any code that anticipates that any call might trigger a SOE. I agree that not much code *does* account for SOE. But, it's often (usually? always?) *possible* to do so, as opposed to OOME due to (say) GC overhead which is nearly always a complete failure condition for the JVM (allocating, say, a single big array is really not the common case here IMO). An SOE doesn't affect the functionality of the JVM outside of the thread that experiences it the way that your typical OOME does. Also, having never personally experienced incorrect program operation due to SOE (as opposed to having experienced various JVM failures due to OOME on many occasions), I'm willing to broadly say that it's usually not a problem in practice, even if few (if any) programmers actively try to account for it. -- - DML From david.lloyd at redhat.com Fri Jan 9 14:10:02 2015 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 09 Jan 2015 08:10:02 -0600 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: <54AFDF57.9000003@ochsenreither.de> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> <54AFDF57.9000003@ochsenreither.de> Message-ID: <54AFE13A.7040000@redhat.com> On 01/09/2015 08:01 AM, Simon Ochsenreither wrote: >> Worth mentioning that SOE is infinitely more recoverable than heap OOME. >> In the latter case (especially with complex systems, where the risk is >> ironically even higher of it happening) there's often not much you can >> do other than kill off the JVM and try again. But if you run out of >> stack, everything should unwind in a very predictable (and fast) manner. >> You could even automatically rebuild your thread pools with larger >> stack sizes fairly easily. > > Your VM might be broken after both. SOE is definitely not recoverable. > Yes, it might work, but no guarantees it does. > > Money quote: "You will find occurrences of StackOverflowError leaving > locks locked, monitors acquired, and "finally" block not invoked, even > if JVM doesn't crash." Well, if this is true then most of my assumptions are plain wrong. Also, that would really be awful. :-) -- - DML From richard.warburton at gmail.com Fri Jan 9 14:11:28 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Fri, 9 Jan 2015 14:11:28 +0000 Subject: Access Modifier Issues Message-ID: Hi gents, Not sure if its a high priority to fix the access modifier problems at the moment, but there still seems to be a bug with Peter Levart's default hack which I didn't see reported in the other thread. public class Default { private T value; private Default() {} public static T value() { return new Default().value; } public static void main(String[] args) { int i = Default.value(); System.out.println(i); long l = Default.value(); System.out.println(l); Object o = Default.value(); System.out.println(o); } } When running: Specializing method Default$value${0=I}.value()Ljava/lang/Object; with class=[] and method=[I] Specializing Default${0=I}; searching for Default.class (not found) Specializing Default${0=I}; searching for Default.class (found) Exception in thread "main" java.lang.IllegalAccessError: tried to access method Default${0=I}.()V from class Default$value${0=I}/793589513 at Default$value${0=I}/793589513.value(Default.java:8) at Default.main(Default.java:12) Looks like its workaround-able by making both the constructor and value field public. I think this also exposes something I don't understand here. My impression was that "Default${0=I}" was the specialisation of the class Default s.t. its first type parameter is an int. So what is "Default$value${0=I}"? regards, Richard Warburton http://insightfullogic.com @RichardWarburto From vitalyd at gmail.com Fri Jan 9 14:14:52 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 9 Jan 2015 09:14:52 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AFDF57.9000003@ochsenreither.de> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> <54AFDF57.9000003@ochsenreither.de> Message-ID: I think some of that true. For example, a lot of lock code looks like: lock.lock(); try { ... } finally { lock.unlock(); } If the lock implementation causes SOE or OOM as part of lock() and yet acquires the lock in the process, then yes, you're hosed. This is a quality of implementation issue. Someone can also abort the thread right after the lock is acquired but before you enter the try region. Thankfully, people are discouraged from doing that and you don't tend to see this in the wild. In terms of finally blocks not being executed, I don't think that's true (modulo JVM crashing) if the SOE was thrown from within a try/catch block. The JVM intentionally reserves some space at the tail of the stack to leave room for itself to initiate unwinding. On Fri, Jan 9, 2015 at 9:01 AM, Simon Ochsenreither wrote: > > Worth mentioning that SOE is infinitely more recoverable than heap OOME. > > In the latter case (especially with complex systems, where the risk is > > ironically even higher of it happening) there's often not much you can > > do other than kill off the JVM and try again. But if you run out of > > stack, everything should unwind in a very predictable (and fast) manner. > > You could even automatically rebuild your thread pools with larger > > stack sizes fairly easily. > > Your VM might be broken after both. SOE is definitely not recoverable. > Yes, it might work, but no guarantees it does. > > Money quote: "You will find occurrences of StackOverflowError leaving > locks locked, monitors acquired, and "finally" block not invoked, even > if JVM doesn't crash." > > Maybe things have changed since then? > From vitalyd at gmail.com Fri Jan 9 14:18:54 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 9 Jan 2015 09:18:54 -0500 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> <54AFDF57.9000003@ochsenreither.de> Message-ID: And by the way, .NET had a similar problem. What they did was change their lock APIs to indicate whether the lock was obtained or not using a by-ref boolean. So your code would now look like (pseudo java code assuming by-ref passing capability): boolean locked = false; try { lock.lock(ref locked); } finally { if (locked) { lock.unlock(); } } This way you can get yourself inside the protected try region when performing the action. On Fri, Jan 9, 2015 at 9:14 AM, Vitaly Davidovich wrote: > I think some of that true. For example, a lot of lock code looks like: > > lock.lock(); > try { > ... > } finally { > lock.unlock(); > } > > If the lock implementation causes SOE or OOM as part of lock() and yet > acquires the lock in the process, then yes, you're hosed. This is a > quality of implementation issue. Someone can also abort the thread right > after the lock is acquired but before you enter the try region. > Thankfully, people are discouraged from doing that and you don't tend to > see this in the wild. > > In terms of finally blocks not being executed, I don't think that's true > (modulo JVM crashing) if the SOE was thrown from within a try/catch block. > The JVM intentionally reserves some space at the tail of the stack to leave > room for itself to initiate unwinding. > > > > > On Fri, Jan 9, 2015 at 9:01 AM, Simon Ochsenreither < > simon at ochsenreither.de> wrote: > >> > Worth mentioning that SOE is infinitely more recoverable than heap OOME. >> > In the latter case (especially with complex systems, where the risk is >> > ironically even higher of it happening) there's often not much you can >> > do other than kill off the JVM and try again. But if you run out of >> > stack, everything should unwind in a very predictable (and fast) manner. >> > You could even automatically rebuild your thread pools with larger >> > stack sizes fairly easily. >> >> Your VM might be broken after both. SOE is definitely not recoverable. >> Yes, it might work, but no guarantees it does. >> >> Money quote: "You will find occurrences of StackOverflowError leaving >> locks locked, monitors acquired, and "finally" block not invoked, even >> if JVM doesn't crash." >> >> Maybe things have changed since then? >> > > From peter.levart at gmail.com Fri Jan 9 14:47:46 2015 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 09 Jan 2015 15:47:46 +0100 Subject: Access Modifier Issues In-Reply-To: References: Message-ID: <54AFEA12.1030109@gmail.com> On 01/09/2015 03:11 PM, Richard Warburton wrote: > Hi gents, > > Not sure if its a high priority to fix the access modifier problems at the > moment, but there still seems to be a bug with Peter Levart's default hack > which I didn't see reported in the other thread. Yes, Maurizio already warned about currently buggy access to private members from specialized methods. I think you can work-around by using package-private access for the time being. > > public class Default { > > private T value; > > private Default() {} > > public static T value() { > return new Default().value; > } > > public static void main(String[] args) { > int i = Default.value(); > System.out.println(i); > long l = Default.value(); > System.out.println(l); > Object o = Default.value(); > System.out.println(o); > } > > } > > When running: > > Specializing method Default$value${0=I}.value()Ljava/lang/Object; with > class=[] and method=[I] > Specializing Default${0=I}; searching for Default.class (not found) > Specializing Default${0=I}; searching for Default.class (found) > Exception in thread "main" java.lang.IllegalAccessError: tried to access > method Default${0=I}.()V from class Default$value${0=I}/793589513 > at Default$value${0=I}/793589513.value(Default.java:8) > at Default.main(Default.java:12) > > Looks like its workaround-able by making both the constructor and value > field public. > > I think this also exposes something I don't understand here. My impression > was that "Default${0=I}" was the specialisation of the class Default s.t. > its first type parameter is an int. So what is "Default$value${0=I}"? I think Default$value${0=I}/793589513 is a special (VM-annonymous) class, containing the specialization of static method Default.value() for T=int. The problem seems to be that private constructor of specialized class Default for T=int is not accessible from it. The specialized static method does have access to private members of nonspecialized class Default (since this is the host class of it's VM-anonymous class where it is defined), but not to the specialized Default class for T=int which is a separate class. Perhaps something similar to how outer class has access to private members of inner classes and vice-versa would have to be devised among non-specialized classes and their specializations. Regards, Peter > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto From maurizio.cimadamore at oracle.com Fri Jan 9 15:11:05 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 09 Jan 2015 15:11:05 +0000 Subject: Access Modifier Issues In-Reply-To: <54AFEA12.1030109@gmail.com> References: <54AFEA12.1030109@gmail.com> Message-ID: <54AFEF89.1060000@oracle.com> On 09/01/15 14:47, Peter Levart wrote: > On 01/09/2015 03:11 PM, Richard Warburton wrote: >> Hi gents, >> >> Not sure if its a high priority to fix the access modifier problems >> at the >> moment, but there still seems to be a bug with Peter Levart's default >> hack >> which I didn't see reported in the other thread. > > Yes, Maurizio already warned about currently buggy access to private > members from specialized methods. I think you can work-around by using > package-private access for the time being. Exact. The underlying issue is a thorny one: the compiler is organized in 'phases', roughly as follows: source -> parse -> symbol enter -> type-checking -> flow analysis -> erasure -> lambda translation -> misc desugaring -> bytecode With specialization, the pipeline has been modified, so that an extra 'specialize' step is added before erasure takes place (it has to be like that, as we need to save full generic info in the AST). The problem is that inner classes, accessibility and all desugaring is done in a later step (misc desugaring above) - meaning that for all the code generated there, there would be no way to generate the magic specialization attributes. Once you wrap your head around that, it's typically easy to fix the code in order to make it working again (i.e. in this case, tweak access) - of course this problem will be addressed in a more general way. Maurizio > >> >> public class Default { >> >> private T value; >> >> private Default() {} >> >> public static T value() { >> return new Default().value; >> } >> >> public static void main(String[] args) { >> int i = Default.value(); >> System.out.println(i); >> long l = Default.value(); >> System.out.println(l); >> Object o = Default.value(); >> System.out.println(o); >> } >> >> } >> >> When running: >> >> Specializing method Default$value${0=I}.value()Ljava/lang/Object; with >> class=[] and method=[I] >> Specializing Default${0=I}; searching for Default.class (not found) >> Specializing Default${0=I}; searching for Default.class (found) >> Exception in thread "main" java.lang.IllegalAccessError: tried to access >> method Default${0=I}.()V from class Default$value${0=I}/793589513 >> at Default$value${0=I}/793589513.value(Default.java:8) >> at Default.main(Default.java:12) >> >> Looks like its workaround-able by making both the constructor and value >> field public. >> >> I think this also exposes something I don't understand here. My >> impression >> was that "Default${0=I}" was the specialisation of the class Default >> s.t. >> its first type parameter is an int. So what is "Default$value${0=I}"? > > I think Default$value${0=I}/793589513 is a special (VM-annonymous) > class, containing the specialization of static method Default.value() > for T=int. > > The problem seems to be that private constructor of specialized class > Default for T=int is not accessible from it. The specialized static > method does have access to private members of nonspecialized class > Default (since this is the host class of it's VM-anonymous class where > it is defined), but not to the specialized Default class for T=int > which is a separate class. Perhaps something similar to how outer > class has access to private members of inner classes and vice-versa > would have to be devised among non-specialized classes and their > specializations. > > Regards, Peter > >> >> regards, >> >> Richard Warburton >> >> http://insightfullogic.com >> @RichardWarburto > From peter.levart at gmail.com Fri Jan 9 16:09:45 2015 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 09 Jan 2015 17:09:45 +0100 Subject: Access Modifier Issues In-Reply-To: <54AFEF89.1060000@oracle.com> References: <54AFEA12.1030109@gmail.com> <54AFEF89.1060000@oracle.com> Message-ID: <54AFFD49.9070007@gmail.com> On 01/09/2015 04:11 PM, Maurizio Cimadamore wrote: > > On 09/01/15 14:47, Peter Levart wrote: >> On 01/09/2015 03:11 PM, Richard Warburton wrote: >>> Hi gents, >>> >>> Not sure if its a high priority to fix the access modifier problems >>> at the >>> moment, but there still seems to be a bug with Peter Levart's >>> default hack >>> which I didn't see reported in the other thread. >> >> Yes, Maurizio already warned about currently buggy access to private >> members from specialized methods. I think you can work-around by >> using package-private access for the time being. > Exact. The underlying issue is a thorny one: the compiler is organized > in 'phases', roughly as follows: > > source -> parse -> symbol enter -> type-checking -> flow analysis -> > erasure -> lambda translation -> misc desugaring -> bytecode > > With specialization, the pipeline has been modified, so that an extra > 'specialize' step is added before erasure takes place (it has to be > like that, as we need to save full generic info in the AST). > The problem is that inner classes, accessibility and all desugaring is > done in a later step (misc desugaring above) - meaning that for all > the code generated there, there would be no way to generate the magic > specialization attributes. Once you wrap your head around that, it's > typically easy to fix the code in order to make it working again (i.e. > in this case, tweak access) - of course this problem will be addressed > in a more general way. > > Maurizio Hm, currently there seem to be two ways of triggering specialization. First way is by referencing classes with specialy-crafted names (SomeClass${0=?}) and ClassLoader then contains a hook to dispatch to the specialization logic. The second is used for invoking specialized generic methods, which works with invokedynamic and bootstrap methods. Would theoretically be possible to replace the 1st way (special class names) with the invokedynamic too? Would it be practical? Would it solve access problems? Regards, Peter >> >>> >>> public class Default { >>> >>> private T value; >>> >>> private Default() {} >>> >>> public static T value() { >>> return new Default().value; >>> } >>> >>> public static void main(String[] args) { >>> int i = Default.value(); >>> System.out.println(i); >>> long l = Default.value(); >>> System.out.println(l); >>> Object o = Default.value(); >>> System.out.println(o); >>> } >>> >>> } >>> >>> When running: >>> >>> Specializing method Default$value${0=I}.value()Ljava/lang/Object; with >>> class=[] and method=[I] >>> Specializing Default${0=I}; searching for Default.class (not found) >>> Specializing Default${0=I}; searching for Default.class (found) >>> Exception in thread "main" java.lang.IllegalAccessError: tried to >>> access >>> method Default${0=I}.()V from class Default$value${0=I}/793589513 >>> at Default$value${0=I}/793589513.value(Default.java:8) >>> at Default.main(Default.java:12) >>> >>> Looks like its workaround-able by making both the constructor and value >>> field public. >>> >>> I think this also exposes something I don't understand here. My >>> impression >>> was that "Default${0=I}" was the specialisation of the class Default >>> s.t. >>> its first type parameter is an int. So what is "Default$value${0=I}"? >> >> I think Default$value${0=I}/793589513 is a special (VM-annonymous) >> class, containing the specialization of static method Default.value() >> for T=int. >> >> The problem seems to be that private constructor of specialized class >> Default for T=int is not accessible from it. The specialized static >> method does have access to private members of nonspecialized class >> Default (since this is the host class of it's VM-anonymous class >> where it is defined), but not to the specialized Default class for >> T=int which is a separate class. Perhaps something similar to how >> outer class has access to private members of inner classes and >> vice-versa would have to be devised among non-specialized classes and >> their specializations. >> >> Regards, Peter >> >>> >>> regards, >>> >>> Richard Warburton >>> >>> http://insightfullogic.com >>> @RichardWarburto >> > From maurizio.cimadamore at oracle.com Fri Jan 9 16:15:04 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 09 Jan 2015 16:15:04 +0000 Subject: Access Modifier Issues In-Reply-To: <54AFFD49.9070007@gmail.com> References: <54AFEA12.1030109@gmail.com> <54AFEF89.1060000@oracle.com> <54AFFD49.9070007@gmail.com> Message-ID: <54AFFE88.7070203@oracle.com> On 09/01/15 16:09, Peter Levart wrote: > On 01/09/2015 04:11 PM, Maurizio Cimadamore wrote: >> >> On 09/01/15 14:47, Peter Levart wrote: >>> On 01/09/2015 03:11 PM, Richard Warburton wrote: >>>> Hi gents, >>>> >>>> Not sure if its a high priority to fix the access modifier problems >>>> at the >>>> moment, but there still seems to be a bug with Peter Levart's >>>> default hack >>>> which I didn't see reported in the other thread. >>> >>> Yes, Maurizio already warned about currently buggy access to private >>> members from specialized methods. I think you can work-around by >>> using package-private access for the time being. >> Exact. The underlying issue is a thorny one: the compiler is >> organized in 'phases', roughly as follows: >> >> source -> parse -> symbol enter -> type-checking -> flow analysis -> >> erasure -> lambda translation -> misc desugaring -> bytecode >> >> With specialization, the pipeline has been modified, so that an extra >> 'specialize' step is added before erasure takes place (it has to be >> like that, as we need to save full generic info in the AST). >> The problem is that inner classes, accessibility and all desugaring >> is done in a later step (misc desugaring above) - meaning that for >> all the code generated there, there would be no way to generate the >> magic specialization attributes. Once you wrap your head around that, >> it's typically easy to fix the code in order to make it working again >> (i.e. in this case, tweak access) - of course this problem will be >> addressed in a more general way. >> >> Maurizio > > Hm, currently there seem to be two ways of triggering specialization. > First way is by referencing classes with specialy-crafted names > (SomeClass${0=?}) and ClassLoader then contains a hook to dispatch to > the specialization logic. The second is used for invoking specialized > generic methods, which works with invokedynamic and bootstrap methods. > Would theoretically be possible to replace the 1st way (special class > names) with the invokedynamic too? Would it be practical? Would it > solve access problems? The current plan is to use a new mechanism called classdynamic for that purpose (this has come up before in this mailing list). I think, regardless on which plumbing you use to emit the generated class (either funky name + Loader changes, or classdyn), if the specialized class is a brand new class which is completely detached from the class it comes from, then access (w/o extra accessors) will always be problematic. Maurizio > > Regards, Peter > >>> >>>> >>>> public class Default { >>>> >>>> private T value; >>>> >>>> private Default() {} >>>> >>>> public static T value() { >>>> return new Default().value; >>>> } >>>> >>>> public static void main(String[] args) { >>>> int i = Default.value(); >>>> System.out.println(i); >>>> long l = Default.value(); >>>> System.out.println(l); >>>> Object o = Default.value(); >>>> System.out.println(o); >>>> } >>>> >>>> } >>>> >>>> When running: >>>> >>>> Specializing method Default$value${0=I}.value()Ljava/lang/Object; with >>>> class=[] and method=[I] >>>> Specializing Default${0=I}; searching for Default.class (not found) >>>> Specializing Default${0=I}; searching for Default.class (found) >>>> Exception in thread "main" java.lang.IllegalAccessError: tried to >>>> access >>>> method Default${0=I}.()V from class >>>> Default$value${0=I}/793589513 >>>> at Default$value${0=I}/793589513.value(Default.java:8) >>>> at Default.main(Default.java:12) >>>> >>>> Looks like its workaround-able by making both the constructor and >>>> value >>>> field public. >>>> >>>> I think this also exposes something I don't understand here. My >>>> impression >>>> was that "Default${0=I}" was the specialisation of the class >>>> Default s.t. >>>> its first type parameter is an int. So what is "Default$value${0=I}"? >>> >>> I think Default$value${0=I}/793589513 is a special (VM-annonymous) >>> class, containing the specialization of static method >>> Default.value() for T=int. >>> >>> The problem seems to be that private constructor of specialized >>> class Default for T=int is not accessible from it. The specialized >>> static method does have access to private members of nonspecialized >>> class Default (since this is the host class of it's VM-anonymous >>> class where it is defined), but not to the specialized Default class >>> for T=int which is a separate class. Perhaps something similar to >>> how outer class has access to private members of inner classes and >>> vice-versa would have to be devised among non-specialized classes >>> and their specializations. >>> >>> Regards, Peter >>> >>>> >>>> regards, >>>> >>>> Richard Warburton >>>> >>>> http://insightfullogic.com >>>> @RichardWarburto >>> >> > From brian.goetz at oracle.com Fri Jan 9 16:25:34 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 09 Jan 2015 11:25:34 -0500 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: <54AFDA5B.7040804@ochsenreither.de> References: <54AFDA5B.7040804@ochsenreither.de> Message-ID: <54B000FE.5010202@oracle.com> Yes, this is the plan. As a possible implementation strategy: - have == simply translate to invocation of .equals() for values - have compiler/VM generate componentwise equals/hashCode - maybe, or maybe not, permit developer to override said equals > What's wrong with giving == and != a sensible implementation for value > types as mentioned earlier? This would avoid introducing a new concept > and would not require rewriting every equals/contains/... method in > existence. > > >>> b) How is this different to the approaches already outlines earlier? >> >> It provides a basic logical building-block (equality check) at the >> language level, in a way that works both for primitives/valuetypes and >> for nullable references. It addresses a very big long-standing pain >> point in Java application development. It avoids writing/ or encouraging >> "Duck.woof()" style-hacks -- null checks -- into a domain that does not >> uniformly support them. > > Isn't this what was proposed earlier, just with == instead of > introducing a new operator? > From brian.goetz at oracle.com Fri Jan 9 16:30:18 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 09 Jan 2015 11:30:18 -0500 Subject: Access Modifier Issues In-Reply-To: <54AFFE88.7070203@oracle.com> References: <54AFEA12.1030109@gmail.com> <54AFEF89.1060000@oracle.com> <54AFFD49.9070007@gmail.com> <54AFFE88.7070203@oracle.com> Message-ID: <54B0021A.6090305@oracle.com> Just to be 100% clear, the "load a class with a funny Foo${T=I} name" is exclusively a temporary prototyping hack. (I wish I didn't have to say this, but I do; a few months ago, some "journalist" saw the first SotS draft and then reported "Java developers will trigger primitive specialization by adding ${T=I} to their class names". No joke.) >> Hm, currently there seem to be two ways of triggering specialization. >> First way is by referencing classes with specialy-crafted names >> (SomeClass${0=?}) and ClassLoader then contains a hook to dispatch to >> the specialization logic. The second is used for invoking specialized >> generic methods, which works with invokedynamic and bootstrap methods. >> Would theoretically be possible to replace the 1st way (special class >> names) with the invokedynamic too? Would it be practical? Would it >> solve access problems? > The current plan is to use a new mechanism called classdynamic for that > purpose (this has come up before in this mailing list). I think, > regardless on which plumbing you use to emit the generated class (either > funky name + Loader changes, or classdyn), if the specialized class is a > brand new class which is completely detached from the class it comes > from, then access (w/o extra accessors) will always be problematic. From simon at ochsenreither.de Fri Jan 9 16:37:19 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Fri, 09 Jan 2015 17:37:19 +0100 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: <54B000FE.5010202@oracle.com> References: <54AFDA5B.7040804@ochsenreither.de> <54B000FE.5010202@oracle.com> Message-ID: <54B003BF.5080900@ochsenreither.de> > As a possible implementation strategy: > - have == simply translate to invocation of .equals() for values > - have compiler/VM generate componentwise equals/hashCode > - maybe, or maybe not, permit developer to override said equals The combination of 1 and 3 sounds terrible, imho. :-) a) Until now, people could expect that == had a fixed, non-changeable interpretation. With that strategy, it could do arbitrary things. b) Additionally it conflates two different concepts: Low-level identity and high-level equality. c) Plus, with references we had to worry whether equals was implemented or was just using the default implementation of pointer identity. With that strategy we would create the reverse problem: Using ==, we would have to wonder whether it was just the default implementation or someone overrode it. What's wrong with the strategy I proposed earlier? From vitalyd at gmail.com Fri Jan 9 16:42:58 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 9 Jan 2015 11:42:58 -0500 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: <54B003BF.5080900@ochsenreither.de> References: <54AFDA5B.7040804@ochsenreither.de> <54B000FE.5010202@oracle.com> <54B003BF.5080900@ochsenreither.de> Message-ID: Honestly, it's going to be terrible if people can't provide their own equals/hashCode for value types. What's the rationale for that? Also, special casing == for just value types doesn't seem appealing as it muddies the water and when I see == in code I have to go check whether it's value types or not to figure out what's going to happen. Sent from my phone On Jan 9, 2015 11:37 AM, "Simon Ochsenreither" wrote: > > As a possible implementation strategy: > > - have == simply translate to invocation of .equals() for values > > - have compiler/VM generate componentwise equals/hashCode > > - maybe, or maybe not, permit developer to override said equals > > The combination of 1 and 3 sounds terrible, imho. :-) > > a) Until now, people could expect that == had a fixed, non-changeable > interpretation. With that strategy, it could do arbitrary things. > > b) Additionally it conflates two different concepts: Low-level identity > and high-level equality. > > c) Plus, with references we had to worry whether equals was implemented > or was just using the default implementation of pointer identity. With > that strategy we would create the reverse problem: Using ==, we would > have to wonder whether it was just the default implementation or someone > overrode it. > > What's wrong with the strategy I proposed earlier? > From simon at ochsenreither.de Fri Jan 9 17:19:06 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Fri, 09 Jan 2015 18:19:06 +0100 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: References: <54AFDA5B.7040804@ochsenreither.de> <54B000FE.5010202@oracle.com> <54B003BF.5080900@ochsenreither.de> Message-ID: <54B00D8A.8080401@ochsenreither.de> > Honestly, it's going to be terrible if people can't provide their own > equals/hashCode for value types. What's the rationale for that? > > Also, special casing == for just value types doesn't seem appealing as > it muddies the water and when I see == in code I have to go check > whether it's value types or not to figure out what's going to happen. Neither of these ideas are part of my suggestion made earlier. From vitalyd at gmail.com Fri Jan 9 17:21:17 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 9 Jan 2015 12:21:17 -0500 Subject: valhalla-dev Digest, Vol 7, Issue 24 In-Reply-To: <54B00D8A.8080401@ochsenreither.de> References: <54AFDA5B.7040804@ochsenreither.de> <54B000FE.5010202@oracle.com> <54B003BF.5080900@ochsenreither.de> <54B00D8A.8080401@ochsenreither.de> Message-ID: Yes sorry, this was meant for Brian. Sent from my phone On Jan 9, 2015 12:19 PM, "Simon Ochsenreither" wrote: > > Honestly, it's going to be terrible if people can't provide their own > > equals/hashCode for value types. What's the rationale for that? > > > > Also, special casing == for just value types doesn't seem appealing as > > it muddies the water and when I see == in code I have to go check > > whether it's value types or not to figure out what's going to happen. > > Neither of these ideas are part of my suggestion made earlier. > From palo.marton at gmail.com Fri Jan 9 17:37:56 2015 From: palo.marton at gmail.com (Palo Marton) Date: Fri, 9 Jan 2015 18:37:56 +0100 Subject: Idea how to implement VT/VO compatibility in JVM Message-ID: (See this thread: http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-January/000710.html ) Main idea is to move information about using VT from method descriptors to metadata. E.g. for method double distanceTo(point p); the corresponding method descriptor will ?pretend? that the actual parameter is boxed object reference, like (LPoint;)D. But in metadata (could be eg. parameter annotation in class that declares the method) it will indicate that this is actually value type. This way the method descriptor for VT and VO will be the same and VO based code should actually be able to call VT based method. (note: this ?VT erasure? should not be done for arguments like ?point p[]? as array of VT is very different from array of VO). Of course such change will require a bit different implementation of VT in JVM. One big issue with such implementation is that when mixing VT/VO code, situation may arise where one method overrides other method and they have different VT flags for their arguments/result, which means that they have also different calling convention for JITed code. The same may happen when implementing interface. This may be solved by allocating another slot in vftable for that function and filling original slot with some small bridge code that will handle needed boxing/unboxing and call method from the right vftable slot. So in case of mixed VT/VO code there may be multiple slots for one method (=method descriptor) in vftable. Big change, I know? Other issue is that if we move info about VT from method descriptor to metadata, we should do the same also for bytecode of method. So bytecode will manipulate VT values using normal reference instructions (aload, astore, ?). This also means that JVM interpreter will use only boxed VT values. It is a performance penalty for interpreter, but may be this is not so important as most code will be JITed. JIT compiler can utilize information from metadata to do more aggressive stack/register allocation (no escape analyses is needed for VT values) and better argument passing and result returning. There are many other issues with this. For some of them I see solution, but there are probably many ?devils in details? that I can not see. Question: Have you considered such approach to VT? It basically means that VT is implemented as value object for which JIT can do more aggressive optimizations thanks to metadata included in places where it was used as VT in the source code. From palo.marton at gmail.com Fri Jan 9 17:44:16 2015 From: palo.marton at gmail.com (Palo Marton) Date: Fri, 9 Jan 2015 18:44:16 +0100 Subject: =?UTF-8?Q?Re=3A_Value_types_=2D_compatibility_with_existing_=E2=80=9Cval?= =?UTF-8?Q?ue_objects=E2=80=9D?= In-Reply-To: <54AEB40C.7030702@oracle.com> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> Message-ID: (reposting this to whole group because of accidental reply to individual address) > > Essentially, yes. There's no hard cutoff of "value types must be less > than X size", but the VM reserves the right to transparently box values > that are too big (this is better than rejecting them either statically or > dynamically). You can still code them as values and get the value > semantics and the VM will try its best to give you the performance, but you > don't get guaranteed flattening. (Note that the benefits of flattening > decrease dramatically once they get big enough anyway.) > It all comes down to how big is performance penalty for declaring bigger value types. If the performance degrades to the same speed as if I have used "value object", then it is OK. But if the performance degrades more than that (extreme case: big VT will be handled just by interpreter), then it is bad. I didn't knew that you plan to internally support both pass-by-value and pass-by-reference (and probably also store-by-value and store-by-reference) and choose one that is better for given situation. It is good to hear that ;-) I am not sure where you got this idea that somehow "too big" values would > require a different syntax. However, its possible you're running on a > machine with a very restricted register set, and a 4x4 matrix of longs is > too big to pass by value, You do not need to have machine with "very restricted register set" to get out of registers. May be if JVM is clever enough it can put 4x4 doubles into upper/lower halves of xmm0-xmm7 (on x64). But in many cases this will not be the only argument of function. E.g. function to multiply two 4x4 matrices will need 32 doubles. Will you use all xmm regsiters just to pass them? Probably not. There are many trade-offs here. May be such big VT should be always stored (internally) as reference to boxed object on heap... I know that current state of development it is not possible to solve all problems and this may be solved later. But it is important that in the long term there will not be some (serious) performance penalty from using VT over value object. > On Thu, Jan 8, 2015 at 5:02 PM, Brian Goetz > > wrote: >> >> Like many questions we get here, this is a fine question which has >> basically arrived at the wrong time, so we're going to ignore it for >> a while. Not because it's not important -- it is! But because you >> have to pour the foundation before you can paint. And the >> foundational work is occupying all of our attention right now. >> >> Of course migrating existing value-based classes is a highly >> desirable thing and it will definitely be on our mind as we work >> through the more foundational aspects. String may in the end be a >> lost cause -- I guarantee you there is plenty of mission-critical >> code out there that synchronizes on strings -- but there are plenty >> of value-based classes in the JDK that could benefit from such a >> treatment and might be less problematic to convert. >> >> This is already on the "to think about next year" list, but since >> the "to think about this year" list is already several years long, >> we probably won't get to it for a while. >> >> On 1/8/2015 9:51 AM, Palo Marton wrote: >> >> Yes, I know that you can not simply replace existing value >> objects with >> value types when executing old pre-values-types code. Such code >> must be >> executed with objects. >> >> My idea is something along these lines: >> >> Declare value types as normal class (final with final fields), >> but add >> secondary "value name" for it, e.g. like this: >> >> public final class *String *value *string *{ >> >> ... >> } >> >> When in your code you declare variable as "String" than it will >> work the >> same as it works now (with identity, lock, null, etc..) >> >> But when you declare variable as "string", you basically say that: >> >> - I don't care about identity and locking and don't plan to use it >> - this will never be null. >> >> Which gives JVM option to store and pass it "by value" instead >> of "by >> reference". >> >> >> >> >> On Thu, Jan 8, 2015 at 3:24 PM, Vitaly Davidovich >> > wrote: >> >> I don't think this has been ignored, certainly not on the >> recent traffic >> on this mailing list. >> >> I think the short answer to the "can existing classes that >> are value >> objects be migrated to value types with no client change" is >> probably no. >> The reason, or one of them, is that there's no way to know >> (without >> analyzing all usages) whether client code uses object >> features of said >> object (e.g. uses reference identity comparison, locks on an >> instance, etc). >> >> In your rephrasing, "works like a class" implies identity of >> the object >> and can be synchronized on. That's not what VT is of course. >> >> Sent from my phone >> On Jan 8, 2015 9:17 AM, "Palo Marton" > > wrote: >> >> My question is not just about String but about all >> existing value types. >> There are many of them in current java code (both JDK >> and user code). >> Current proposal seems to ignore them. >> >> I think that the original idea of VT: >> >> "Codes like a class, works like an int!" >> >> can be changed to: >> >> "Codes like a class, works like class, but can be stored >> and passed as >> int!" >> >> as the problem with current value objects (like String) >> is not with the >> fact that they are objects, but in how they are stored >> in memory and passed >> between functions. >> >> >> >> On Thu, Jan 8, 2015 at 3:02 PM, Vitaly Davidovich >> > >> wrote: >> >> I had a suspicion this would be the gist. However, >> I think better >> solution is to simply change string to have inline >> storage of the data, so >> no indirection to the char[]. Ultimately, the >> string data will have to >> live on the heap one way or another, but it'd be >> nice if String had that >> data inlined into its storage so when you load >> address of string you get >> the data at a small fixed offset from that address. >> >> Sent from my phone >> On Jan 8, 2015 8:54 AM, "Peter Levart" >> > > wrote: >> >> On 01/08/2015 02:50 PM, Vitaly Davidovich wrote: >> >> Why do you want string to be value type? >> What problem (s) will that >> address? >> >> >> String as a value type would eliminate one >> indirection. If would >> effectively become an immutable char[] with a >> bunch of operations. >> >> Peter >> >> >> Sent from my phone >> On Jan 8, 2015 7:04 AM, "Palo Marton" >> > > wrote: >> >> Thank you for response. I have read that >> thread. But apart from first >> >> few >> messages it seems to deal with something >> else - mutable types. >> >> My idea with rewriting String as value >> is not about changing meaning >> of >> java.lang.String. That identifier should >> still mean heap object with >> identity, null, etc... The idea is to >> give a new name for string >> value, >> e.g. java.lang.string (lowercase s), or >> just "string". So "String" >> will be >> boxed version of "string" and all old >> pre-value-types code will use >> only >> boxed objects as before and there will >> be no semantic change. In new >> code >> you will use "string" in most cases, but >> you can use also "String" in >> places where you need it. >> >> From what I have read there are no >> plans to support something like >> that. Am >> I right? >> >> Reason why I am asking this is that some >> time ago I was thinking about >> value types in java and how they should >> be implemented in JVM. My idea >> about how to implement it in JVM was >> somehow different, but it can >> support >> this old-code / new-code compatibility. >> >> On Thu, Jan 8, 2015 at 11:42 AM, Richard >> Warburton < >> richard.warburton at gmail.com >> > >> wrote: >> >> Hi, >> >> >> Will it be possible (within >> valhalla) to rewrite java String as >> value >> >> type >> >> in JDK and use it in all places in >> JDK where it makes sense? (eg. >> >> Class.forName(string), >> Object.toString(), >> Integer.parseInt(string). >> ?) >> >> Or more general question: >> >> There are already plenty of >> ?value objects? in existing java >> code. >> In >> >> JDK >> >> >> we have, String, File, Point2D,... >> Also many people have already >> >> >> declared >> >> >> their own class Point { final x,y; } >> and similar objects. Once we >> >> move >> >> to >> >> >> value types in java, will it be >> possible to rewrite these to value >> >> types >> without breaking compatibility >> with old pre-value-types code? >> E.g. >> if I >> change my Point object to value >> type in my library, can this new >> library >> (jar) still be used in other >> projects that were written >> before this >> change? >> And without need to recompile >> those other projects? >> >> >> See this thread: >> >> http://mail.openjdk.java.net/_ >> _pipermail/valhalla-dev/2015- >> > pipermail/valhalla-dev/2015-> >> >> January/000546.html >> >> regards, >> >> Richard Warburton >> >> http://insightfullogic.com >> @RichardWarburto >> > > >> >> >> >> >> >> >> From brian.goetz at oracle.com Fri Jan 9 17:50:42 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 09 Jan 2015 12:50:42 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: Message-ID: <54B014F2.9080805@oracle.com> Short answer: yes. See Albert Noll's post about heisenboxes, which is one piece of a story that could address one piece of this. Where this approach runs out of gas is when data hits the heap: layout of flattened value fields in objects, and layout of flattened value elements in arrays. (These are the same challenges as the "erase to Any" approach.) The VM let's you override methods, and with bridge methods or other techniques, we can effectively "override" method signatures. But the VM doesn't have any such capability for fields. Once you say a field is of type Object, that's the type. Think about Box: class Box { T t; } We want Box to have a single field that behaves like an 'int' without boxing and without taking more than four bytes of data payload. That's the fundamental sticking point with most of the alternate suggestions; they don't have a path to get there. On 1/9/2015 12:37 PM, Palo Marton wrote: > (See this thread: > http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-January/000710.html) > > Main idea is to move information about using VT from method descriptors > to metadata. > > E.g. for method > > double distanceTo(point p); > > the corresponding method descriptor will ?pretend? that the actual > parameter is boxed object reference, like (LPoint;)D. But in metadata > (could be eg. parameter annotation in class that declares the method) it > will indicate that this is actually value type. This way the method > descriptor for VT and VO will be the same and VO based code should > actually be able to call VT based method. (note: this ?VT erasure? > should not be done for arguments like ?point p[]? as array of VT is very > different from array of VO). > > Of course such change will require a bit different implementation of VT > in JVM. > > One big issue with such implementation is that when mixing VT/VO code, > situation may arise where one method overrides other method and they > have different VT flags for their arguments/result, which means that > they have also different calling convention for JITed code. The same may > happen when implementing interface. This may be solved by allocating > another slot in vftable for that function and filling original slot with > some small bridge code that will handle needed boxing/unboxing and call > method from the right vftable slot. So in case of mixed VT/VO code there > may be multiple slots for one method (=method descriptor) in vftable. > Big change, I know? > > Other issue is that if we move info about VT from method descriptor to > metadata, we should do the same also for bytecode of method. So bytecode > will manipulate VT values using normal reference instructions (aload, > astore, ?). This also means that JVM interpreter will use only boxed VT > values. It is a performance penalty for interpreter, but may be this is > not so important as most code will be JITed. JIT compiler can utilize > information from metadata to do more aggressive stack/register > allocation (no escape analyses is needed for VT values) and better > argument passing and result returning. > > There are many other issues with this. For some of them I see solution, > but there are probably many ?devils in details? that I can not see. > > > Question: Have you considered such approach to VT? It basically means > that VT is implemented as value object for which JIT can do more > aggressive optimizations thanks to metadata included in places where it > was used as VT in the source code. From palo.marton at gmail.com Fri Jan 9 18:38:16 2015 From: palo.marton at gmail.com (Palo Marton) Date: Fri, 9 Jan 2015 19:38:16 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54B014F2.9080805@oracle.com> References: <54B014F2.9080805@oracle.com> Message-ID: > > See Albert Noll's post about heisenboxes, which is one piece of a story > that could address one piece of this. > I have tried to google this, but without success. Can you please send a link to that post? > Where this approach runs out of gas is when data hits the heap: layout of > flattened value fields in objects, and layout of flattened value elements > in arrays. (These are the same challenges as the "erase to Any" approach.) > I know that this does not work for arrays (have mentioned that in the post). For object field it should work this way: Descriptor of such field will be "VT erased" to boxed class and metadata will indicate that the field was actually declared as VT. When JVM loads such class, it checks the metadata and lays out the class according to that. The field will be flattened inside the object. This will also change semantic of "getfield" and "putfield" instructions for such field. In interpreter getfield will read flattened data and box it on heap, putfield will write flattened data into the object. In JITed code it will do the best think possible. > > The VM let's you override methods, and with bridge methods or other > techniques, we can effectively "override" method signatures. But the VM > doesn't have any such capability for fields. Once you say a field is of > type Object, that's the type. > > Think about Box: > > class Box { > T t; > } > > We want Box to have a single field that behaves like an 'int' without > boxing and without taking more than four bytes of data payload. That's the > fundamental sticking point with most of the alternate suggestions; they > don't have a path to get there. My idea was about implementation of value types, not about generic specialization. So I am not sure what you mean here. Did you meant Box being value type? > > On 1/9/2015 12:37 PM, Palo Marton wrote: > >> (See this thread: >> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- >> January/000710.html) >> >> Main idea is to move information about using VT from method descriptors >> to metadata. >> >> E.g. for method >> >> double distanceTo(point p); >> >> the corresponding method descriptor will ?pretend? that the actual >> parameter is boxed object reference, like (LPoint;)D. But in metadata >> (could be eg. parameter annotation in class that declares the method) it >> will indicate that this is actually value type. This way the method >> descriptor for VT and VO will be the same and VO based code should >> actually be able to call VT based method. (note: this ?VT erasure? >> should not be done for arguments like ?point p[]? as array of VT is very >> different from array of VO). >> >> Of course such change will require a bit different implementation of VT >> in JVM. >> >> One big issue with such implementation is that when mixing VT/VO code, >> situation may arise where one method overrides other method and they >> have different VT flags for their arguments/result, which means that >> they have also different calling convention for JITed code. The same may >> happen when implementing interface. This may be solved by allocating >> another slot in vftable for that function and filling original slot with >> some small bridge code that will handle needed boxing/unboxing and call >> method from the right vftable slot. So in case of mixed VT/VO code there >> may be multiple slots for one method (=method descriptor) in vftable. >> Big change, I know? >> >> Other issue is that if we move info about VT from method descriptor to >> metadata, we should do the same also for bytecode of method. So bytecode >> will manipulate VT values using normal reference instructions (aload, >> astore, ?). This also means that JVM interpreter will use only boxed VT >> values. It is a performance penalty for interpreter, but may be this is >> not so important as most code will be JITed. JIT compiler can utilize >> information from metadata to do more aggressive stack/register >> allocation (no escape analyses is needed for VT values) and better >> argument passing and result returning. >> >> There are many other issues with this. For some of them I see solution, >> but there are probably many ?devils in details? that I can not see. >> >> >> Question: Have you considered such approach to VT? It basically means >> that VT is implemented as value object for which JIT can do more >> aggressive optimizations thanks to metadata included in places where it >> was used as VT in the source code. >> > From brian.goetz at oracle.com Fri Jan 9 19:12:10 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 09 Jan 2015 14:12:10 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> Message-ID: <54B0280A.3020707@oracle.com> > For object field it should work this way: > > Descriptor of such field will be "VT erased" to boxed class and metadata > will indicate that the field was actually declared as VT. When JVM loads > such class, it checks the metadata and lays out the class according to > that. The field will be flattened inside the object. This will also > change semantic of "getfield" and "putfield" instructions for such > field. In interpreter getfield will read flattened data and box it on > heap, putfield will write flattened data into the object. In JITed code > it will do the best think possible. Sure, that's the easy part (and even that is not totally easy.) The hard part is: the entire VM field-access model is built around translating field accesses like getfield Foo.x [ receiver ] to native code that looks like: (ideally elided) verify receiver is a Foo result = *(receiver + sizeof(HEADER) + offsetof(x)) based on the assumption that all instances of Foo have Foo's fields at the same offsets. (For methods we achieve polymorphism with vtables; for fields, we achieve it by adding subtype fields to the end of base type fields.) But, if I have a class Box with a single T-valued field, I need different layouts; one for erased refs, one for ints, a longer one for longs, etc. (And of course this perturbs the offset of other fields.) Code that is generic in T that wants to access Box.t now has to deal with different layouts; this is a pretty big change to our compilation story (with consequences for performance.) So, I think we're back at "just completely rearchitect the JVM, and it is all easy" :( Of course, we are willing to consider changes in the VM architecture. But we also have to be realistic... From palo.marton at gmail.com Fri Jan 9 19:29:59 2015 From: palo.marton at gmail.com (Palo Marton) Date: Fri, 9 Jan 2015 20:29:59 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54B0280A.3020707@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> Message-ID: Seems that there is some confusion here. I was not addressing generics and their specialization, but other problem. It is described in the first post of this thread: http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-January/000710.html On Fri, Jan 9, 2015 at 8:12 PM, Brian Goetz wrote: > For object field it should work this way: >> >> Descriptor of such field will be "VT erased" to boxed class and metadata >> will indicate that the field was actually declared as VT. When JVM loads >> such class, it checks the metadata and lays out the class according to >> that. The field will be flattened inside the object. This will also >> change semantic of "getfield" and "putfield" instructions for such >> field. In interpreter getfield will read flattened data and box it on >> heap, putfield will write flattened data into the object. In JITed code >> it will do the best think possible. >> > > Sure, that's the easy part (and even that is not totally easy.) > > The hard part is: the entire VM field-access model is built around > translating field accesses like > > getfield Foo.x [ receiver ] > > to native code that looks like: > > (ideally elided) verify receiver is a Foo > result = *(receiver + sizeof(HEADER) + offsetof(x)) > > based on the assumption that all instances of Foo have Foo's fields at the > same offsets. (For methods we achieve polymorphism with vtables; for > fields, we achieve it by adding subtype fields to the end of base type > fields.) > > But, if I have a class Box with a single T-valued field, I need > different layouts; one for erased refs, one for ints, a longer one for > longs, etc. (And of course this perturbs the offset of other fields.) > > Code that is generic in T that wants to access Box.t now has to deal > with different layouts; this is a pretty big change to our compilation > story (with consequences for performance.) > > So, I think we're back at "just completely rearchitect the JVM, and it is > all easy" :( > > Of course, we are willing to consider changes in the VM architecture. But > we also have to be realistic... > > From palo.marton at gmail.com Fri Jan 9 19:55:19 2015 From: palo.marton at gmail.com (Palo Marton) Date: Fri, 9 Jan 2015 20:55:19 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54B02D0B.20508@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <54B02D0B.20508@oracle.com> Message-ID: May be to avoid such confusion it will be better to split project valhalla (or just the mailing list) into separate parts: - value types - generic specialization (+enhanced volatiles) Seems that your current focus is on generic specialization. Or are you working in the same time also on value types? On Fri, Jan 9, 2015 at 8:33 PM, Brian Goetz wrote: > You may be right; I may have thought I was responding to a different > "redesign the VM" suggestion (the fact that we get so many that they can't > be kept straight, is already a sign of something else gone badly wrong.) > > > > On 1/9/2015 2:29 PM, Palo Marton wrote: > >> Seems that there is some confusion here. I was not addressing generics >> and their specialization, but other problem. It is described in the >> first post of this thread: >> >> http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- >> January/000710.html >> >> On Fri, Jan 9, 2015 at 8:12 PM, Brian Goetz > > wrote: >> >> For object field it should work this way: >> >> Descriptor of such field will be "VT erased" to boxed class and >> metadata >> will indicate that the field was actually declared as VT. When >> JVM loads >> such class, it checks the metadata and lays out the class >> according to >> that. The field will be flattened inside the object. This will >> also >> change semantic of "getfield" and "putfield" instructions for such >> field. In interpreter getfield will read flattened data and box >> it on >> heap, putfield will write flattened data into the object. In >> JITed code >> it will do the best think possible. >> >> >> Sure, that's the easy part (and even that is not totally easy.) >> >> The hard part is: the entire VM field-access model is built around >> translating field accesses like >> >> getfield Foo.x [ receiver ] >> >> to native code that looks like: >> >> (ideally elided) verify receiver is a Foo >> result = *(receiver + sizeof(HEADER) + offsetof(x)) >> >> based on the assumption that all instances of Foo have Foo's fields >> at the same offsets. (For methods we achieve polymorphism with >> vtables; for fields, we achieve it by adding subtype fields to the >> end of base type fields.) >> >> But, if I have a class Box with a single T-valued field, I >> need different layouts; one for erased refs, one for ints, a longer >> one for longs, etc. (And of course this perturbs the offset of >> other fields.) >> >> Code that is generic in T that wants to access Box.t now has to >> deal with different layouts; this is a pretty big change to our >> compilation story (with consequences for performance.) >> >> So, I think we're back at "just completely rearchitect the JVM, and >> it is all easy" :( >> >> Of course, we are willing to consider changes in the VM >> architecture. But we also have to be realistic... >> >> >> From brian.goetz at oracle.com Fri Jan 9 20:48:25 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 09 Jan 2015 15:48:25 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <54B02D0B.20508@oracle.com> Message-ID: <54B03E99.2060900@oracle.com> The projects are inter-related for sure (we probably wouldn't even be doing specialization without value types), but for now, our head is mostly on the generics side of the story. I don't think splitting the lists would be helpful, though. (The same people would be on both; many people would post in the wrong place; even threads that started out in the right place would eventually end up in the wrong place as the focus of discussion meandered as is typical for such discussions.) I think the real problem is that we're at a stage where there's less people can do to actually contribute than they'd like (which you can think of as a good thing), and so that contribution-energy gets channeled into directions that may be well-intentioned but ultimately are counterproductive. Probably soon, we'll be setting up a limited-write -experts list, at which point this -dev list will revert to its intended role, which is discussions pertaining to the implementation. We can also do, as we did for lambda, set up a publicly writable -observers list for broader participation. On 1/9/2015 2:55 PM, Palo Marton wrote: > May be to avoid such confusion it will be better to split project > valhalla (or just the mailing list) into separate parts: > > - value types > - generic specialization (+enhanced volatiles) > > Seems that your current focus is on generic specialization. Or are you > working in the same time also on value types? > > > > On Fri, Jan 9, 2015 at 8:33 PM, Brian Goetz > wrote: > > You may be right; I may have thought I was responding to a different > "redesign the VM" suggestion (the fact that we get so many that they > can't be kept straight, is already a sign of something else gone > badly wrong.) > > > > On 1/9/2015 2:29 PM, Palo Marton wrote: > > Seems that there is some confusion here. I was not addressing > generics > and their specialization, but other problem. It is described in the > first post of this thread: > > http://mail.openjdk.java.net/__pipermail/valhalla-dev/2015-__January/000710.html > > > On Fri, Jan 9, 2015 at 8:12 PM, Brian Goetz > > __>> wrote: > > For object field it should work this way: > > Descriptor of such field will be "VT erased" to boxed > class and > metadata > will indicate that the field was actually declared as > VT. When > JVM loads > such class, it checks the metadata and lays out the class > according to > that. The field will be flattened inside the object. > This will also > change semantic of "getfield" and "putfield" > instructions for such > field. In interpreter getfield will read flattened data > and box > it on > heap, putfield will write flattened data into the > object. In > JITed code > it will do the best think possible. > > > Sure, that's the easy part (and even that is not totally easy.) > > The hard part is: the entire VM field-access model is built > around > translating field accesses like > > getfield Foo.x [ receiver ] > > to native code that looks like: > > (ideally elided) verify receiver is a Foo > result = *(receiver + sizeof(HEADER) + offsetof(x)) > > based on the assumption that all instances of Foo have > Foo's fields > at the same offsets. (For methods we achieve polymorphism with > vtables; for fields, we achieve it by adding subtype fields > to the > end of base type fields.) > > But, if I have a class Box with a single T-valued > field, I > need different layouts; one for erased refs, one for ints, > a longer > one for longs, etc. (And of course this perturbs the offset of > other fields.) > > Code that is generic in T that wants to access Box.t now > has to > deal with different layouts; this is a pretty big change to our > compilation story (with consequences for performance.) > > So, I think we're back at "just completely rearchitect the > JVM, and > it is all easy" :( > > Of course, we are willing to consider changes in the VM > architecture. But we also have to be realistic... > > > From brian.goetz at oracle.com Fri Jan 9 21:44:42 2015 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Fri, 09 Jan 2015 21:44:42 +0000 Subject: hg: valhalla/valhalla/jdk: Fix specializer bug reported by Peter Levart related to improper handling of not-fully-instantiated class literals Message-ID: <201501092144.t09Lih7T024582@aojmv0008> Changeset: 7a3e4270521d Author: briangoetz Date: 2015-01-09 16:44 -0500 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/7a3e4270521d Fix specializer bug reported by Peter Levart related to improper handling of not-fully-instantiated class literals ! src/java.base/share/classes/valhalla/specializer/Specializer.java ! test/valhalla/test/valhalla/specializer/GenericMethodsTest.java From brian.goetz at oracle.com Fri Jan 9 21:45:26 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 09 Jan 2015 16:45:26 -0500 Subject: runtime failure when "any T" generic method calls another "any T" generic method In-Reply-To: <54AE3B3C.5080006@gmail.com> References: <54AE3B3C.5080006@gmail.com> Message-ID: <54B04BF6.4040609@oracle.com> Thanks Peter! I've pushed a fix, which was basically a one-line change in the specializer; I was missing a .getDescAsClassName(erasures) after doing the substitution on the operand of the LDC instruction. On 1/8/2015 3:09 AM, Peter Levart wrote: > Hi, > > Here's another problem I found: > > > public class ValhallaTest { > > static void test2() { > Class clazz = ValhallaTest.class; // line 7 > System.out.println("test2 >>> " + clazz); > } > > static void test1() { > Class clazz = ValhallaTest.class; > System.out.println("test1 >>> " + clazz); > > ValhallaTest.test2(); > } > > public static void main(String[] args) { > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > ValhallaTest.test1(); > } > } > > > Which prints: > > > Specializing method ValhallaTest$test1${0=Z}.test1()V with class=[] and > method=[Z] > Specializing ValhallaTest${0=Z}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=Z}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=Z} > Specializing method ValhallaTest$test2${0=Z}.test2()V with class=[] and > method=[Z] > test2 >>> class ValhallaTest${0=Z} > Specializing method ValhallaTest$test1${0=B}.test1()V with class=[] and > method=[B] > Specializing ValhallaTest${0=B}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=B}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=B} > Specializing method ValhallaTest$test2${0=B}.test2()V with class=[] and > method=[B] > test2 >>> class ValhallaTest${0=B} > Specializing method ValhallaTest$test1${0=C}.test1()V with class=[] and > method=[C] > Specializing ValhallaTest${0=C}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=C}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=C} > Specializing method ValhallaTest$test2${0=C}.test2()V with class=[] and > method=[C] > test2 >>> class ValhallaTest${0=C} > Specializing method ValhallaTest$test1${0=S}.test1()V with class=[] and > method=[S] > Specializing ValhallaTest${0=S}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=S}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=S} > Specializing method ValhallaTest$test2${0=S}.test2()V with class=[] and > method=[S] > test2 >>> class ValhallaTest${0=S} > Specializing method ValhallaTest$test1${0=I}.test1()V with class=[] and > method=[I] > Specializing ValhallaTest${0=I}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=I}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=I} > Specializing method ValhallaTest$test2${0=I}.test2()V with class=[] and > method=[I] > test2 >>> class ValhallaTest${0=I} > Specializing method ValhallaTest$test1${0=J}.test1()V with class=[] and > method=[J] > Specializing ValhallaTest${0=J}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=J}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=J} > Specializing method ValhallaTest$test2${0=J}.test2()V with class=[] and > method=[J] > test2 >>> class ValhallaTest${0=J} > Specializing method ValhallaTest$test1${0=F}.test1()V with class=[] and > method=[F] > Specializing ValhallaTest${0=F}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=F}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=F} > Specializing method ValhallaTest$test2${0=F}.test2()V with class=[] and > method=[F] > test2 >>> class ValhallaTest${0=F} > Specializing method ValhallaTest$test1${0=D}.test1()V with class=[] and > method=[D] > Specializing ValhallaTest${0=D}; searching for ValhallaTest.class (not > found) > Specializing ValhallaTest${0=D}; searching for ValhallaTest.class (found) > test1 >>> class ValhallaTest${0=D} > Specializing method ValhallaTest$test2${0=D}.test2()V with class=[] and > method=[D] > test2 >>> class ValhallaTest${0=D} > test1 >>> class ValhallaTest > Specializing method ValhallaTest$test2${}.test2()V with class=[] and > method=[TT;] > Exception in thread "main" java.lang.NoClassDefFoundError: ValhallaTest at ValhallaTest$test2${}/401625763.test2(ValhallaTest.java:7) > at ValhallaTest.test1(ValhallaTest.java:15) > at ValhallaTest.main(ValhallaTest.java:27) > Caused by: java.lang.ClassNotFoundException: ValhallaTest at java.net.URLClassLoader.findClass(URLClassLoader.java:451) > at java.lang.ClassLoader.loadClass(ClassLoader.java:426) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317) > at java.lang.ClassLoader.loadClass(ClassLoader.java:359) > ... 3 more > > > Within tes1() method, everything works fine, but cascading call to > test2() fails in line 7 where ValhallaTest.class is evaluated, but > only for reference type T. > > Note that there's no specialization for test1() being triggered for > T=reference (which is understandable), but there is a specialization > being triggered for cascaded call to test2() in this case. In my > superficial understanding a call to: > > ValhallaTest.test1(); > > ...is implemented as invokestatic, but a call from test1() to: > > ValhallaTest.test2(); > > ...is implemented as invokedynamic. So in case such call comes from > non-specialized test1(), it should be wired directly to the > nonspecialized test2() method, right? > > Or, couldn't the call from nonspecialized test1() to .test2() also be > implemented as invokestatic and be converted to invokedynamic only by > specialization? > > Regards, Peter > From brian.goetz at oracle.com Fri Jan 9 21:56:12 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 09 Jan 2015 16:56:12 -0500 Subject: runtime failure when "any T" generic method calls another "any T" generic method In-Reply-To: <54B04BF6.4040609@oracle.com> References: <54AE3B3C.5080006@gmail.com> <54B04BF6.4040609@oracle.com> Message-ID: <54B04E7C.6020608@oracle.com> >> Or, couldn't the call from nonspecialized test1() to .test2() also be >> implemented as invokestatic and be converted to invokedynamic only by >> specialization? Yes. We've got some comments in the code to identify places like this where such optimizations can be made. For now, we're starting with "do the dumb thing", though, for obvious reasons. From duncan.macgregor at ge.com Sat Jan 10 02:29:10 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Sat, 10 Jan 2015 02:29:10 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54AAE765.6040402@oracle.com> References: <54AAE765.6040402@oracle.com> Message-ID: <978D194CF4618446926EDAD23C949D9939BC6E38@LONURLNA08.e2k.ad.ge.com> Continuing to play around with the specialisation work, and I've hit a snag that I haven't seen reported on this list, implicit boxing and varargs. If you had a small object in Java today which consisted of a mixture of primitive and object fields then you might reasonably write a little toString() method as public String toString() { String.format("(%s, %s, %s, %s)", field1, field2, field3, field4); } and the compiler will helpfully autobox any of those fields which were declared as primitives. If I try something similar with a specialised class then it currently gives an error because the types are any so it can't construct any Object[] from the four fields. In simple cases this could be solved by layering, but it would feel very strange to have to resort to that, and if we consider types tuples down the line then the combinatorial explosion would be ridiculous. This behaviour is entirely consistent with the type system as is, but will probably be the cause of real frustration as we go on. Can the byte code mapping attribute be used to store some indication that boxing must be performed at a particular index? Duncan. ________________________________________ From: valhalla-dev [valhalla-dev-bounces at openjdk.java.net] on behalf of Brian Goetz [brian.goetz at oracle.com] Sent: 05 January 2015 19:35 To: valhalla-dev at openjdk.java.net Subject: Can we also get some feedback on specialization, please? There's been a lot of passionate discussion here about the directions we should or should not take Java. And this is all understandable, and perhaps inevitable, but ... it seems to be crowding out what is actually the primary purpose of this (-dev) list. We had hoped, in taking the effort to write up State of the Specialization, to get some comments on ... specialization :) So far, nearly all the comments have been on the speculative parts of the writeup (layers/conditional methods), as well as the usual (supersized) portion of "I think you should do X/Y/Z instead". Which is all OK, but we were actually hoping to get some feedback on the part that is written up in some detail -- and mostly implemented! I know it's fun to discuss the big sexy stuff, but we can't let this crowd out the main goal. So my question is: has anyone tried to write a program with specialized generics with the valhalla implementation? Well, we'd really appreciate it. I know people want to contribute. At least right now, here's what contribution looks like from our perspective: 1.) Please try it out! Write some code! 2.) Tell us what works and what doesn't! 3.) Nice comments are always a bonus! (And are welcome even if you didn't do (1) and (2)). 4.) If you can't do (1) and (2), and still have something important to say in the big-picture department, please do it a) nicely, b) constructively, c) succinctly. Thanks, -Brian From duncan.macgregor at ge.com Sat Jan 10 02:53:10 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Sat, 10 Jan 2015 02:53:10 +0000 Subject: Casting reference array to any-T array. In-Reply-To: <54ADA9AA.6080802@oracle.com> References: <54ADA3F2.6050102@oracle.com> , <54ADA9AA.6080802@oracle.com> Message-ID: <978D194CF4618446926EDAD23C949D9939BC6E4A@LONURLNA08.e2k.ad.ge.com> Well... since we are now effectively compiling a mapping of which variables and stack entries refer to which type variables, does this give us enough information to help the compiler make better type decisions? If we were to annotate the byte code to also record known concrete types (such as a field being declared as List), then would that provide enough information to trace through subsequent usages and method calls in the compiler's graph? We'd have to record all type variables (rather than just anys) in the type variable map, and it clearly wouldn't allow for checkcasts to be removed from the byte code since pre-generics code could still break it (as could reflection or method & var handles), but it might allow for more aggressive inlining and optimisations. Duncan ________________________________________ From: valhalla-dev [valhalla-dev-bounces at openjdk.java.net] on behalf of Brian Goetz [brian.goetz at oracle.com] Sent: 07 January 2015 21:48 To: Vitaly Davidovich Cc: valhalla-dev at openjdk.java.net Subject: Re: Casting reference array to any-T array. > As a side note, is the plan to continue erasing T > for ref types or will the specializer ultimately create a non-erased T > array for them? The current functionality continues with the erasure plan. However, I wouldn't mind doing better! From duncan.macgregor at ge.com Sat Jan 10 03:47:20 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Sat, 10 Jan 2015 03:47:20 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54AAE765.6040402@oracle.com> References: <54AAE765.6040402@oracle.com> Message-ID: <978D194CF4618446926EDAD23C949D9939BC6E71@LONURLNA08.e2k.ad.ge.com> And another thing... If I want to try and combine the use of VarHandles and specialisation to make something with atomic properties (as I'm sure we'll all want to do for concurrent collections) then at the moment I can't use static fields for the var handles, and the JIT is almost certainly going to hate me if I they are stored on instance fields. I think we'll need some way to specify static fields as specialised and provide initialisers for them. This cannot be done by simply having different statics for different layers since each specialised class will need it's own statics. I'm sure there's also some evil I can perpetrate by specialising CallSites, but I haven't though of a really convincing case for doing so yet (at least not until I can define typed tuples when I can maybe replace various varargs fallbacks), and I hit the same statics problem as above with MethodHandles anyway. ________________________________________ From: valhalla-dev [valhalla-dev-bounces at openjdk.java.net] on behalf of Brian Goetz [brian.goetz at oracle.com] Sent: 05 January 2015 19:35 To: valhalla-dev at openjdk.java.net Subject: Can we also get some feedback on specialization, please? There's been a lot of passionate discussion here about the directions we should or should not take Java. And this is all understandable, and perhaps inevitable, but ... it seems to be crowding out what is actually the primary purpose of this (-dev) list. We had hoped, in taking the effort to write up State of the Specialization, to get some comments on ... specialization :) So far, nearly all the comments have been on the speculative parts of the writeup (layers/conditional methods), as well as the usual (supersized) portion of "I think you should do X/Y/Z instead". Which is all OK, but we were actually hoping to get some feedback on the part that is written up in some detail -- and mostly implemented! I know it's fun to discuss the big sexy stuff, but we can't let this crowd out the main goal. So my question is: has anyone tried to write a program with specialized generics with the valhalla implementation? Well, we'd really appreciate it. I know people want to contribute. At least right now, here's what contribution looks like from our perspective: 1.) Please try it out! Write some code! 2.) Tell us what works and what doesn't! 3.) Nice comments are always a bonus! (And are welcome even if you didn't do (1) and (2)). 4.) If you can't do (1) and (2), and still have something important to say in the big-picture department, please do it a) nicely, b) constructively, c) succinctly. Thanks, -Brian From brian.goetz at oracle.com Sat Jan 10 03:49:30 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 09 Jan 2015 22:49:30 -0500 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <978D194CF4618446926EDAD23C949D9939BC6E71@LONURLNA08.e2k.ad.ge.com> References: <54AAE765.6040402@oracle.com> <978D194CF4618446926EDAD23C949D9939BC6E71@LONURLNA08.e2k.ad.ge.com> Message-ID: <54B0A14A.5000007@oracle.com> You might look at ClassValue as a means for this sort of per-class static constants? On 1/9/2015 10:47 PM, MacGregor, Duncan (GE Energy Management) wrote: > And another thing... > > If I want to try and combine the use of VarHandles and specialisation > to make something with atomic properties (as I'm sure we'll all want > to do for concurrent collections) then at the moment I can't use > static fields for the var handles, and the JIT is almost certainly > going to hate me if I they are stored on instance fields. I think > we'll need some way to specify static fields as specialised and > provide initialisers for them. This cannot be done by simply having > different statics for different layers since each specialised class > will need it's own statics. > > I'm sure there's also some evil I can perpetrate by specialising > CallSites, but I haven't though of a really convincing case for doing > so yet (at least not until I can define typed tuples when I can maybe > replace various varargs fallbacks), and I hit the same statics > problem as above with MethodHandles anyway. From forax at univ-mlv.fr Sat Jan 10 09:54:47 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 10 Jan 2015 10:54:47 +0100 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <978D194CF4618446926EDAD23C949D9939BC6E38@LONURLNA08.e2k.ad.ge.com> References: <54AAE765.6040402@oracle.com> <978D194CF4618446926EDAD23C949D9939BC6E38@LONURLNA08.e2k.ad.ge.com> Message-ID: <54B0F6E7.2070102@univ-mlv.fr> The other solution is to stop to let the compiler to do the boxing for varargs and let the VM do the boxing by emitting an invokedynamic. It will simplify the bytecode and optimize simple case like when the array is empty. R?mi On 01/10/2015 03:29 AM, MacGregor, Duncan (GE Energy Management) wrote: > Continuing to play around with the specialisation work, and I've hit a snag that I haven't seen reported on this list, implicit boxing and varargs. > > If you had a small object in Java today which consisted of a mixture of primitive and object fields then you might reasonably write a little toString() method as > > public String toString() { > String.format("(%s, %s, %s, %s)", field1, field2, field3, field4); > } > > and the compiler will helpfully autobox any of those fields which were declared as primitives. > > If I try something similar with a specialised class then it currently gives an error because the types are any so it can't construct any Object[] from the four fields. In simple cases this could be solved by layering, but it would feel very strange to have to resort to that, and if we consider types tuples down the line then the combinatorial explosion would be ridiculous. This behaviour is entirely consistent with the type system as is, but will probably be the cause of real frustration as we go on. > > Can the byte code mapping attribute be used to store some indication that boxing must be performed at a particular index? > > Duncan. > ________________________________________ > From: valhalla-dev [valhalla-dev-bounces at openjdk.java.net] on behalf of Brian Goetz [brian.goetz at oracle.com] > Sent: 05 January 2015 19:35 > To: valhalla-dev at openjdk.java.net > Subject: Can we also get some feedback on specialization, please? > > There's been a lot of passionate discussion here about the directions we > should or should not take Java. And this is all understandable, and > perhaps inevitable, but ... it seems to be crowding out what is actually > the primary purpose of this (-dev) list. We had hoped, in taking the > effort to write up State of the Specialization, to get some comments on > ... specialization :) > > So far, nearly all the comments have been on the speculative parts of > the writeup (layers/conditional methods), as well as the usual > (supersized) portion of "I think you should do X/Y/Z instead". Which is > all OK, but we were actually hoping to get some feedback on the part > that is written up in some detail -- and mostly implemented! I know > it's fun to discuss the big sexy stuff, but we can't let this crowd out > the main goal. > > So my question is: has anyone tried to write a program with specialized > generics with the valhalla implementation? Well, we'd really > appreciate it. > > I know people want to contribute. At least right now, here's what > contribution looks like from our perspective: > > 1.) Please try it out! Write some code! > 2.) Tell us what works and what doesn't! > 3.) Nice comments are always a bonus! (And are welcome even if you > didn't do (1) and (2)). > 4.) If you can't do (1) and (2), and still have something important > to say in the big-picture department, please do it a) nicely, b) > constructively, c) succinctly. > > Thanks, > -Brian > From forax at univ-mlv.fr Sat Jan 10 09:58:46 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 10 Jan 2015 10:58:46 +0100 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54B0A14A.5000007@oracle.com> References: <54AAE765.6040402@oracle.com> <978D194CF4618446926EDAD23C949D9939BC6E71@LONURLNA08.e2k.ad.ge.com> <54B0A14A.5000007@oracle.com> Message-ID: <54B0F7D6.6070609@univ-mlv.fr> On 01/10/2015 04:49 AM, Brian Goetz wrote: > You might look at ClassValue as a means for this sort of per-class > static constants? The result of ClassValue.get() with a constant class is not a constant at least with the current implementation. R?mi > > On 1/9/2015 10:47 PM, MacGregor, Duncan (GE Energy Management) wrote: >> And another thing... >> >> If I want to try and combine the use of VarHandles and specialisation >> to make something with atomic properties (as I'm sure we'll all want >> to do for concurrent collections) then at the moment I can't use >> static fields for the var handles, and the JIT is almost certainly >> going to hate me if I they are stored on instance fields. I think >> we'll need some way to specify static fields as specialised and >> provide initialisers for them. This cannot be done by simply having >> different statics for different layers since each specialised class >> will need it's own statics. >> >> I'm sure there's also some evil I can perpetrate by specialising >> CallSites, but I haven't though of a really convincing case for doing >> so yet (at least not until I can define typed tuples when I can maybe >> replace various varargs fallbacks), and I hit the same statics >> problem as above with MethodHandles anyway. From duncan.macgregor at ge.com Sat Jan 10 12:39:35 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Sat, 10 Jan 2015 12:39:35 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54B0F7D6.6070609@univ-mlv.fr> References: <54AAE765.6040402@oracle.com> <978D194CF4618446926EDAD23C949D9939BC6E71@LONURLNA08.e2k.ad.ge.com> <54B0A14A.5000007@oracle.com>,<54B0F7D6.6070609@univ-mlv.fr> Message-ID: <978D194CF4618446926EDAD23C949D9939BD130C@LONURLNA08.e2k.ad.ge.com> There's that, and also the issue that the computeValue method will then have to use reflection to work out the specialised types. Equally I think we should consider how to support classes that want to both specialise and intern values by providing methods alongs the lines of MyType.myType(T t, ...) { } it certainly feels like something you should be able to do. Along similar lines, is there a proposal on how to handle specialised collections and things like public static Set singleton(T o) from Collections if statics are not allowed to be specialised over any T. Duncan. ________________________________________ From: valhalla-dev [valhalla-dev-bounces at openjdk.java.net] on behalf of Remi Forax [forax at univ-mlv.fr] Sent: 10 January 2015 09:58 To: valhalla-dev at openjdk.java.net Subject: Re: Can we also get some feedback on specialization, please? On 01/10/2015 04:49 AM, Brian Goetz wrote: > You might look at ClassValue as a means for this sort of per-class > static constants? The result of ClassValue.get() with a constant class is not a constant at least with the current implementation. R?mi > > On 1/9/2015 10:47 PM, MacGregor, Duncan (GE Energy Management) wrote: >> And another thing... >> >> If I want to try and combine the use of VarHandles and specialisation >> to make something with atomic properties (as I'm sure we'll all want >> to do for concurrent collections) then at the moment I can't use >> static fields for the var handles, and the JIT is almost certainly >> going to hate me if I they are stored on instance fields. I think >> we'll need some way to specify static fields as specialised and >> provide initialisers for them. This cannot be done by simply having >> different statics for different layers since each specialised class >> will need it's own statics. >> >> I'm sure there's also some evil I can perpetrate by specialising >> CallSites, but I haven't though of a really convincing case for doing >> so yet (at least not until I can define typed tuples when I can maybe >> replace various varargs fallbacks), and I hit the same statics >> problem as above with MethodHandles anyway. From richard.warburton at gmail.com Sat Jan 10 12:56:59 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Sat, 10 Jan 2015 12:56:59 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <54B02D0B.20508@oracle.com> Message-ID: Hi, May be to avoid such confusion it will be better to split project valhalla > (or just the mailing list) into separate parts: > > - value types > - generic specialization (+enhanced volatiles) > > Seems that your current focus is on generic specialization. Or are you > working in the same time also on value types? > I strongly disagree - OpenJDK mailing list bifurcation is a serious barrier to casual user contribution. Added to the fact that its not at all clear which project covers what and where the overlap is and there's no aggressive project deprecation. Offtopic - but has to be said since you raise it I'm afraid. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From peter.levart at gmail.com Sat Jan 10 13:39:27 2015 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 10 Jan 2015 14:39:27 +0100 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54B0F7D6.6070609@univ-mlv.fr> References: <54AAE765.6040402@oracle.com> <978D194CF4618446926EDAD23C949D9939BC6E71@LONURLNA08.e2k.ad.ge.com> <54B0A14A.5000007@oracle.com> <54B0F7D6.6070609@univ-mlv.fr> Message-ID: <54B12B8F.2030200@gmail.com> On 01/10/2015 10:58 AM, Remi Forax wrote: > > On 01/10/2015 04:49 AM, Brian Goetz wrote: >> You might look at ClassValue as a means for this sort of per-class >> static constants? > > The result of ClassValue.get() with a constant class is not a constant > at least with the current implementation. > > R?mi The alternative is to use the class literal as a key into CHM for example: class HolderClass { static final ConcurrentMap, X> constants = new CHM<>(); static X getConstant() { return constants.computeIfAbsent(HolderClass.class, (c) -> { ... }); } See also here, when you know the values for all specializations upfront: http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/util/AnyHelper.java Regards, Peter > >> >> On 1/9/2015 10:47 PM, MacGregor, Duncan (GE Energy Management) wrote: >>> And another thing... >>> >>> If I want to try and combine the use of VarHandles and specialisation >>> to make something with atomic properties (as I'm sure we'll all want >>> to do for concurrent collections) then at the moment I can't use >>> static fields for the var handles, and the JIT is almost certainly >>> going to hate me if I they are stored on instance fields. I think >>> we'll need some way to specify static fields as specialised and >>> provide initialisers for them. This cannot be done by simply having >>> different statics for different layers since each specialised class >>> will need it's own statics. >>> >>> I'm sure there's also some evil I can perpetrate by specialising >>> CallSites, but I haven't though of a really convincing case for doing >>> so yet (at least not until I can define typed tuples when I can maybe >>> replace various varargs fallbacks), and I hit the same statics >>> problem as above with MethodHandles anyway. > From peter.levart at gmail.com Sat Jan 10 14:20:35 2015 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 10 Jan 2015 15:20:35 +0100 Subject: hg: valhalla/valhalla/jdk: Fix specializer bug reported by Peter Levart related to improper handling of not-fully-instantiated class literals In-Reply-To: <201501092144.t09Lih7T024582@aojmv0008> References: <201501092144.t09Lih7T024582@aojmv0008> Message-ID: <54B13533.2010605@gmail.com> On 01/09/2015 10:44 PM, brian.goetz at oracle.com wrote: > Changeset: 7a3e4270521d > Author: briangoetz > Date: 2015-01-09 16:44 -0500 > URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/7a3e4270521d > > Fix specializer bug reported by Peter Levart related to improper handling of not-fully-instantiated class literals > > ! src/java.base/share/classes/valhalla/specializer/Specializer.java > ! test/valhalla/test/valhalla/specializer/GenericMethodsTest.java > Thanks, I have another one. This time I think there's a problem with array initializer in specialized code: public class Pair { T[] array; public Pair(T v1, T v2) { array = new T[]{v1, v2}; System.out.println("Ok for " + getClass()); } public static void main(String[] args) { new Pair("aaa", "bbb"); new Pair(1, 2); new Pair(1.1f, 2.2f); } } This is the runtime failure: Ok for class Pair Specializing Pair${0=I}; searching for Pair.class (not found) Specializing Pair${0=I}; searching for Pair.class (found) Exception in thread "main" java.lang.VerifyError: Bad type on operand stack Exception Details: Location: Pair${0=I}.(II)V @11: aastore Reason: Type integer (current frame, stack[4]) is not assignable to 'java/lang/Object' Current Frame: bci: @11 flags: { } locals: { 'Pair${0=I}', integer, integer } stack: { 'Pair${0=I}', '[I', '[I', integer, integer } Bytecode: 0000000: 2ab7 000d 2a05 bc0a 5903 1b53 5904 1c53 0000010: b500 0fb2 0015 bb00 1759 b700 1812 1ab6 0000020: 001e 2ab6 0022 b600 25b6 0029 b600 2fb1 0000030: at Pair.main(Pair.java:15) Regards, Peter From peter.levart at gmail.com Sat Jan 10 17:50:46 2015 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 10 Jan 2015 18:50:46 +0100 Subject: Array initializer aastores missing BMAs In-Reply-To: <54B13533.2010605@gmail.com> References: <201501092144.t09Lih7T024582@aojmv0008> <54B13533.2010605@gmail.com> Message-ID: <54B16676.3000506@gmail.com> Hi, The following: public class Pair { A[] array; public Pair(A v1, A v2) { array = new A[]{v1, v2}; } } vs. public class Pair { A[] array; public Pair(A v1, A v2) { array = new A[2]; array[0] = v1; array[1] = v2; } } Produces the following constructor: public Pair(A, A); descriptor: (Ljava/lang/Object;Ljava/lang/Object;)V flags: ACC_PUBLIC Code: stack=5, locals=3, args_size=3 0: aload_0 1: invokespecial #1 // Method java/lang/Object."":()V 4: aload_0 5: iconst_2 6: anewarray #2 // class java/lang/Object 9: dup 10: iconst_0 11: aload_1 12: aastore 13: dup 14: iconst_1 15: aload_2 16: aastore 17: putfield #3 // Field array:[Ljava/lang/Object; 20: return LineNumberTable: line 8: 0 line 9: 4 line 10: 20 BytecodeMapping: Code_idx Signature 6: [TA; 11: TA; 15: TA; 17: LPair;::[TA; Signature: #25 // (TA;TA;)V vs. public Pair(A, A); descriptor: (Ljava/lang/Object;Ljava/lang/Object;)V flags: ACC_PUBLIC Code: stack=3, locals=3, args_size=3 0: aload_0 1: invokespecial #1 // Method java/lang/Object."":()V 4: aload_0 5: iconst_2 6: anewarray #2 // class java/lang/Object 9: putfield #3 // Field array:[Ljava/lang/Object; 12: aload_0 13: getfield #3 // Field array:[Ljava/lang/Object; 16: iconst_0 17: aload_1 18: aastore 19: aload_0 20: getfield #3 // Field array:[Ljava/lang/Object; 23: iconst_1 24: aload_2 25: aastore 26: return LineNumberTable: line 8: 0 line 9: 4 line 10: 12 line 11: 19 line 12: 26 BytecodeMapping: Code_idx Signature 6: [TA; 9: LPair;::[TA; 13: LPair;::[TA; 17: TA; 18: TA; 20: LPair;::[TA; 24: TA; 25: TA; Signature: #25 // (TA;TA;)V Seems like aastore(s) at BCI(s) 12 & 16 are missing BMA(s) in the variant using array initializer... Regards, Peter On 01/10/2015 03:20 PM, Peter Levart wrote: > > On 01/09/2015 10:44 PM, brian.goetz at oracle.com wrote: >> Changeset: 7a3e4270521d >> Author: briangoetz >> Date: 2015-01-09 16:44 -0500 >> URL:http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/7a3e4270521d >> >> Fix specializer bug reported by Peter Levart related to improper handling of not-fully-instantiated class literals >> >> ! src/java.base/share/classes/valhalla/specializer/Specializer.java >> ! test/valhalla/test/valhalla/specializer/GenericMethodsTest.java >> > > Thanks, I have another one. This time I think there's a problem with > array initializer in specialized code: > > public class Pair { > > T[] array; > > public Pair(T v1, T v2) { > array = new T[]{v1, v2}; > System.out.println("Ok for " + getClass()); > } > > public static void main(String[] args) { > new Pair("aaa", "bbb"); > new Pair(1, 2); > new Pair(1.1f, 2.2f); > } > } > > > This is the runtime failure: > > Ok for class Pair > Specializing Pair${0=I}; searching for Pair.class (not found) > Specializing Pair${0=I}; searching for Pair.class (found) > Exception in thread "main" java.lang.VerifyError: Bad type on operand > stack > Exception Details: > Location: > Pair${0=I}.(II)V @11: aastore > Reason: > Type integer (current frame, stack[4]) is not assignable to > 'java/lang/Object' > Current Frame: > bci: @11 > flags: { } > locals: { 'Pair${0=I}', integer, integer } > stack: { 'Pair${0=I}', '[I', '[I', integer, integer } > Bytecode: > 0000000: 2ab7 000d 2a05 bc0a 5903 1b53 5904 1c53 > 0000010: b500 0fb2 0015 bb00 1759 b700 1812 1ab6 > 0000020: 001e 2ab6 0022 b600 25b6 0029 b600 2fb1 > 0000030: > > at Pair.main(Pair.java:15) > > > Regards, Peter > From brian.goetz at oracle.com Sat Jan 10 20:46:02 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 10 Jan 2015 15:46:02 -0500 Subject: Array initializer aastores missing BMAs In-Reply-To: <54B16676.3000506@gmail.com> References: <201501092144.t09Lih7T024582@aojmv0008> <54B13533.2010605@gmail.com> <54B16676.3000506@gmail.com> Message-ID: <54B18F8A.6030805@oracle.com> Good catch. The specialized *should* be ready to act on these if they are present, so hopefully this is a compiler-only fix. On 1/10/2015 12:50 PM, Peter Levart wrote: > Hi, > > The following: > > public class Pair { > > A[] array; > > public Pair(A v1, A v2) { > array = new A[]{v1, v2}; > } > } > > vs. > > public class Pair { > > A[] array; > > public Pair(A v1, A v2) { > array = new A[2]; > array[0] = v1; > array[1] = v2; > } > } > > Produces the following constructor: > > public Pair(A, A); > descriptor: (Ljava/lang/Object;Ljava/lang/Object;)V > flags: ACC_PUBLIC > Code: > stack=5, locals=3, args_size=3 > 0: aload_0 > 1: invokespecial #1 // Method > java/lang/Object."":()V > 4: aload_0 > 5: iconst_2 > 6: anewarray #2 // class java/lang/Object > 9: dup > 10: iconst_0 > 11: aload_1 > 12: aastore > 13: dup > 14: iconst_1 > 15: aload_2 > 16: aastore > 17: putfield #3 // Field > array:[Ljava/lang/Object; > 20: return > LineNumberTable: > line 8: 0 > line 9: 4 > line 10: 20 > BytecodeMapping: > Code_idx Signature > 6: [TA; > 11: TA; > 15: TA; > 17: LPair;::[TA; > Signature: #25 // (TA;TA;)V > > vs. > > public Pair(A, A); > descriptor: (Ljava/lang/Object;Ljava/lang/Object;)V > flags: ACC_PUBLIC > Code: > stack=3, locals=3, args_size=3 > 0: aload_0 > 1: invokespecial #1 // Method > java/lang/Object."":()V > 4: aload_0 > 5: iconst_2 > 6: anewarray #2 // class java/lang/Object > 9: putfield #3 // Field > array:[Ljava/lang/Object; > 12: aload_0 > 13: getfield #3 // Field > array:[Ljava/lang/Object; > 16: iconst_0 > 17: aload_1 > 18: aastore > 19: aload_0 > 20: getfield #3 // Field > array:[Ljava/lang/Object; > 23: iconst_1 > 24: aload_2 > 25: aastore > 26: return > LineNumberTable: > line 8: 0 > line 9: 4 > line 10: 12 > line 11: 19 > line 12: 26 > BytecodeMapping: > Code_idx Signature > 6: [TA; > 9: LPair;::[TA; > 13: LPair;::[TA; > 17: TA; > 18: TA; > 20: LPair;::[TA; > 24: TA; > 25: TA; > Signature: #25 // (TA;TA;)V > > > > Seems like aastore(s) at BCI(s) 12 & 16 are missing BMA(s) in the > variant using array initializer... > > > Regards, Peter > > On 01/10/2015 03:20 PM, Peter Levart wrote: >> >> On 01/09/2015 10:44 PM, brian.goetz at oracle.com wrote: >>> Changeset: 7a3e4270521d >>> Author: briangoetz >>> Date: 2015-01-09 16:44 -0500 >>> URL:http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/7a3e4270521d >>> >>> Fix specializer bug reported by Peter Levart related to improper handling of not-fully-instantiated class literals >>> >>> ! src/java.base/share/classes/valhalla/specializer/Specializer.java >>> ! test/valhalla/test/valhalla/specializer/GenericMethodsTest.java >>> >> >> Thanks, I have another one. This time I think there's a problem with >> array initializer in specialized code: >> >> public class Pair { >> >> T[] array; >> >> public Pair(T v1, T v2) { >> array = new T[]{v1, v2}; >> System.out.println("Ok for " + getClass()); >> } >> >> public static void main(String[] args) { >> new Pair("aaa", "bbb"); >> new Pair(1, 2); >> new Pair(1.1f, 2.2f); >> } >> } >> >> >> This is the runtime failure: >> >> Ok for class Pair >> Specializing Pair${0=I}; searching for Pair.class (not found) >> Specializing Pair${0=I}; searching for Pair.class (found) >> Exception in thread "main" java.lang.VerifyError: Bad type on operand >> stack >> Exception Details: >> Location: >> Pair${0=I}.(II)V @11: aastore >> Reason: >> Type integer (current frame, stack[4]) is not assignable to >> 'java/lang/Object' >> Current Frame: >> bci: @11 >> flags: { } >> locals: { 'Pair${0=I}', integer, integer } >> stack: { 'Pair${0=I}', '[I', '[I', integer, integer } >> Bytecode: >> 0000000: 2ab7 000d 2a05 bc0a 5903 1b53 5904 1c53 >> 0000010: b500 0fb2 0015 bb00 1759 b700 1812 1ab6 >> 0000020: 001e 2ab6 0022 b600 25b6 0029 b600 2fb1 >> 0000030: >> >> at Pair.main(Pair.java:15) >> >> >> Regards, Peter >> > From palo.marton at gmail.com Sat Jan 10 21:01:40 2015 From: palo.marton at gmail.com (Palo Marton) Date: Sat, 10 Jan 2015 22:01:40 +0100 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54B12B8F.2030200@gmail.com> References: <54AAE765.6040402@oracle.com> <978D194CF4618446926EDAD23C949D9939BC6E71@LONURLNA08.e2k.ad.ge.com> <54B0A14A.5000007@oracle.com> <54B0F7D6.6070609@univ-mlv.fr> <54B12B8F.2030200@gmail.com> Message-ID: CHM will prevent class unloading and that can be a problem for long running server. On Saturday, January 10, 2015, Peter Levart wrote: > > On 01/10/2015 10:58 AM, Remi Forax wrote: > >> >> On 01/10/2015 04:49 AM, Brian Goetz wrote: >> >>> You might look at ClassValue as a means for this sort of per-class >>> static constants? >>> >> >> The result of ClassValue.get() with a constant class is not a constant at >> least with the current implementation. >> >> R?mi >> > > The alternative is to use the class literal as a key into CHM for example: > > class HolderClass { > static final ConcurrentMap, X> constants = new CHM<>(); > > static X getConstant() { > return constants.computeIfAbsent(HolderClass.class, (c) -> { > ... }); > } > > > See also here, when you know the values for all specializations upfront: > > http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/util/AnyHelper.java > > > Regards, Peter > > >> >>> On 1/9/2015 10:47 PM, MacGregor, Duncan (GE Energy Management) wrote: >>> >>>> And another thing... >>>> >>>> If I want to try and combine the use of VarHandles and specialisation >>>> to make something with atomic properties (as I'm sure we'll all want >>>> to do for concurrent collections) then at the moment I can't use >>>> static fields for the var handles, and the JIT is almost certainly >>>> going to hate me if I they are stored on instance fields. I think >>>> we'll need some way to specify static fields as specialised and >>>> provide initialisers for them. This cannot be done by simply having >>>> different statics for different layers since each specialised class >>>> will need it's own statics. >>>> >>>> I'm sure there's also some evil I can perpetrate by specialising >>>> CallSites, but I haven't though of a really convincing case for doing >>>> so yet (at least not until I can define typed tuples when I can maybe >>>> replace various varargs fallbacks), and I hit the same statics >>>> problem as above with MethodHandles anyway. >>>> >>> >> > From duncan.macgregor at ge.com Sat Jan 10 21:06:12 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Sat, 10 Jan 2015 21:06:12 +0000 Subject: Can we also get some feedback on specialization, please? In-Reply-To: <54B12B8F.2030200@gmail.com> References: <54AAE765.6040402@oracle.com> <978D194CF4618446926EDAD23C949D9939BC6E71@LONURLNA08.e2k.ad.ge.com> <54B0A14A.5000007@oracle.com> <54B0F7D6.6070609@univ-mlv.fr>,<54B12B8F.2030200@gmail.com> Message-ID: <978D194CF4618446926EDAD23C949D9939BD634F@LONURLNA08.e2k.ad.ge.com> Ah, I hadn't realised static methods could be specialised? that probably need to be called out in Brain's doc in the Statics section. As Remi has said the class value approach doesn't produce something that is a constant, and nor does the CHM method, so I would not expect either to be a good basis for performant code, at least at this time. Maybe statics on the specialised classes isn't the right answer, but there are good reasons for wanting to be able to specialise static fields in some way, especially final ones which can be seen by the JIT to be constant. Maybe allow the addition of type variables to their declaration and reference? Duncan ________________________________________ From: valhalla-dev [valhalla-dev-bounces at openjdk.java.net] on behalf of Peter Levart [peter.levart at gmail.com] Sent: 10 January 2015 13:39 To: Remi Forax; valhalla-dev at openjdk.java.net Subject: Re: Can we also get some feedback on specialization, please? On 01/10/2015 10:58 AM, Remi Forax wrote: > > On 01/10/2015 04:49 AM, Brian Goetz wrote: >> You might look at ClassValue as a means for this sort of per-class >> static constants? > > The result of ClassValue.get() with a constant class is not a constant > at least with the current implementation. > > R?mi The alternative is to use the class literal as a key into CHM for example: class HolderClass { static final ConcurrentMap, X> constants = new CHM<>(); static X getConstant() { return constants.computeIfAbsent(HolderClass.class, (c) -> { ... }); } See also here, when you know the values for all specializations upfront: http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/util/AnyHelper.java Regards, Peter > >> >> On 1/9/2015 10:47 PM, MacGregor, Duncan (GE Energy Management) wrote: >>> And another thing... >>> >>> If I want to try and combine the use of VarHandles and specialisation >>> to make something with atomic properties (as I'm sure we'll all want >>> to do for concurrent collections) then at the moment I can't use >>> static fields for the var handles, and the JIT is almost certainly >>> going to hate me if I they are stored on instance fields. I think >>> we'll need some way to specify static fields as specialised and >>> provide initialisers for them. This cannot be done by simply having >>> different statics for different layers since each specialised class >>> will need it's own statics. >>> >>> I'm sure there's also some evil I can perpetrate by specialising >>> CallSites, but I haven't though of a really convincing case for doing >>> so yet (at least not until I can define typed tuples when I can maybe >>> replace various varargs fallbacks), and I hit the same statics >>> problem as above with MethodHandles anyway. > From palo.marton at gmail.com Sat Jan 10 21:34:40 2015 From: palo.marton at gmail.com (Palo Marton) Date: Sat, 10 Jan 2015 22:34:40 +0100 Subject: Static fields and specialization Message-ID: If static fields are not specialized, how will you handle things like Optional.EMPTY? From benlewisj at gmail.com Sat Jan 10 22:08:46 2015 From: benlewisj at gmail.com (Ben Lewis) Date: Sun, 11 Jan 2015 11:08:46 +1300 Subject: Static fields and specialization In-Reply-To: References: Message-ID: Static fields cannot be specialized as what type would the field be Optional, Optional or Optional. This approach has problems with generics anyways, both Optional.empty() and Collections.empty*() are methods to maintain type safety as stated in the javadocs. On Sun, Jan 11, 2015 at 10:34 AM, Palo Marton wrote: > If static fields are not specialized, how will you handle things > like Optional.EMPTY? > From peter.levart at gmail.com Sat Jan 10 22:16:54 2015 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 10 Jan 2015 23:16:54 +0100 Subject: Can we also get some feedback on specialization, please? In-Reply-To: References: <54AAE765.6040402@oracle.com> <978D194CF4618446926EDAD23C949D9939BC6E71@LONURLNA08.e2k.ad.ge.com> <54B0A14A.5000007@oracle.com> <54B0F7D6.6070609@univ-mlv.fr> <54B12B8F.2030200@gmail.com> Message-ID: <54B1A4D6.4090701@gmail.com> On 01/10/2015 10:01 PM, Palo Marton wrote: > CHM will prevent class unloading and that can be a problem for long > running server. In this particular case the specialized classes will be referenced from static field of non-specialized class, all of them loaded by same ClassLoader and all of them eligible for GC at the same time. Peter > > On Saturday, January 10, 2015, Peter Levart > wrote: > > > On 01/10/2015 10:58 AM, Remi Forax wrote: > > > On 01/10/2015 04:49 AM, Brian Goetz wrote: > > You might look at ClassValue as a means for this sort of > per-class > static constants? > > > The result of ClassValue.get() with a constant class is not a > constant at least with the current implementation. > > R?mi > > > The alternative is to use the class literal as a key into CHM for > example: > > class HolderClass { > static final ConcurrentMap, X> constants = new CHM<>(); > > static X getConstant() { > return constants.computeIfAbsent(HolderClass.class, (c) > -> { ... }); > } > > > See also here, when you know the values for all specializations > upfront: > > http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/util/AnyHelper.java > > > > Regards, Peter > > > > On 1/9/2015 10:47 PM, MacGregor, Duncan (GE Energy > Management) wrote: > > And another thing... > > If I want to try and combine the use of VarHandles and > specialisation > to make something with atomic properties (as I'm sure > we'll all want > to do for concurrent collections) then at the moment I > can't use > static fields for the var handles, and the JIT is > almost certainly > going to hate me if I they are stored on instance > fields. I think > we'll need some way to specify static fields as > specialised and > provide initialisers for them. This cannot be done by > simply having > different statics for different layers since each > specialised class > will need it's own statics. > > I'm sure there's also some evil I can perpetrate by > specialising > CallSites, but I haven't though of a really convincing > case for doing > so yet (at least not until I can define typed tuples > when I can maybe > replace various varargs fallbacks), and I hit the same > statics > problem as above with MethodHandles anyway. > > > From maurizio.cimadamore at oracle.com Sat Jan 10 22:17:19 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Sat, 10 Jan 2015 22:17:19 +0000 Subject: Array initializer aastores missing BMAs In-Reply-To: <54B18F8A.6030805@oracle.com> References: <201501092144.t09Lih7T024582@aojmv0008> <54B13533.2010605@gmail.com> <54B16676.3000506@gmail.com> <54B18F8A.6030805@oracle.com> Message-ID: <54B1A4EF.90101@oracle.com> On 10/01/15 20:46, Brian Goetz wrote: > Good catch. The specialized *should* be ready to act on these if they > are present, so hopefully this is a compiler-only fix. Thanks - as you can see not all the possible paths are adequately covered; this is very useful feedback, please keep at it! Maurizio > > On 1/10/2015 12:50 PM, Peter Levart wrote: >> Hi, >> >> The following: >> >> public class Pair { >> >> A[] array; >> >> public Pair(A v1, A v2) { >> array = new A[]{v1, v2}; >> } >> } >> >> vs. >> >> public class Pair { >> >> A[] array; >> >> public Pair(A v1, A v2) { >> array = new A[2]; >> array[0] = v1; >> array[1] = v2; >> } >> } >> >> Produces the following constructor: >> >> public Pair(A, A); >> descriptor: (Ljava/lang/Object;Ljava/lang/Object;)V >> flags: ACC_PUBLIC >> Code: >> stack=5, locals=3, args_size=3 >> 0: aload_0 >> 1: invokespecial #1 // Method >> java/lang/Object."":()V >> 4: aload_0 >> 5: iconst_2 >> 6: anewarray #2 // class java/lang/Object >> 9: dup >> 10: iconst_0 >> 11: aload_1 >> 12: aastore >> 13: dup >> 14: iconst_1 >> 15: aload_2 >> 16: aastore >> 17: putfield #3 // Field >> array:[Ljava/lang/Object; >> 20: return >> LineNumberTable: >> line 8: 0 >> line 9: 4 >> line 10: 20 >> BytecodeMapping: >> Code_idx Signature >> 6: [TA; >> 11: TA; >> 15: TA; >> 17: LPair;::[TA; >> Signature: #25 // (TA;TA;)V >> >> vs. >> >> public Pair(A, A); >> descriptor: (Ljava/lang/Object;Ljava/lang/Object;)V >> flags: ACC_PUBLIC >> Code: >> stack=3, locals=3, args_size=3 >> 0: aload_0 >> 1: invokespecial #1 // Method >> java/lang/Object."":()V >> 4: aload_0 >> 5: iconst_2 >> 6: anewarray #2 // class java/lang/Object >> 9: putfield #3 // Field >> array:[Ljava/lang/Object; >> 12: aload_0 >> 13: getfield #3 // Field >> array:[Ljava/lang/Object; >> 16: iconst_0 >> 17: aload_1 >> 18: aastore >> 19: aload_0 >> 20: getfield #3 // Field >> array:[Ljava/lang/Object; >> 23: iconst_1 >> 24: aload_2 >> 25: aastore >> 26: return >> LineNumberTable: >> line 8: 0 >> line 9: 4 >> line 10: 12 >> line 11: 19 >> line 12: 26 >> BytecodeMapping: >> Code_idx Signature >> 6: [TA; >> 9: LPair;::[TA; >> 13: LPair;::[TA; >> 17: TA; >> 18: TA; >> 20: LPair;::[TA; >> 24: TA; >> 25: TA; >> Signature: #25 // (TA;TA;)V >> >> >> >> Seems like aastore(s) at BCI(s) 12 & 16 are missing BMA(s) in the >> variant using array initializer... >> >> >> Regards, Peter >> >> On 01/10/2015 03:20 PM, Peter Levart wrote: >>> >>> On 01/09/2015 10:44 PM, brian.goetz at oracle.com wrote: >>>> Changeset: 7a3e4270521d >>>> Author: briangoetz >>>> Date: 2015-01-09 16:44 -0500 >>>> URL:http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/7a3e4270521d >>>> >>>> Fix specializer bug reported by Peter Levart related to improper >>>> handling of not-fully-instantiated class literals >>>> >>>> ! src/java.base/share/classes/valhalla/specializer/Specializer.java >>>> ! test/valhalla/test/valhalla/specializer/GenericMethodsTest.java >>>> >>> >>> Thanks, I have another one. This time I think there's a problem with >>> array initializer in specialized code: >>> >>> public class Pair { >>> >>> T[] array; >>> >>> public Pair(T v1, T v2) { >>> array = new T[]{v1, v2}; >>> System.out.println("Ok for " + getClass()); >>> } >>> >>> public static void main(String[] args) { >>> new Pair("aaa", "bbb"); >>> new Pair(1, 2); >>> new Pair(1.1f, 2.2f); >>> } >>> } >>> >>> >>> This is the runtime failure: >>> >>> Ok for class Pair >>> Specializing Pair${0=I}; searching for Pair.class (not found) >>> Specializing Pair${0=I}; searching for Pair.class (found) >>> Exception in thread "main" java.lang.VerifyError: Bad type on operand >>> stack >>> Exception Details: >>> Location: >>> Pair${0=I}.(II)V @11: aastore >>> Reason: >>> Type integer (current frame, stack[4]) is not assignable to >>> 'java/lang/Object' >>> Current Frame: >>> bci: @11 >>> flags: { } >>> locals: { 'Pair${0=I}', integer, integer } >>> stack: { 'Pair${0=I}', '[I', '[I', integer, integer } >>> Bytecode: >>> 0000000: 2ab7 000d 2a05 bc0a 5903 1b53 5904 1c53 >>> 0000010: b500 0fb2 0015 bb00 1759 b700 1812 1ab6 >>> 0000020: 001e 2ab6 0022 b600 25b6 0029 b600 2fb1 >>> 0000030: >>> >>> at Pair.main(Pair.java:15) >>> >>> >>> Regards, Peter >>> >> From brian.goetz at oracle.com Sat Jan 10 22:18:08 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 10 Jan 2015 17:18:08 -0500 Subject: Static fields and specialization In-Reply-To: References: Message-ID: <54B1A520.2020006@oracle.com> I think his question was: how will we maintain the optimization where Optional.empty() returns a statically-allocated, shared singleton instance when the value being returned is type-specific. On 1/10/2015 5:08 PM, Ben Lewis wrote: > Static fields cannot be specialized as what type would the field be > Optional, Optional or Optional. This approach has > problems with generics anyways, both Optional.empty() and > Collections.empty*() are methods to maintain type safety as stated in the > javadocs. > > On Sun, Jan 11, 2015 at 10:34 AM, Palo Marton wrote: > >> If static fields are not specialized, how will you handle things >> like Optional.EMPTY? >> From palo.marton at gmail.com Sat Jan 10 22:23:51 2015 From: palo.marton at gmail.com (Palo Marton) Date: Sat, 10 Jan 2015 23:23:51 +0100 Subject: Static fields and specialization In-Reply-To: <54B1A520.2020006@oracle.com> References: <54B1A520.2020006@oracle.com> Message-ID: Yes. I think there is no other solution to this except to somehow allow some static fields to be specialized. Soutions with CHM or ClassValue will mean serious performance degradation. On Saturday, January 10, 2015, Brian Goetz wrote: > I think his question was: how will we maintain the optimization where > Optional.empty() returns a statically-allocated, shared singleton instance > when the value being returned is type-specific. > > On 1/10/2015 5:08 PM, Ben Lewis wrote: > >> Static fields cannot be specialized as what type would the field be >> Optional, Optional or Optional. This approach has >> problems with generics anyways, both Optional.empty() and >> Collections.empty*() are methods to maintain type safety as stated in the >> javadocs. >> >> On Sun, Jan 11, 2015 at 10:34 AM, Palo Marton >> wrote: >> >> If static fields are not specialized, how will you handle things >>> like Optional.EMPTY? >>> >>> From alahijani at gmail.com Sun Jan 11 06:33:29 2015 From: alahijani at gmail.com (Ali Lahijani) Date: Sat, 10 Jan 2015 22:33:29 -0800 Subject: Usage-site value types: "lowercase" string Message-ID: Hi, This was originally a question asked by Palo Marton, but that thread digressed into another discussion, so I want to ask it again here. Brian has hinted that String may be a lost cause. But to me, it seems that there is a theoretical migration path for existing final immutable classes like String, which unlike Optional, have been used since the 90's with synchronized and ==. The idea is to move __ByValue from declaration-site to usage-site. One will keep the definition of String as is, and "String" in source code will continue to mean Ljava/lang/String; supporting all reference operations including synchronized. But __ByValue String (or a better syntax like Palo's lowercase string) will denote Qjava/lang/String;. Method calls on __ByValue String, including intern(), will be functionally equivalent to method calls on the boxed version. The slogan would be: Declaration-site like a class, usage-site like an int! (Note: Here __ByValue is like a type annotation, not a declaration annotation, so for example you can say List<__ByValue String>). To enforce the structural constraints on String, an annotation can be used, similar to JDK 8's @FunctionalInterface: @ValueClass public final class String {... @ValueClass checks the same structural constraints as the current proposal's declaration-site __ByValue does. But besides that, it does nothing. In particular, adding @ValueClass to an existing class does not change the validity or meaning of any Java program or class file that was valid before. It's source- and binary-compatible. On top of @ValueClass and usage-site __ByValue, more appealing high level language features can be implemented in the compiler. The compiler can easily implement the current proposal's declaration-site value types (immutable C# structs) on top of usage-site value types. From david.holmes at oracle.com Mon Jan 12 04:36:10 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Jan 2015 14:36:10 +1000 Subject: Value types - compatibility with existing =?UTF-8?B?4oCcdmFs?= =?UTF-8?B?dWUgb2JqZWN0c+KAnQ==?= In-Reply-To: <54AFDF57.9000003@ochsenreither.de> References: <54AE8C27.50400@gmail.com> <54AEAA16.2010306@oracle.com> <54AEB40C.7030702@oracle.com> <756B639C-2DF4-498F-AC1A-8350AF4AF787@oracle.com> <21AE20E5-A557-4C9B-A669-AC656DD89C76@oracle.com> <54AF357F.9060604@redhat.com> <54AFDF57.9000003@ochsenreither.de> Message-ID: <54B34F3A.4030700@oracle.com> On 10/01/2015 12:01 AM, Simon Ochsenreither wrote: >> Worth mentioning that SOE is infinitely more recoverable than heap OOME. >> In the latter case (especially with complex systems, where the risk is >> ironically even higher of it happening) there's often not much you can >> do other than kill off the JVM and try again. But if you run out of >> stack, everything should unwind in a very predictable (and fast) manner. >> You could even automatically rebuild your thread pools with larger >> stack sizes fairly easily. > > Your VM might be broken after both. SOE is definitely not recoverable. > Yes, it might work, but no guarantees it does. > > Money quote: "You will find occurrences of StackOverflowError leaving > locks locked, monitors acquired, and "finally" block not invoked, even > if JVM doesn't crash." "Locks locked" is true for ReeentrantLock because an unlock can throw SOE when trying to set the owner to null - this leaves the lock "unlocked" but with an apparent owner and so no lock() will then succeed. This is a telling example of how SOE can result in broken invariants, and there is no simple fix. It is not true for monitors or finally blocks. Of course if the finally block contains a call then it too could trigger a secondary SOE. > Maybe things have changed since then? Since when? I've never known it to be true for monitors or finally blocks. David From richard.warburton at gmail.com Mon Jan 12 11:35:24 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Mon, 12 Jan 2015 11:35:24 +0000 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> Message-ID: Hi, I think there is no other solution to this except to somehow allow some > static fields to be specialized. Soutions with CHM or ClassValue will mean > serious performance degradation. > In the case of Optional.empty() and others like Collections.emptyList() it's a method that you're invoking. There is no need to implement static field specialization. You can just get your specialized implementations to return different singleton instances. Ok - so this means you now have 9 empty instances rather than 1 but that's basically nothing in terms of memory consumption. In the case of Optional you already have manual specialisation for 3 primitive cases in the form of Optionalint and friends so its really 9 rather than 4 anyway. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From palo.marton at gmail.com Mon Jan 12 11:44:09 2015 From: palo.marton at gmail.com (Palo Marton) Date: Mon, 12 Jan 2015 12:44:09 +0100 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> Message-ID: Yes, but you can not create specialized implementations for user defined value types. So these will be left with much slower implementation. Singleton implementation compiles to just 0-1 instructions and implementation that allocates new instance will be much slower. And other problem - such approach is against goals of specialization: You will have to write separate code for each primitive type. Pavol Marton, aSc www.asctimetables.com > In the case of Optional.empty() and others like Collections.emptyList() > it's a method that you're invoking. There is no need to implement static > field specialization. You can just get your specialized implementations to > return different singleton instances. Ok - so this means you now have 9 > empty instances rather than 1 but that's basically nothing in terms of > memory consumption. In the case of Optional you already have manual > specialisation for 3 primitive cases in the form of Optionalint and friends > so its really 9 rather than 4 anyway. > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > From palo.marton at gmail.com Mon Jan 12 12:51:11 2015 From: palo.marton at gmail.com (Palo Marton) Date: Mon, 12 Jan 2015 13:51:11 +0100 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> Message-ID: One possible solution how to support specialized static fields is to use same approach as with generic methods. E.g. with this syntax: private static final Optional EMPTY = new Optional(); This will compile initializing expression to generic static method: private static Optional EMPTY$init() { return new Optional(); } And all get/set access to EMPTY will use invocedynamic. Bootsrap will call Optional.EMPTY$init() to get initial value, store it in some holder object on heap (eg Variable) and returm get/set MethodHandle for that field. On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton wrote: > Yes, but you can not create specialized implementations for user defined > value types. So these will be left with much slower implementation. > Singleton implementation compiles to just 0-1 instructions and > implementation that allocates new instance will be much slower. > > And other problem - such approach is against goals of specialization: You > will have to write separate code for each primitive type. > > Pavol Marton, aSc > www.asctimetables.com > > >> In the case of Optional.empty() and others like Collections.emptyList() >> it's a method that you're invoking. There is no need to implement static >> field specialization. You can just get your specialized implementations to >> return different singleton instances. Ok - so this means you now have 9 >> empty instances rather than 1 but that's basically nothing in terms of >> memory consumption. In the case of Optional you already have manual >> specialisation for 3 primitive cases in the form of Optionalint and friends >> so its really 9 rather than 4 anyway. >> >> regards, >> >> Richard Warburton >> >> http://insightfullogic.com >> @RichardWarburto >> > > From forax at univ-mlv.fr Mon Jan 12 13:26:53 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 12 Jan 2015 14:26:53 +0100 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> Message-ID: <54B3CB9D.5000007@univ-mlv.fr> On 01/12/2015 01:51 PM, Palo Marton wrote: > One possible solution how to support specialized static fields is to use > same approach as with generic methods. E.g. with this syntax: > > private static final Optional EMPTY = new Optional(); > > This will compile initializing expression to generic static method: > > private static Optional EMPTY$init() { > return new Optional(); > } > > And all get/set access to EMPTY will use invocedynamic. Bootsrap will > call Optional.EMPTY$init() to get initial value, store it in some holder > object on heap (eg Variable) and returm get/set MethodHandle for that > field. You don't need any holder, there is a special MethodHandle named MethodHandles.constant() which is able to always return the same value, the JIT will consider the value as a true constant. R?mi > > > > > > On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton wrote: > >> Yes, but you can not create specialized implementations for user defined >> value types. So these will be left with much slower implementation. >> Singleton implementation compiles to just 0-1 instructions and >> implementation that allocates new instance will be much slower. >> >> And other problem - such approach is against goals of specialization: You >> will have to write separate code for each primitive type. >> >> Pavol Marton, aSc >> www.asctimetables.com >> >> >>> In the case of Optional.empty() and others like Collections.emptyList() >>> it's a method that you're invoking. There is no need to implement static >>> field specialization. You can just get your specialized implementations to >>> return different singleton instances. Ok - so this means you now have 9 >>> empty instances rather than 1 but that's basically nothing in terms of >>> memory consumption. In the case of Optional you already have manual >>> specialisation for 3 primitive cases in the form of Optionalint and friends >>> so its really 9 rather than 4 anyway. >>> >>> regards, >>> >>> Richard Warburton >>> >>> http://insightfullogic.com >>> @RichardWarburto >>> >> From duncan.macgregor at ge.com Mon Jan 12 13:26:43 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Mon, 12 Jan 2015 13:26:43 +0000 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> Message-ID: That sounds fundamentally okay, but probably also requires a typed static initialiser syntax for fields that could throw an exception during initialisation. Something like static final MethodHandle mh; static { try { // Do MethodHandle lookup } (catch e) { Throw new Error(); } } Duncan. On 12/01/2015 12:51, "Palo Marton" wrote: >One possible solution how to support specialized static fields is to use >same approach as with generic methods. E.g. with this syntax: > >private static final Optional EMPTY = new Optional(); > >This will compile initializing expression to generic static method: > >private static Optional EMPTY$init() { > return new Optional(); >} > >And all get/set access to EMPTY will use invocedynamic. Bootsrap will >call Optional.EMPTY$init() to get initial value, store it in some >holder >object on heap (eg Variable) and returm get/set MethodHandle for that >field. > > > > > >On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton >wrote: > >> Yes, but you can not create specialized implementations for user defined >> value types. So these will be left with much slower implementation. >> Singleton implementation compiles to just 0-1 instructions and >> implementation that allocates new instance will be much slower. >> >> And other problem - such approach is against goals of specialization: >>You >> will have to write separate code for each primitive type. >> >> Pavol Marton, aSc >> www.asctimetables.com >> >> >>> In the case of Optional.empty() and others like Collections.emptyList() >>> it's a method that you're invoking. There is no need to implement >>>static >>> field specialization. You can just get your specialized >>>implementations to >>> return different singleton instances. Ok - so this means you now have 9 >>> empty instances rather than 1 but that's basically nothing in terms of >>> memory consumption. In the case of Optional you already have manual >>> specialisation for 3 primitive cases in the form of Optionalint and >>>friends >>> so its really 9 rather than 4 anyway. >>> >>> regards, >>> >>> Richard Warburton >>> >>> http://insightfullogic.com >>> @RichardWarburto >>> >> >> From duncan.macgregor at ge.com Mon Jan 12 13:29:45 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Mon, 12 Jan 2015 13:29:45 +0000 Subject: Static fields and specialization In-Reply-To: <54B3CB9D.5000007@univ-mlv.fr> References: <54B1A520.2020006@oracle.com> <54B3CB9D.5000007@univ-mlv.fr> Message-ID: You won?t need any holder for a _final_ static with a type variable, but non-final statics would definitely require a holder. On 12/01/2015 13:26, "Remi Forax" wrote: > >On 01/12/2015 01:51 PM, Palo Marton wrote: >> One possible solution how to support specialized static fields is to use >> same approach as with generic methods. E.g. with this syntax: >> >> private static final Optional EMPTY = new Optional(); >> >> This will compile initializing expression to generic static method: >> >> private static Optional EMPTY$init() { >> return new Optional(); >> } >> >> And all get/set access to EMPTY will use invocedynamic. Bootsrap will >> call Optional.EMPTY$init() to get initial value, store it in some >>holder >> object on heap (eg Variable) and returm get/set MethodHandle for that >> field. > >You don't need any holder, there is a special MethodHandle >named MethodHandles.constant() which is able to always >return the same value, the JIT will consider the value as a true constant. > >R?mi > >> >> >> >> >> >> On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton >>wrote: >> >>> Yes, but you can not create specialized implementations for user >>>defined >>> value types. So these will be left with much slower implementation. >>> Singleton implementation compiles to just 0-1 instructions and >>> implementation that allocates new instance will be much slower. >>> >>> And other problem - such approach is against goals of specialization: >>>You >>> will have to write separate code for each primitive type. >>> >>> Pavol Marton, aSc >>> www.asctimetables.com >>> >>> >>>> In the case of Optional.empty() and others like >>>>Collections.emptyList() >>>> it's a method that you're invoking. There is no need to implement >>>>static >>>> field specialization. You can just get your specialized >>>>implementations to >>>> return different singleton instances. Ok - so this means you now have >>>>9 >>>> empty instances rather than 1 but that's basically nothing in terms of >>>> memory consumption. In the case of Optional you already have manual >>>> specialisation for 3 primitive cases in the form of Optionalint and >>>>friends >>>> so its really 9 rather than 4 anyway. >>>> >>>> regards, >>>> >>>> Richard Warburton >>>> >>>> http://insightfullogic.com >>>> @RichardWarburto >>>> >>> > From forax at univ-mlv.fr Mon Jan 12 13:40:23 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 12 Jan 2015 14:40:23 +0100 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> <54B3CB9D.5000007@univ-mlv.fr> Message-ID: <54B3CEC7.3030707@univ-mlv.fr> On 01/12/2015 02:29 PM, MacGregor, Duncan (GE Energy Management) wrote: > You won?t need any holder for a _final_ static with a type variable, but > non-final statics would definitely require a holder. On 01/12/2015 02:29 PM, Palo Marton wrote: > > You don't need any holder, there is a special MethodHandle > named MethodHandles.constant() which is able to always > return the same value, the JIT will consider the value as a true > constant. > > > Yes, that can be optimization for final fields. You need holder just > for non-final fields. Am i the only one to live in a world where static non final field are considered as evil ? R?mi > > On 12/01/2015 13:26, "Remi Forax" wrote: > >> On 01/12/2015 01:51 PM, Palo Marton wrote: >>> One possible solution how to support specialized static fields is to use >>> same approach as with generic methods. E.g. with this syntax: >>> >>> private static final Optional EMPTY = new Optional(); >>> >>> This will compile initializing expression to generic static method: >>> >>> private static Optional EMPTY$init() { >>> return new Optional(); >>> } >>> >>> And all get/set access to EMPTY will use invocedynamic. Bootsrap will >>> call Optional.EMPTY$init() to get initial value, store it in some >>> holder >>> object on heap (eg Variable) and returm get/set MethodHandle for that >>> field. >> You don't need any holder, there is a special MethodHandle >> named MethodHandles.constant() which is able to always >> return the same value, the JIT will consider the value as a true constant. >> >> R?mi >> >>> >>> >>> >>> >>> On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton >>> wrote: >>> >>>> Yes, but you can not create specialized implementations for user >>>> defined >>>> value types. So these will be left with much slower implementation. >>>> Singleton implementation compiles to just 0-1 instructions and >>>> implementation that allocates new instance will be much slower. >>>> >>>> And other problem - such approach is against goals of specialization: >>>> You >>>> will have to write separate code for each primitive type. >>>> >>>> Pavol Marton, aSc >>>> www.asctimetables.com >>>> >>>> >>>>> In the case of Optional.empty() and others like >>>>> Collections.emptyList() >>>>> it's a method that you're invoking. There is no need to implement >>>>> static >>>>> field specialization. You can just get your specialized >>>>> implementations to >>>>> return different singleton instances. Ok - so this means you now have >>>>> 9 >>>>> empty instances rather than 1 but that's basically nothing in terms of >>>>> memory consumption. In the case of Optional you already have manual >>>>> specialisation for 3 primitive cases in the form of Optionalint and >>>>> friends >>>>> so its really 9 rather than 4 anyway. >>>>> >>>>> regards, >>>>> >>>>> Richard Warburton >>>>> >>>>> http://insightfullogic.com >>>>> @RichardWarburto >>>>> From palo.marton at gmail.com Mon Jan 12 13:41:53 2015 From: palo.marton at gmail.com (Palo Marton) Date: Mon, 12 Jan 2015 14:41:53 +0100 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> Message-ID: > That sounds fundamentally okay, but probably also requires a typed static > initialiser syntax for fields that could throw an exception during > initialisation. > You can wrap such initialization to generic method by yourself: static final MethodHandle mh=mh_init(); static MethodHandle mh_init() {.....throw...} Supporting " static {... }" will bring many challenges with order of execution for such initializes. > > Something like > > static final MethodHandle mh; > > static { > try { > // Do MethodHandle lookup > } (catch e) { > Throw new Error(); > } > } > > Duncan. > > On 12/01/2015 12:51, "Palo Marton" wrote: > > >One possible solution how to support specialized static fields is to use > >same approach as with generic methods. E.g. with this syntax: > > > >private static final Optional EMPTY = new Optional(); > > > >This will compile initializing expression to generic static method: > > > >private static Optional EMPTY$init() { > > return new Optional(); > >} > > > >And all get/set access to EMPTY will use invocedynamic. Bootsrap will > >call Optional.EMPTY$init() to get initial value, store it in some > >holder > >object on heap (eg Variable) and returm get/set MethodHandle for that > >field. > > > > > > > > > > > >On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton > >wrote: > > > >> Yes, but you can not create specialized implementations for user defined > >> value types. So these will be left with much slower implementation. > >> Singleton implementation compiles to just 0-1 instructions and > >> implementation that allocates new instance will be much slower. > >> > >> And other problem - such approach is against goals of specialization: > >>You > >> will have to write separate code for each primitive type. > >> > >> Pavol Marton, aSc > >> www.asctimetables.com > >> > >> > >>> In the case of Optional.empty() and others like Collections.emptyList() > >>> it's a method that you're invoking. There is no need to implement > >>>static > >>> field specialization. You can just get your specialized > >>>implementations to > >>> return different singleton instances. Ok - so this means you now have 9 > >>> empty instances rather than 1 but that's basically nothing in terms of > >>> memory consumption. In the case of Optional you already have manual > >>> specialisation for 3 primitive cases in the form of Optionalint and > >>>friends > >>> so its really 9 rather than 4 anyway. > >>> > >>> regards, > >>> > >>> Richard Warburton > >>> > >>> http://insightfullogic.com > >>> @RichardWarburto > >>> > >> > >> > > From duncan.macgregor at ge.com Mon Jan 12 13:45:19 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Mon, 12 Jan 2015 13:45:19 +0000 Subject: Static fields and specialization In-Reply-To: <54B3CEC7.3030707@univ-mlv.fr> References: <54B1A520.2020006@oracle.com> <54B3CB9D.5000007@univ-mlv.fr> <54B3CEC7.3030707@univ-mlv.fr> Message-ID: Oh, they?re definitely evil, but they probably still need to be supported. From: R?mi Forax > Date: Monday, 12 January 2015 13:40 To: Duncan MacGregor >, "valhalla-dev at openjdk.java.net" > Subject: Re: Static fields and specialization On 01/12/2015 02:29 PM, MacGregor, Duncan (GE Energy Management) wrote: You won?t need any holder for a _final_ static with a type variable, but non-final statics would definitely require a holder. On 01/12/2015 02:29 PM, Palo Marton wrote: You don't need any holder, there is a special MethodHandle named MethodHandles.constant() which is able to always return the same value, the JIT will consider the value as a true constant. Yes, that can be optimization for final fields. You need holder just for non-final fields. Am i the only one to live in a world where static non final field are considered as evil ? R?mi On 12/01/2015 13:26, "Remi Forax" wrote: On 01/12/2015 01:51 PM, Palo Marton wrote: One possible solution how to support specialized static fields is to use same approach as with generic methods. E.g. with this syntax: private static final Optional EMPTY = new Optional(); This will compile initializing expression to generic static method: private static Optional EMPTY$init() { return new Optional(); } And all get/set access to EMPTY will use invocedynamic. Bootsrap will call Optional.EMPTY$init() to get initial value, store it in some holder object on heap (eg Variable) and returm get/set MethodHandle for that field. You don't need any holder, there is a special MethodHandle named MethodHandles.constant() which is able to always return the same value, the JIT will consider the value as a true constant. R?mi On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton wrote: Yes, but you can not create specialized implementations for user defined value types. So these will be left with much slower implementation. Singleton implementation compiles to just 0-1 instructions and implementation that allocates new instance will be much slower. And other problem - such approach is against goals of specialization: You will have to write separate code for each primitive type. Pavol Marton, aSc www.asctimetables.com In the case of Optional.empty() and others like Collections.emptyList() it's a method that you're invoking. There is no need to implement static field specialization. You can just get your specialized implementations to return different singleton instances. Ok - so this means you now have 9 empty instances rather than 1 but that's basically nothing in terms of memory consumption. In the case of Optional you already have manual specialisation for 3 primitive cases in the form of Optionalint and friends so its really 9 rather than 4 anyway. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From duncan.macgregor at ge.com Mon Jan 12 13:52:49 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Mon, 12 Jan 2015 13:52:49 +0000 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> Message-ID: Good point, using a function to fetch the value should be fine. From: Palo Marton > Date: Monday, 12 January 2015 13:41 To: Duncan MacGregor > Cc: Richard Warburton >, valhalla-dev > Subject: Re: Static fields and specialization That sounds fundamentally okay, but probably also requires a typed static initialiser syntax for fields that could throw an exception during initialisation. You can wrap such initialization to generic method by yourself: static final MethodHandle mh=mh_init(); static MethodHandle mh_init() {.....throw...} Supporting " static {... }" will bring many challenges with order of execution for such initializes. Something like static final MethodHandle mh; static { try { // Do MethodHandle lookup } (catch e) { Throw new Error(); } } Duncan. On 12/01/2015 12:51, "Palo Marton" > wrote: >One possible solution how to support specialized static fields is to use >same approach as with generic methods. E.g. with this syntax: > >private static final Optional EMPTY = new Optional(); > >This will compile initializing expression to generic static method: > >private static Optional EMPTY$init() { > return new Optional(); >} > >And all get/set access to EMPTY will use invocedynamic. Bootsrap will >call Optional.EMPTY$init() to get initial value, store it in some >holder >object on heap (eg Variable) and returm get/set MethodHandle for that >field. > > > > > >On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton > >wrote: > >> Yes, but you can not create specialized implementations for user defined >> value types. So these will be left with much slower implementation. >> Singleton implementation compiles to just 0-1 instructions and >> implementation that allocates new instance will be much slower. >> >> And other problem - such approach is against goals of specialization: >>You >> will have to write separate code for each primitive type. >> >> Pavol Marton, aSc >> www.asctimetables.com >> >> >>> In the case of Optional.empty() and others like Collections.emptyList() >>> it's a method that you're invoking. There is no need to implement >>>static >>> field specialization. You can just get your specialized >>>implementations to >>> return different singleton instances. Ok - so this means you now have 9 >>> empty instances rather than 1 but that's basically nothing in terms of >>> memory consumption. In the case of Optional you already have manual >>> specialisation for 3 primitive cases in the form of Optionalint and >>>friends >>> so its really 9 rather than 4 anyway. >>> >>> regards, >>> >>> Richard Warburton >>> >>> http://insightfullogic.com >>> @RichardWarburto >>> >> >> From timo.kinnunen at gmail.com Mon Jan 12 13:46:40 2015 From: timo.kinnunen at gmail.com (=?utf-8?Q?Timo_Kinnunen?=) Date: Mon, 12 Jan 2015 13:46:40 +0000 Subject: =?utf-8?Q?Re:_Static_fields_and_specialization?= In-Reply-To: <54B3CEC7.3030707@univ-mlv.fr> References: <54B1A520.2020006@oracle.com> <54B3CB9D.5000007@univ-mlv.fr> , <54B3CEC7.3030707@univ-mlv.fr> Message-ID: <54b3d1e6.2d4cb40a.56e7.6a1d@mx.google.com> They?re fine as long as you never change them from multiple threads at the same time (and why would you do that..?) -- Have a nice day, Timo. Sent from Windows Mail From: Remi Forax Sent: ?Monday?, ?January? ?12?, ?2015 ?14?:?40 To: MacGregor, Duncan (GE Energy Management), valhalla-dev at openjdk.java.net On 01/12/2015 02:29 PM, MacGregor, Duncan (GE Energy Management) wrote: > You won?t need any holder for a _final_ static with a type variable, but > non-final statics would definitely require a holder. On 01/12/2015 02:29 PM, Palo Marton wrote: > > You don't need any holder, there is a special MethodHandle > named MethodHandles.constant() which is able to always > return the same value, the JIT will consider the value as a true > constant. > > > Yes, that can be optimization for final fields. You need holder just > for non-final fields. Am i the only one to live in a world where static non final field are considered as evil ? R?mi > > On 12/01/2015 13:26, "Remi Forax" wrote: > >> On 01/12/2015 01:51 PM, Palo Marton wrote: >>> One possible solution how to support specialized static fields is to use >>> same approach as with generic methods. E.g. with this syntax: >>> >>> private static final Optional EMPTY = new Optional(); >>> >>> This will compile initializing expression to generic static method: >>> >>> private static Optional EMPTY$init() { >>> return new Optional(); >>> } >>> >>> And all get/set access to EMPTY will use invocedynamic. Bootsrap will >>> call Optional.EMPTY$init() to get initial value, store it in some >>> holder >>> object on heap (eg Variable) and returm get/set MethodHandle for that >>> field. >> You don't need any holder, there is a special MethodHandle >> named MethodHandles.constant() which is able to always >> return the same value, the JIT will consider the value as a true constant. >> >> R?mi >> >>> >>> >>> >>> >>> On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton >>> wrote: >>> >>>> Yes, but you can not create specialized implementations for user >>>> defined >>>> value types. So these will be left with much slower implementation. >>>> Singleton implementation compiles to just 0-1 instructions and >>>> implementation that allocates new instance will be much slower. >>>> >>>> And other problem - such approach is against goals of specialization: >>>> You >>>> will have to write separate code for each primitive type. >>>> >>>> Pavol Marton, aSc >>>> www.asctimetables.com >>>> >>>> >>>>> In the case of Optional.empty() and others like >>>>> Collections.emptyList() >>>>> it's a method that you're invoking. There is no need to implement >>>>> static >>>>> field specialization. You can just get your specialized >>>>> implementations to >>>>> return different singleton instances. Ok - so this means you now have >>>>> 9 >>>>> empty instances rather than 1 but that's basically nothing in terms of >>>>> memory consumption. In the case of Optional you already have manual >>>>> specialisation for 3 primitive cases in the form of Optionalint and >>>>> friends >>>>> so its really 9 rather than 4 anyway. >>>>> >>>>> regards, >>>>> >>>>> Richard Warburton >>>>> >>>>> http://insightfullogic.com >>>>> @RichardWarburto >>>>> From palo.marton at gmail.com Mon Jan 12 13:54:32 2015 From: palo.marton at gmail.com (Palo Marton) Date: Mon, 12 Jan 2015 14:54:32 +0100 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> <54B3CB9D.5000007@univ-mlv.fr> <54B3CEC7.3030707@univ-mlv.fr> Message-ID: > Oh, they?re definitely evil, but they probably still need to be supported. > You folks probably live in highly multithreaded world ;-) But there still exists some single threaded apps or people who don't care much about issues surrounding multitrheaded access. You can find some example of mutable static field here: class Loader { private static int thingsLoaded; T load(String name) { ++thingsLoaded; // load and return the thing } int getLoadCount() { return thingsLoaded; } } (from http://cr.openjdk.java.net/~briangoetz/valhalla/specialization.html). > From: R?mi Forax > > Date: Monday, 12 January 2015 13:40 > To: Duncan MacGregor duncan.macgregor at ge.com>>, "valhalla-dev at openjdk.java.net valhalla-dev at openjdk.java.net>" valhalla-dev at openjdk.java.net>> > Subject: Re: Static fields and specialization > > > On 01/12/2015 02:29 PM, MacGregor, Duncan (GE Energy Management) wrote: > > You won?t need any holder for a _final_ static with a type variable, but > non-final statics would definitely require a holder. > > On 01/12/2015 02:29 PM, Palo Marton wrote: > > You don't need any holder, there is a special MethodHandle > named MethodHandles.constant() which is able to always > return the same value, the JIT will consider the value as a true constant. > > Yes, that can be optimization for final fields. You need holder just for > non-final fields. > > > Am i the only one to live in a world where static non final field are > considered as evil ? > > R?mi > > > On 12/01/2015 13:26, "Remi Forax" forax at univ-mlv.fr> wrote: > > > > On 01/12/2015 01:51 PM, Palo Marton wrote: > > > One possible solution how to support specialized static fields is to use > same approach as with generic methods. E.g. with this syntax: > > private static final Optional EMPTY = new Optional(); > > This will compile initializing expression to generic static method: > > private static Optional EMPTY$init() { > return new Optional(); > } > > And all get/set access to EMPTY will use invocedynamic. Bootsrap will > call Optional.EMPTY$init() to get initial value, store it in some > holder > object on heap (eg Variable) and returm get/set MethodHandle for that > field. > > > You don't need any holder, there is a special MethodHandle > named MethodHandles.constant() which is able to always > return the same value, the JIT will consider the value as a true constant. > > R?mi > > > > > > > On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton > > wrote: > > > > Yes, but you can not create specialized implementations for user > defined > value types. So these will be left with much slower implementation. > Singleton implementation compiles to just 0-1 instructions and > implementation that allocates new instance will be much slower. > > And other problem - such approach is against goals of specialization: > You > will have to write separate code for each primitive type. > > Pavol Marton, aSc > www.asctimetables.com > > In the case of Optional.empty() and others like > Collections.emptyList() > it's a method that you're invoking. There is no need to implement > static > field specialization. You can just get your specialized > implementations to > return different singleton instances. Ok - so this means you now have > 9 > empty instances rather than 1 but that's basically nothing in terms of > memory consumption. In the case of Optional you already have manual > specialisation for 3 primitive cases in the form of Optionalint and > friends > so its really 9 rather than 4 anyway. > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto < > http://twitter.com/richardwarburto> > > From forax at univ-mlv.fr Mon Jan 12 13:58:05 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 12 Jan 2015 14:58:05 +0100 Subject: Static fields and specialization In-Reply-To: <54b3d1e6.2d4cb40a.56e7.6a1d@mx.google.com> References: <54B1A520.2020006@oracle.com> <54B3CB9D.5000007@univ-mlv.fr> , <54B3CEC7.3030707@univ-mlv.fr> <54b3d1e6.2d4cb40a.56e7.6a1d@mx.google.com> Message-ID: <54B3D2ED.7020506@univ-mlv.fr> On 01/12/2015 02:46 PM, Timo Kinnunen wrote: > They?re fine as long as you never change them from multiple threads at > the same time (and why would you do that..?) and you can not test any code that depend on it easily and if the type is an object you keep an objects graph forever in memory. pure evil as i said. R?mi > > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > *From:* Remi Forax > *Sent:* ?Monday?, ?January? ?12?, ?2015 ?14?:?40 > *To:* MacGregor, Duncan (GE Energy Management) > , valhalla-dev at openjdk.java.net > > > > On 01/12/2015 02:29 PM, MacGregor, Duncan (GE Energy Management) wrote: > > You won?t need any holder for a _final_ static with a type variable, but > > non-final statics would definitely require a holder. > > On 01/12/2015 02:29 PM, Palo Marton wrote: > > > > You don't need any holder, there is a special MethodHandle > > named MethodHandles.constant() which is able to always > > return the same value, the JIT will consider the value as a true > > constant. > > > > > > Yes, that can be optimization for final fields. You need holder just > > for non-final fields. > > Am i the only one to live in a world where static non final field are > considered as evil ? > > R?mi > > > > > On 12/01/2015 13:26, "Remi Forax" wrote: > > > >> On 01/12/2015 01:51 PM, Palo Marton wrote: > >>> One possible solution how to support specialized static fields is > to use > >>> same approach as with generic methods. E.g. with this syntax: > >>> > >>> private static final Optional EMPTY = new Optional(); > >>> > >>> This will compile initializing expression to generic static method: > >>> > >>> private static Optional EMPTY$init() { > >>> return new Optional(); > >>> } > >>> > >>> And all get/set access to EMPTY will use invocedynamic. > Bootsrap will > >>> call Optional.EMPTY$init() to get initial value, store it in some > >>> holder > >>> object on heap (eg Variable) and returm get/set MethodHandle > for that > >>> field. > >> You don't need any holder, there is a special MethodHandle > >> named MethodHandles.constant() which is able to always > >> return the same value, the JIT will consider the value as a true > constant. > >> > >> R?mi > >> > >>> > >>> > >>> > >>> > >>> On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton > >>> wrote: > >>> > >>>> Yes, but you can not create specialized implementations for user > >>>> defined > >>>> value types. So these will be left with much slower implementation. > >>>> Singleton implementation compiles to just 0-1 instructions and > >>>> implementation that allocates new instance will be much slower. > >>>> > >>>> And other problem - such approach is against goals of specialization: > >>>> You > >>>> will have to write separate code for each primitive type. > >>>> > >>>> Pavol Marton, aSc > >>>> www.asctimetables.com > >>>> > >>>> > >>>>> In the case of Optional.empty() and others like > >>>>> Collections.emptyList() > >>>>> it's a method that you're invoking. There is no need to implement > >>>>> static > >>>>> field specialization. You can just get your specialized > >>>>> implementations to > >>>>> return different singleton instances. Ok - so this means you now > have > >>>>> 9 > >>>>> empty instances rather than 1 but that's basically nothing in > terms of > >>>>> memory consumption. In the case of Optional you already have manual > >>>>> specialisation for 3 primitive cases in the form of Optionalint and > >>>>> friends > >>>>> so its really 9 rather than 4 anyway. > >>>>> > >>>>> regards, > >>>>> > >>>>> Richard Warburton > >>>>> > >>>>> http://insightfullogic.com > >>>>> @RichardWarburto > >>>>> > From duncan.macgregor at ge.com Mon Jan 12 14:00:41 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Mon, 12 Jan 2015 14:00:41 +0000 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> <54B3CB9D.5000007@univ-mlv.fr> <54B3CEC7.3030707@univ-mlv.fr> Message-ID: I?d tend to make that a final atomic integer, but I can easily imagine it being does as a volatile int and some var handle stuff to handle the increment. Though I have used a plain int when I know thread safety won?t be an issue. From: Palo Marton > Date: Monday, 12 January 2015 13:54 To: Duncan MacGregor > Cc: R?mi Forax >, "valhalla-dev at openjdk.java.net" > Subject: Re: Static fields and specialization Oh, they?re definitely evil, but they probably still need to be supported. You folks probably live in highly multithreaded world ;-) But there still exists some single threaded apps or people who don't care much about issues surrounding multitrheaded access. You can find some example of mutable static field here: class Loader { private static int thingsLoaded; T load(String name) { ++thingsLoaded; // load and return the thing } int getLoadCount() { return thingsLoaded; } } (from http://cr.openjdk.java.net/~briangoetz/valhalla/specialization.html). From: R?mi Forax >> Date: Monday, 12 January 2015 13:40 To: Duncan MacGregor >>, "valhalla-dev at openjdk.java.net>" >> Subject: Re: Static fields and specialization On 01/12/2015 02:29 PM, MacGregor, Duncan (GE Energy Management) wrote: You won?t need any holder for a _final_ static with a type variable, but non-final statics would definitely require a holder. On 01/12/2015 02:29 PM, Palo Marton wrote: You don't need any holder, there is a special MethodHandle named MethodHandles.constant() which is able to always return the same value, the JIT will consider the value as a true constant. Yes, that can be optimization for final fields. You need holder just for non-final fields. Am i the only one to live in a world where static non final field are considered as evil ? R?mi On 12/01/2015 13:26, "Remi Forax" >> wrote: On 01/12/2015 01:51 PM, Palo Marton wrote: One possible solution how to support specialized static fields is to use same approach as with generic methods. E.g. with this syntax: private static final Optional EMPTY = new Optional(); This will compile initializing expression to generic static method: private static Optional EMPTY$init() { return new Optional(); } And all get/set access to EMPTY will use invocedynamic. Bootsrap will call Optional.EMPTY$init() to get initial value, store it in some holder object on heap (eg Variable) and returm get/set MethodHandle for that field. You don't need any holder, there is a special MethodHandle named MethodHandles.constant() which is able to always return the same value, the JIT will consider the value as a true constant. R?mi On Mon, Jan 12, 2015 at 12:44 PM, Palo Marton >> wrote: Yes, but you can not create specialized implementations for user defined value types. So these will be left with much slower implementation. Singleton implementation compiles to just 0-1 instructions and implementation that allocates new instance will be much slower. And other problem - such approach is against goals of specialization: You will have to write separate code for each primitive type. Pavol Marton, aSc www.asctimetables.com In the case of Optional.empty() and others like Collections.emptyList() it's a method that you're invoking. There is no need to implement static field specialization. You can just get your specialized implementations to return different singleton instances. Ok - so this means you now have 9 empty instances rather than 1 but that's basically nothing in terms of memory consumption. In the case of Optional you already have manual specialisation for 3 primitive cases in the form of Optionalint and friends so its really 9 rather than 4 anyway. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From brian.goetz at oracle.com Mon Jan 12 15:32:40 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 12 Jan 2015 10:32:40 -0500 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> Message-ID: <54B3E918.60408@oracle.com> > Yes, but you can not create specialized implementations for user defined > value types. Where do you get this idea, that value types and specialization don't play together? Also, note that allocation for value types doesn't allocate. So your mental performance model about what "new Optional()" costs is probably just wrong. From duncan.macgregor at ge.com Mon Jan 12 15:41:35 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Mon, 12 Jan 2015 15:41:35 +0000 Subject: Static fields and specialization In-Reply-To: <54B3E918.60408@oracle.com> References: <54B1A520.2020006@oracle.com> <54B3E918.60408@oracle.com> Message-ID: I don?t think anybody is suggesting that they don?t play together, but while many problems can be resolved over the small set of primitive types by explicitly create separate static fields for each type if need be, this technique does not scale once arbitrary value types have been introduced. At that point we can?t simply write static final MethodHandle mhObj =... Static final MethodHandle mhInt =... And so on, and rely on layering to let us specify specialised methods that use these, but have to start writing something more like Final static MethodHandle mh =... Duncan. On 12/01/2015 15:32, "Brian Goetz" wrote: >> Yes, but you can not create specialized implementations for user defined >> value types. > >Where do you get this idea, that value types and specialization don't >play together? > >Also, note that allocation for value types doesn't allocate. So your >mental performance model about what "new Optional()" costs is >probably just wrong. > From palo.marton at gmail.com Mon Jan 12 15:43:54 2015 From: palo.marton at gmail.com (Palo Marton) Date: Mon, 12 Jan 2015 16:43:54 +0100 Subject: Static fields and specialization In-Reply-To: <54B3E918.60408@oracle.com> References: <54B1A520.2020006@oracle.com> <54B3E918.60408@oracle.com> Message-ID: > Where do you get this idea, that value types and specialization don't play > together? > You probably misunderstood me (again) :( Richard suggested that you can keep speed of Optional.empty() by creating separate layers for , , etc... by it is not possible to do that for every possible T, e.g. this Optional.empty(); can not be implemented as super-fast-singleton-return without supporting some form of static field specialization (at least I think so). PS: See also some suggestion later in this thread how such specialization can be implemented. From maurizio.cimadamore at oracle.com Mon Jan 12 15:56:08 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 12 Jan 2015 15:56:08 +0000 Subject: hg: valhalla/valhalla/langtools: Add experimental/hacky peeling support: Message-ID: <201501121556.t0CFu8se027062@aojmv0008> Changeset: 7513a72966b6 Author: mcimadamore Date: 2015-01-12 15:55 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/7513a72966b6 Add experimental/hacky peeling support: * peeling is supported by adding special __WhereRef/Val blocks inside a method (no support for true layers) * change parser to support peeled methods * add support to type-check where clauses in blocks * peeled methos are desigared away into separate methods in SpecializeTypes * tweak Where bytecode attribute; a flag says as to whether the method is peeled vs. restricted * meet rule for where blocks * added tests ! src/jdk.compiler/share/classes/com/sun/tools/classfile/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/classfile/Where_attribute.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Flags.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/WhereClause.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Flow.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/MemberEnter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassReader.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties ! src/jdk.compiler/share/classes/com/sun/tools/javac/tree/JCTree.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeInfo.java ! src/jdk.compiler/share/classes/com/sun/tools/javap/AttributeWriter.java + test/tools/javac/diags/examples/PeeledMethAmbiguousBlocks.java + test/tools/javac/diags/examples/PeeledMethMissingBlock.java + test/tools/javac/valhalla/typespec/Auxiliary05.java + test/tools/javac/valhalla/typespec/Peel01.java + test/tools/javac/valhalla/typespec/Peel01.out + test/tools/javac/valhalla/typespec/Peel02.java + test/tools/javac/valhalla/typespec/Peel02.out + test/tools/javac/valhalla/typespec/Peel03.java + test/tools/javac/valhalla/typespec/Peel03.out ! test/tools/javac/valhalla/typespec/items/BytecodeMappingHarness.java + test/tools/javac/valhalla/typespec/items/TemplateMethod.java ! test/tools/javac/valhalla/typespec/items/tests/TestValOnly.java + test/tools/javac/valhalla/typespec/separate04/A.java + test/tools/javac/valhalla/typespec/separate04/TestSeparate04.java From stef at epardaud.fr Mon Jan 12 15:57:10 2015 From: stef at epardaud.fr (Stef Epardaud) Date: Mon, 12 Jan 2015 16:57:10 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54B0280A.3020707@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> Message-ID: <20150112155709.GA15204@inforealm.org> On Fri, Jan 09, 2015 at 02:12:10PM -0500, Brian Goetz wrote: > > The hard part is: the entire VM field-access model is built around > translating field accesses like > > getfield Foo.x [ receiver ] > > to native code that looks like: > > (ideally elided) verify receiver is a Foo > result = *(receiver + sizeof(HEADER) + offsetof(x)) > > based on the assumption that all instances of Foo have Foo's fields > at the same offsets. (For methods we achieve polymorphism with > vtables; for fields, we achieve it by adding subtype fields to the > end of base type fields.) > > But, if I have a class Box with a single T-valued field, I > need different layouts; one for erased refs, one for ints, a longer > one for longs, etc. (And of course this perturbs the offset of > other fields.) > > Code that is generic in T that wants to access Box.t now has to > deal with different layouts; this is a pretty big change to our > compilation story (with consequences for performance.) Is this really true? I thought that because there's no instantiation of the any type, every any-generic method has to be specialised anyways, so there isn't really generic code that abstracts over both Object-generic and value-generic. To me the compromise that we can't write code that iterates say, a list, without knowing at compile-time what type of value-generics it may hold is a compromise that will haunt Java forever. I understand how you came to that compromise, and I understand that you can push the isue to the caller by making the iteration code Any-generic, but that only works (like in C++) for compile-time instantiations: if you write a framework that needs to traverse lists of any value type, including any sort of value type, then you can't. Unless I got this wrong and then I apologise, but please explain how ;) This goes back to the fact that the division between List and List is a terrible compromise. It will be even worse when you have a List<@ValueType Date> and you want to iterate it like it's a List, which it isn't. Especially confusing since there is autoboxing for both Java primitives and value types. The idea of specialisation is good, clearly, and it doesn't work for fields, OK. But why not say that List<@ValueType Date> is also a List (boxed values)? Not as subtyping just as type equivalence. I know you can't implement the same interface twice with difference type arguments in Java but AFAICT the VM doesn't actually care that much (I say this with experimental evidence). You could get an instance of the List class that has bridge methods for all the boxed methods that would get called when using the List or List type (boxed calls) which would forward to the specialised methods for List<@ValueType Date> (unboxed calls). And call-sites that know we're dealing with a List<@ValueType Date> would call the specialised methods directly and not box. This clearly doesn't work with fields, but so what? Add a language limitation that fields of Any-generic types must be private. Fields are not virtual so this would work with specialised types. IMO this restriction would be acceptable for collections and much less of a compromise than the split between Object-generics and Value-generics. OK this may not be retrofitable to JVM primitives and List/List depending on how possible it is to substitute value types for JVM boxed primitives (in particular I know not being able to lock boxed value types may exclude JVM boxed primitives from that evolution), but at least it will be possible to make it work for other value types. Now, perhaps I'm out of my depth here, I'm not a VM guy, even though I understand about compilation, bytecode, memory and assembly, but I'm clearly not as much an expert as you guys, but still I'd like to understand why my proposal would not work. Thanks! -- St?phane Epardaud From maurizio.cimadamore at oracle.com Mon Jan 12 15:57:43 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 12 Jan 2015 15:57:43 +0000 Subject: hg: valhalla/valhalla/jdk: Add peeling support:: Message-ID: <201501121557.t0CFvhSe027292@aojmv0008> Changeset: 475417bdc9a0 Author: mcimadamore Date: 2015-01-12 15:57 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/475417bdc9a0 Add peeling support:: * add support for de-mangling mangled template method names * added smoke test ! src/java.base/share/classes/valhalla/specializer/Specializer.java ! src/java.base/share/classes/valhalla/specializer/WhereAttribute.java + test/valhalla/test/valhalla/specializer/PeelingTest.java From brian.goetz at oracle.com Mon Jan 12 16:05:06 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 12 Jan 2015 11:05:06 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <20150112155709.GA15204@inforealm.org> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> Message-ID: <54B3F0B2.3000208@oracle.com> >> Code that is generic in T that wants to access Box.t now has to >> deal with different layouts; this is a pretty big change to our >> compilation story (with consequences for performance.) > > Is this really true? I thought that because there's no instantiation of the > any type, every any-generic method has to be specialised anyways, so there > isn't really generic code that abstracts over both Object-generic and > value-generic. Part of what you may be missing is: code that was compiled with older compilers. While we have the opportunity to change the translation of newly compiled code, there will be existing code out there that we also need to be compatible with. > To me the compromise that we can't write code that iterates say, a > list, without knowing at compile-time what type of value-generics it may > hold is a compromise that will haunt Java forever. This is just incorrect. You can write a generic method: forEach(Collection c, Consumer action) > This clearly doesn't work with fields, but so what? Add a language limitation that > fields of Any-generic types must be private. Which means that any class that had public fields can't ever be anyfied. Also, as people become more aware of the evils of mutability, more people are (correctly) writing classes with public final fields rather than intermediating with unnecessary getters. (Hint: when you're inclined to say "but so what", that's the warning sign that you're spiraling off into wishful-thinking territory.) From brian.goetz at oracle.com Mon Jan 12 16:11:40 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 12 Jan 2015 11:11:40 -0500 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> <54B3E918.60408@oracle.com> Message-ID: <54B3F23C.7030606@oracle.com> > You probably misunderstood me (again) :( > > Richard suggested that you can keep speed of Optional.empty() by > creating separate layers for , , etc... Remember, for value types (hopefully Optional will be one), instantiation is essentially free, and in most cases where this idiom is used, it is the allocation that we're trying to optimize away. So the idiom of "cache one and return the same instance", in these cases, can be retired. I realize this still doesn't help us for reference types, like Collections.emptyList(); this point got made a while ago (so it didn't occur to me we were still talking about it.) From maurizio.cimadamore at oracle.com Mon Jan 12 16:17:38 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 12 Jan 2015 16:17:38 +0000 Subject: caution: bleeding-edge ahead! Message-ID: <54B3F3A2.8050403@oracle.com> Dear honorable warriors, I've just pushed a compiler/specializer patch that should add some basic support for peeled methods. This is totally experimental (as I mentioned to Peter last week), and the support is based on the clunky __WhereRef/Val keywords - i.e. it's not an implementation of the layering proposal as described in Brian's document, nor is it a serious proposal of how the syntax would work ... it's just a hack to move the experiments forward. Here's an example on how you might want to peel a method: class Test { void peeledMethod() { __WhereRef(T) { //ref implemnetation goes here } __WhereVal(T) { //val implementation goes here } } } The compiler will type-check the two bodies as if they were the 'only' body - meaning flow analysis should work as expected (i.e. you can have two return statements, one in each body, and the compiler will be happy with that). At specialization time, the whole method will be splitted in two parts - a real method (for the reference impl) and a 'template method' - the class will look like: class Test { void peeledMethod() { //ref implemnetation goes here } void template$peeledMethod() { //val implementation goes here } } The two methods will contain special Where bytecode attributes to let the specializer reconstruct the specialized classes with the right methods in them. One obvious limitation (I'm sure there are many other I haven't thought about) of this implemented, hacky route is that you cannot have a method that is part concrete, part abstract; but I think you can emulate that with the following idiom: class Test { void partiallyAbstractMethod() { __WhereRef(T) { throw new AbstractMethodError(); } __WhereVal(T) { //val implementation goes here } } } One last note: the compiler will try to detect situations where multiple where blocks might apply on a given specialized class: class Test { void ambiguiousPeeling() { __WhereVal(Y) { //impl1 } __WhereRef(X) { //impl2 } } } This is flagged as 'ambiguous' - think of what happens if you have i.e. a Test - both blocks will match - meaning that the specializer won't know which to pick. There's also a dual check that prevents a peeling method from having 'missing' blocks - i.e. a combination of ref/val instantiation which is not matched by any block in the peeled method. Good luck with the experiments - and please refrain from commenting on the specifics of the implemented syntax - we all know it's very, very ugly :-) :-) :-) Cheers Maurizio From stef at epardaud.fr Mon Jan 12 16:30:27 2015 From: stef at epardaud.fr (Stephane Epardaud) Date: Mon, 12 Jan 2015 17:30:27 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54B3F0B2.3000208@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> Message-ID: <54B3F6A3.8050500@epardaud.fr> On 01/12/2015 05:05 PM, Brian Goetz wrote: > Part of what you may be missing is: code that was compiled with older > compilers. While we have the opportunity to change the translation of > newly compiled code, there will be existing code out there that we > also need to be compatible with. Well, these would call the boxed methods, that would be _more_ compatible since they can traverse collections of value types by boxing them. > >> To me the compromise that we can't write code that iterates say, a >> list, without knowing at compile-time what type of value-generics it may >> hold is a compromise that will haunt Java forever. > > This is just incorrect. You can write a generic method: > > forEach(Collection c, Consumer action) But there's no instantiation of "any T" that would let me not know the generic type at compile-time, no? Frameworks that need to traverse collections without having to know what type of thing they're traversing can't do that with value types ATM. Obviously with my proposal they'd box, but at least they could traverse any collection. > >> This clearly doesn't work with fields, but so what? Add a language >> limitation that >> fields of Any-generic types must be private. > > Which means that any class that had public fields can't ever be > anyfied. Also, as people become more aware of the evils of > mutability, more people are (correctly) writing classes with public > final fields rather than intermediating with unnecessary getters. Well, do we have numbers on how many classes expose their state like this via public fields? Does the Java collection API do that? Perhaps you're right and it's unacceptable, but it's not really clear. I suppose that one way to deal with this if this was an unacceptable limitation would be to generate boxing specialised methods that could provide boxed values for virtual fields backed by actual value fields. > (Hint: when you're inclined to say "but so what", that's the warning > sign that you're spiraling off into wishful-thinking territory.) It's not about "so what" it's about the cost of the compromise. I happen to find that tradeoff to be much much more acceptable than treating List and List<@ValueType Date> disjoined. I do appreciate that it's a matter of personal opinion and I may be in the minority, but I've no evidence of that ;) From ali.ebrahimi1781 at gmail.com Mon Jan 12 23:32:17 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Tue, 13 Jan 2015 03:02:17 +0330 Subject: caution: bleeding-edge ahead! In-Reply-To: <54B3F3A2.8050403@oracle.com> References: <54B3F3A2.8050403@oracle.com> Message-ID: Thanks Maurizio for your hard work. Just one concern for your mentioned limitation (maybe for Now) when having multiple any type vars. One note I want mention here is that I see where clauses as conditional block better and more flexible answer than layers for problems trying answer. Why we can not adapt C++'s #if, #elif, #else, and #endif Directives for java with java-like syntax. You can see that in hotspot code the similar problems (OS depends-code) perfectly to be solved by Directives. So we can have support for multiple any type vars and nested layers, and compiler can do flow analysis for nested layers (where clauses or what ever you want). class Foo{ private A a, private B b, // shared for val and refs // #if(A is val){ //A is val B[] getBArray(){ return new B[]{}; } #if(B is ref){//A is val and B is ref }#else{//A is val and B is val #if(A is int, B is int){ //A is int and B is int int maxAB(){ return a > b? a: b; } } } }#else { //A is ref void some(){ #if(B is ref){ //A is refand B is ref }#else{//A is ref and B is val }// #endif } } // shared for val and refs // } Consider that Foo f = new F(); , f.maxAB() does not compile :Error undeclared maxAB member in Foo On Mon, Jan 12, 2015 at 7:47 PM, Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > Dear honorable warriors, > I've just pushed a compiler/specializer patch that should add some basic > support for peeled methods. This is totally experimental (as I mentioned to > Peter last week), and the support is based on the clunky __WhereRef/Val > keywords - i.e. it's not an implementation of the layering proposal as > described in Brian's document, nor is it a serious proposal of how the > syntax would work ... it's just a hack to move the experiments forward. > Here's an example on how you might want to peel a method: > > class Test { > void peeledMethod() { > __WhereRef(T) { > //ref implemnetation goes here > } > __WhereVal(T) { > //val implementation goes here > } > } > } > > The compiler will type-check the two bodies as if they were the 'only' > body - meaning flow analysis should work as expected (i.e. you can have two > return statements, one in each body, and the compiler will be happy with > that). At specialization time, the whole method will be splitted in two > parts - a real method (for the reference impl) and a 'template method' - > the class will look like: > > class Test { > void peeledMethod() { > //ref implemnetation goes here > } > void template$peeledMethod() { > //val implementation goes here > } > } > > The two methods will contain special Where bytecode attributes to let the > specializer reconstruct the specialized classes with the right methods in > them. > > One obvious limitation (I'm sure there are many other I haven't thought > about) of this implemented, hacky route is that you cannot have a method > that is part concrete, part abstract; but I think you can emulate that with > the following idiom: > > class Test { > void partiallyAbstractMethod() { > __WhereRef(T) { > throw new AbstractMethodError(); > } > __WhereVal(T) { > //val implementation goes here > } > } > } > > One last note: the compiler will try to detect situations where multiple > where blocks might apply on a given specialized class: > > class Test { > void ambiguiousPeeling() { > __WhereVal(Y) { > //impl1 > } > __WhereRef(X) { > //impl2 > } > } > } > > This is flagged as 'ambiguous' - think of what happens if you have i.e. a > Test - both blocks will match - meaning that the specializer > won't know which to pick. There's also a dual check that prevents a peeling > method from having 'missing' blocks - i.e. a combination of ref/val > instantiation which is not matched by any block in the peeled method. > > Good luck with the experiments - and please refrain from commenting on the > specifics of the implemented syntax - we all know it's very, very ugly :-) > :-) :-) > > Cheers > Maurizio > -- Best Regards, Ali Ebrahimi From maurizio.cimadamore at oracle.com Tue Jan 13 00:21:01 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 13 Jan 2015 00:21:01 +0000 Subject: caution: bleeding-edge ahead! In-Reply-To: References: <54B3F3A2.8050403@oracle.com> Message-ID: <54B464ED.7080007@oracle.com> On 12/01/15 23:32, Ali Ebrahimi wrote: > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives > for java with java-like syntax. You can see that in hotspot code the > similar problems (OS depends-code) perfectly to be solved by Directives. > > So we can have support for multiple any type vars and nested layers, > and compiler can do flow analysis for nested layers (where clauses or > what ever you want). Hi Ali, I agree that, to some extent, the end goal of layers is to effectively enable some form of 'optional' membership/overriding which could also be thought of in terms of classic C++-style macros around method declarations/blocks etc. In fact, I think that, apart from partial abstractness, what's implemented right now is more or less functionally equivalent to layers (modulo bugs, of course). That said, I think we are in the search of something that would sit better with the Java programming model; granted, #ifdefs and friends will take you there, but I think it will also be overly powerful - and prone to be abused (and perhaps, as some of you have noted in this mailing list, layers has that problem); what if there was a nice little construct that, with minimal footprint could take you all the way there - meaning that you could retrofit the libraries you care about, w/o really adding a new powerful hammer to the language? I think that 'something' is the sort of magic we are after here. Maurizio From timo.kinnunen at gmail.com Tue Jan 13 06:46:49 2015 From: timo.kinnunen at gmail.com (=?utf-8?Q?Timo_Kinnunen?=) Date: Tue, 13 Jan 2015 06:46:49 +0000 Subject: =?utf-8?Q?Re:_caution:_bleeding-edge_ahead!?= In-Reply-To: <54B464ED.7080007@oracle.com> References: <54B3F3A2.8050403@oracle.com> , <54B464ED.7080007@oracle.com> Message-ID: <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> How about a new keyword ?__aliasOf?, which would be used in a class declaration like this: public class IntArrayList __aliasOf ArrayList implements List { // int layer is implemented here } The effect of __aliasOf would be that user code could call new ArrayList() but this would actually execute like a call to new IntArrayList() and user code would then be compiled against the methods and fields from IntArrayList. (And the name of the keyword is subject to change, of course.) -- Have a nice day, Timo. Sent from Windows Mail From: Maurizio Cimadamore Sent: ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 To: Ali Ebrahimi, Brian Goetz Cc: valhalla-dev at openjdk.java.net On 12/01/15 23:32, Ali Ebrahimi wrote: > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives > for java with java-like syntax. You can see that in hotspot code the > similar problems (OS depends-code) perfectly to be solved by Directives. > > So we can have support for multiple any type vars and nested layers, > and compiler can do flow analysis for nested layers (where clauses or > what ever you want). Hi Ali, I agree that, to some extent, the end goal of layers is to effectively enable some form of 'optional' membership/overriding which could also be thought of in terms of classic C++-style macros around method declarations/blocks etc. In fact, I think that, apart from partial abstractness, what's implemented right now is more or less functionally equivalent to layers (modulo bugs, of course). That said, I think we are in the search of something that would sit better with the Java programming model; granted, #ifdefs and friends will take you there, but I think it will also be overly powerful - and prone to be abused (and perhaps, as some of you have noted in this mailing list, layers has that problem); what if there was a nice little construct that, with minimal footprint could take you all the way there - meaning that you could retrofit the libraries you care about, w/o really adding a new powerful hammer to the language? I think that 'something' is the sort of magic we are after here. Maurizio From palo.marton at gmail.com Tue Jan 13 08:16:08 2015 From: palo.marton at gmail.com (Palo Marton) Date: Tue, 13 Jan 2015 09:16:08 +0100 Subject: caution: bleeding-edge ahead! In-Reply-To: <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> Message-ID: One possible abuse for all previously suggested syntaxes is that people may tend to create completelly different set of methods for different layers. And I think that IDE developers will hate us for that. (think of outline view or refactorings for generics). I would preferer to think of these as optional methods with syntax like this: class Test { optional void peeledMethod() case : { //impl1 } case : { //impl2 } default: { // imp3 } } } It is switch-like syntax and behaviour. First case that matches will be choosen as implementation and others ignored. If no case matches and there is no default, then method is not supported in that specialization. On Tue, Jan 13, 2015 at 7:46 AM, Timo Kinnunen wrote: > How about a new keyword ?__aliasOf?, which would be used in a class > declaration like this: > > > public class IntArrayList __aliasOf ArrayList implements List { > > // int layer is implemented here > > } > > > > > The effect of __aliasOf would be that user code could call > > > new ArrayList() > > > but this would actually execute like a call to > > > new IntArrayList() > > > and user code would then be compiled against the methods and fields from > IntArrayList. > > > > (And the name of the keyword is subject to change, of course.) > > > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > > > > > From: Maurizio Cimadamore > Sent: ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 > To: Ali Ebrahimi, Brian Goetz > Cc: valhalla-dev at openjdk.java.net > > > > > > > On 12/01/15 23:32, Ali Ebrahimi wrote: > > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives > > for java with java-like syntax. You can see that in hotspot code the > > similar problems (OS depends-code) perfectly to be solved by Directives. > > > > So we can have support for multiple any type vars and nested layers, > > and compiler can do flow analysis for nested layers (where clauses or > > what ever you want). > Hi Ali, > I agree that, to some extent, the end goal of layers is to effectively > enable some form of 'optional' membership/overriding which could also be > thought of in terms of classic C++-style macros around method > declarations/blocks etc. In fact, I think that, apart from partial > abstractness, what's implemented right now is more or less functionally > equivalent to layers (modulo bugs, of course). That said, I think we are > in the search of something that would sit better with the Java > programming model; granted, #ifdefs and friends will take you there, but > I think it will also be overly powerful - and prone to be abused (and > perhaps, as some of you have noted in this mailing list, layers has that > problem); what if there was a nice little construct that, with minimal > footprint could take you all the way there - meaning that you could > retrofit the libraries you care about, w/o really adding a new powerful > hammer to the language? I think that 'something' is the sort of magic we > are after here. > > Maurizio > From peter.levart at gmail.com Tue Jan 13 08:32:01 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 13 Jan 2015 09:32:01 +0100 Subject: Static fields and specialization In-Reply-To: <54B3F23C.7030606@oracle.com> References: <54B1A520.2020006@oracle.com> <54B3E918.60408@oracle.com> <54B3F23C.7030606@oracle.com> Message-ID: <54B4D801.5040306@gmail.com> On 01/12/2015 05:11 PM, Brian Goetz wrote: >> You probably misunderstood me (again) :( >> >> Richard suggested that you can keep speed of Optional.empty() by >> creating separate layers for , , etc... > > Remember, for value types (hopefully Optional will be one), > instantiation is essentially free, and in most cases where this idiom > is used, it is the allocation that we're trying to optimize away. So > the idiom of "cache one and return the same instance", in these cases, > can be retired. > > I realize this still doesn't help us for reference types, like > Collections.emptyList(); this point got made a while ago (so it didn't > occur to me we were still talking about it.) > > Hi, "generic" or "specialized" static fields are a concept I have been thinking of too, but their initialization presents a lot of challenges and consequently possible irregularities in language. The example we 1st imagine is of course the following: On 01/12/2015 01:51 PM, Palo Marton wrote: > private static final Optional EMPTY = new Optional(); > > This will compile initializing expression to generic static method: > > private static Optional EMPTY$init() { > return new Optional(); > } > ...but we know that classic static fields can be initialized in hand-crafted static initializers too: static final int x; static { ... x = ... } So the next logical thing we consider is "generic" or "specialized" static initialization blocks: On 01/12/2015 02:26 PM, MacGregor, Duncan (GE Energy Management) wrote: > That sounds fundamentally okay, but probably also requires a typed static > initialiser syntax for fields that could throw an exception during > initialisation. > > Something like > > static final MethodHandle mh; > > static { > try { > // Do MethodHandle lookup > } (catch e) { > Throw new Error(); > } > } ...but we can argue that above syntax is unsound. Currently there's no construct in Java language that would associate two lexicaly independent scopes together just by the name of generic type variable. Above two "any Ts" are distinct type variables. So we might need a top-level static scope that would allow declaring anything - from static fields to static initializer blocks and/or static methods. We already have all that. Just remove the word "static" - it's the class! So what Java might need is singleton objects (similar to Scala's): public object Singleton { final T specializedValue; Singleton() { ... specializedValue = ... } } Before you argue that such specializedValue can not be constant, hold-back... The @java.lang.invoke.Stable annotation has shown that VM already has support for constant-folding such final instance fields. Voila, here's the syntax to make this functionality "public". Serialization can be extended to support such singleton objects in a special way (like enums), reflection can be special-cased for such classes (don't allow setting final instance fields), so @Stable attribute can be apppied to them... Regards, Peter From ali.ebrahimi1781 at gmail.com Tue Jan 13 08:54:28 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Tue, 13 Jan 2015 12:24:28 +0330 Subject: caution: bleeding-edge ahead! In-Reply-To: References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> Message-ID: What is behavior if (Y is val and X is ref) or (Y is val or X is ref)? We can have nested cases? cases can have break statements? case blocks can occur on class level? .....? On Tue, Jan 13, 2015 at 11:46 AM, Palo Marton wrote: > One possible abuse for all previously suggested syntaxes is that people > may tend to create completelly different set of methods for different > layers. And I think that IDE developers will hate us for that. (think of > outline view or refactorings for generics). > > I would preferer to think of these as optional methods with syntax like > this: > > class Test { > optional void peeledMethod() > case : { > //impl1 > } > case : { > //impl2 > } > default: { > // imp3 > } > } > } > > It is switch-like syntax and behaviour. First case that matches will be > choosen as implementation and others ignored. If no case matches and there > is no default, then method is not supported in that specialization. > > On Tue, Jan 13, 2015 at 7:46 AM, Timo Kinnunen > wrote: > >> How about a new keyword ?__aliasOf?, which would be used in a class >> declaration like this: >> >> >> public class IntArrayList __aliasOf ArrayList implements List { >> >> // int layer is implemented here >> >> } >> >> >> >> >> The effect of __aliasOf would be that user code could call >> >> >> new ArrayList() >> >> >> but this would actually execute like a call to >> >> >> new IntArrayList() >> >> >> and user code would then be compiled against the methods and fields from >> IntArrayList. >> >> >> >> (And the name of the keyword is subject to change, of course.) >> >> >> >> >> >> -- >> Have a nice day, >> Timo. >> >> Sent from Windows Mail >> >> >> >> >> >> From: Maurizio Cimadamore >> Sent: ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 >> To: Ali Ebrahimi, Brian Goetz >> Cc: valhalla-dev at openjdk.java.net >> >> >> >> >> >> >> On 12/01/15 23:32, Ali Ebrahimi wrote: >> > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives >> > for java with java-like syntax. You can see that in hotspot code the >> > similar problems (OS depends-code) perfectly to be solved by Directives. >> > >> > So we can have support for multiple any type vars and nested layers, >> > and compiler can do flow analysis for nested layers (where clauses or >> > what ever you want). >> Hi Ali, >> I agree that, to some extent, the end goal of layers is to effectively >> enable some form of 'optional' membership/overriding which could also be >> thought of in terms of classic C++-style macros around method >> declarations/blocks etc. In fact, I think that, apart from partial >> abstractness, what's implemented right now is more or less functionally >> equivalent to layers (modulo bugs, of course). That said, I think we are >> in the search of something that would sit better with the Java >> programming model; granted, #ifdefs and friends will take you there, but >> I think it will also be overly powerful - and prone to be abused (and >> perhaps, as some of you have noted in this mailing list, layers has that >> problem); what if there was a nice little construct that, with minimal >> footprint could take you all the way there - meaning that you could >> retrofit the libraries you care about, w/o really adding a new powerful >> hammer to the language? I think that 'something' is the sort of magic we >> are after here. >> >> Maurizio >> > > -- Best Regards, Ali Ebrahimi From palo.marton at gmail.com Tue Jan 13 09:24:47 2015 From: palo.marton at gmail.com (Palo Marton) Date: Tue, 13 Jan 2015 10:24:47 +0100 Subject: caution: bleeding-edge ahead! In-Reply-To: References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> Message-ID: > > What is behavior if (Y is val and X is ref) > In the above example it matches with first case , so that block will be used for implementation and others ignored (even if they match). Rule is "First case that matches will be choosen as implementation and others ignored.". They are taken in order as they are declared in the source code. If there is a need to specialize on both X and Y, it can look like this: case : > or (Y is val or X is ref)? > You will jave to write that as two case statements and repeat the same code block in both of them (there is no fallthrough or break here). But do you have any real life use case for something like that? We can have nested cases? > No. cases can have break statements? > No. What is between case statemant are just actual code blocks. Only one of them will be choosen for each specialization. case blocks can occur on class level? > This has to be decided, whether it should look like this: optional void peeledMethod() case <...>: { ... } case <...>: { ... } case <...>: { ... } Or this: optional void peeledMethod() { case <...>: { ... } case <...>: { ... } case <...>: { ... } } Somebody who has experience with java source code parser should decide which is better (I don't belong to that group). > .....? > > On Tue, Jan 13, 2015 at 11:46 AM, Palo Marton > wrote: > >> One possible abuse for all previously suggested syntaxes is that people >> may tend to create completelly different set of methods for different >> layers. And I think that IDE developers will hate us for that. (think of >> outline view or refactorings for generics). >> >> I would preferer to think of these as optional methods with syntax like >> this: >> >> class Test { >> optional void peeledMethod() >> case : { >> //impl1 >> } >> case : { >> //impl2 >> } >> default: { >> // imp3 >> } >> } >> } >> >> It is switch-like syntax and behaviour. First case that matches will be >> choosen as implementation and others ignored. If no case matches and there >> is no default, then method is not supported in that specialization. >> >> On Tue, Jan 13, 2015 at 7:46 AM, Timo Kinnunen >> wrote: >> >>> How about a new keyword ?__aliasOf?, which would be used in a class >>> declaration like this: >>> >>> >>> public class IntArrayList __aliasOf ArrayList implements List { >>> >>> // int layer is implemented here >>> >>> } >>> >>> >>> >>> >>> The effect of __aliasOf would be that user code could call >>> >>> >>> new ArrayList() >>> >>> >>> but this would actually execute like a call to >>> >>> >>> new IntArrayList() >>> >>> >>> and user code would then be compiled against the methods and fields from >>> IntArrayList. >>> >>> >>> >>> (And the name of the keyword is subject to change, of course.) >>> >>> >>> >>> >>> >>> -- >>> Have a nice day, >>> Timo. >>> >>> Sent from Windows Mail >>> >>> >>> >>> >>> >>> From: Maurizio Cimadamore >>> Sent: ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 >>> To: Ali Ebrahimi, Brian Goetz >>> Cc: valhalla-dev at openjdk.java.net >>> >>> >>> >>> >>> >>> >>> On 12/01/15 23:32, Ali Ebrahimi wrote: >>> > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives >>> > for java with java-like syntax. You can see that in hotspot code the >>> > similar problems (OS depends-code) perfectly to be solved by >>> Directives. >>> > >>> > So we can have support for multiple any type vars and nested layers, >>> > and compiler can do flow analysis for nested layers (where clauses or >>> > what ever you want). >>> Hi Ali, >>> I agree that, to some extent, the end goal of layers is to effectively >>> enable some form of 'optional' membership/overriding which could also be >>> thought of in terms of classic C++-style macros around method >>> declarations/blocks etc. In fact, I think that, apart from partial >>> abstractness, what's implemented right now is more or less functionally >>> equivalent to layers (modulo bugs, of course). That said, I think we are >>> in the search of something that would sit better with the Java >>> programming model; granted, #ifdefs and friends will take you there, but >>> I think it will also be overly powerful - and prone to be abused (and >>> perhaps, as some of you have noted in this mailing list, layers has that >>> problem); what if there was a nice little construct that, with minimal >>> footprint could take you all the way there - meaning that you could >>> retrofit the libraries you care about, w/o really adding a new powerful >>> hammer to the language? I think that 'something' is the sort of magic we >>> are after here. >>> >>> Maurizio >>> >> >> > > > -- > > Best Regards, > Ali Ebrahimi > From ali.ebrahimi1781 at gmail.com Tue Jan 13 09:31:28 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Tue, 13 Jan 2015 13:01:28 +0330 Subject: caution: bleeding-edge ahead! In-Reply-To: <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> Message-ID: I don't see this how solves the problems layering tries to solve. Just now we can alias IntStream as Stream in User code maybe in import statements. import j.u.IntStream as Stream; Stream intStream; // compiles as IntStream intStream; On Tue, Jan 13, 2015 at 10:16 AM, Timo Kinnunen wrote: > How about a new keyword ?__aliasOf?, which would be used in a class > declaration like this: > > public class IntArrayList __aliasOf ArrayList implements List { > // int layer is implemented here > } > > The effect of __aliasOf would be that user code could call > > new ArrayList() > > but this would actually execute like a call to > > new IntArrayList() > > and user code would then be compiled against the methods and fields from > IntArrayList. > > > (And the name of the keyword is subject to change, of course.) > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > *From:* Maurizio Cimadamore > *Sent:* ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 > *To:* Ali Ebrahimi , Brian Goetz > > *Cc:* valhalla-dev at openjdk.java.net > > > On 12/01/15 23:32, Ali Ebrahimi wrote: > > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives > > for java with java-like syntax. You can see that in hotspot code the > > similar problems (OS depends-code) perfectly to be solved by Directives. > > > > So we can have support for multiple any type vars and nested layers, > > and compiler can do flow analysis for nested layers (where clauses or > > what ever you want). > Hi Ali, > I agree that, to some extent, the end goal of layers is to effectively > enable some form of 'optional' membership/overriding which could also be > thought of in terms of classic C++-style macros around method > declarations/blocks etc. In fact, I think that, apart from partial > abstractness, what's implemented right now is more or less functionally > equivalent to layers (modulo bugs, of course). That said, I think we are > in the search of something that would sit better with the Java > programming model; granted, #ifdefs and friends will take you there, but > I think it will also be overly powerful - and prone to be abused (and > perhaps, as some of you have noted in this mailing list, layers has that > problem); what if there was a nice little construct that, with minimal > footprint could take you all the way there - meaning that you could > retrofit the libraries you care about, w/o really adding a new powerful > hammer to the language? I think that 'something' is the sort of magic we > are after here. > > Maurizio > -- Best Regards, Ali Ebrahimi From ali.ebrahimi1781 at gmail.com Tue Jan 13 09:47:32 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Tue, 13 Jan 2015 13:17:32 +0330 Subject: caution: bleeding-edge ahead! In-Reply-To: References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> Message-ID: I sorry to say that your proposal is heavily depend on code order and may we get unexpected (or not easily understandable) result if code order changed or if a new case inserted in between of existing cases. What if we want different signatures for peeledMethod? I'm thinking of maximum code reuse. On Tue, Jan 13, 2015 at 12:54 PM, Palo Marton wrote: > What is behavior if (Y is val and X is ref) >> > > In the above example it matches with first case , so that block > will be used for implementation and others ignored (even if they match). > Rule is "First case that matches will be choosen as implementation and > others ignored.". They are taken in order as they are declared in the > source code. > > If there is a need to specialize on both X and Y, it can look like this: > > case : > > >> or (Y is val or X is ref)? >> > > You will jave to write that as two case statements and repeat the same > code block in both of them (there is no fallthrough or break here). But do > you have any real life use case for something like that? > > We can have nested cases? >> > > No. > > cases can have break statements? >> > > No. What is between case statemant are just actual code blocks. Only one > of them will be choosen for each specialization. > > case blocks can occur on class level? >> > > This has to be decided, whether it should look like this: > > optional void peeledMethod() > case <...>: { ... } > case <...>: { ... } > case <...>: { ... } > > Or this: > > optional void peeledMethod() { > case <...>: { ... } > case <...>: { ... } > case <...>: { ... } > } > > Somebody who has experience with java source code parser should decide > which is better (I don't belong to that group). > > >> .....? >> >> On Tue, Jan 13, 2015 at 11:46 AM, Palo Marton >> wrote: >> >>> One possible abuse for all previously suggested syntaxes is that people >>> may tend to create completelly different set of methods for different >>> layers. And I think that IDE developers will hate us for that. (think of >>> outline view or refactorings for generics). >>> >>> I would preferer to think of these as optional methods with syntax like >>> this: >>> >>> class Test { >>> optional void peeledMethod() >>> case : { >>> //impl1 >>> } >>> case : { >>> //impl2 >>> } >>> default: { >>> // imp3 >>> } >>> } >>> } >>> >>> It is switch-like syntax and behaviour. First case that matches will be >>> choosen as implementation and others ignored. If no case matches and there >>> is no default, then method is not supported in that specialization. >>> >>> On Tue, Jan 13, 2015 at 7:46 AM, Timo Kinnunen >>> wrote: >>> >>>> How about a new keyword ?__aliasOf?, which would be used in a class >>>> declaration like this: >>>> >>>> >>>> public class IntArrayList __aliasOf ArrayList implements List >>>> { >>>> >>>> // int layer is implemented here >>>> >>>> } >>>> >>>> >>>> >>>> >>>> The effect of __aliasOf would be that user code could call >>>> >>>> >>>> new ArrayList() >>>> >>>> >>>> but this would actually execute like a call to >>>> >>>> >>>> new IntArrayList() >>>> >>>> >>>> and user code would then be compiled against the methods and fields >>>> from IntArrayList. >>>> >>>> >>>> >>>> (And the name of the keyword is subject to change, of course.) >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Have a nice day, >>>> Timo. >>>> >>>> Sent from Windows Mail >>>> >>>> >>>> >>>> >>>> >>>> From: Maurizio Cimadamore >>>> Sent: ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 >>>> To: Ali Ebrahimi, Brian Goetz >>>> Cc: valhalla-dev at openjdk.java.net >>>> >>>> >>>> >>>> >>>> >>>> >>>> On 12/01/15 23:32, Ali Ebrahimi wrote: >>>> > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives >>>> > for java with java-like syntax. You can see that in hotspot code the >>>> > similar problems (OS depends-code) perfectly to be solved by >>>> Directives. >>>> > >>>> > So we can have support for multiple any type vars and nested layers, >>>> > and compiler can do flow analysis for nested layers (where clauses or >>>> > what ever you want). >>>> Hi Ali, >>>> I agree that, to some extent, the end goal of layers is to effectively >>>> enable some form of 'optional' membership/overriding which could also be >>>> thought of in terms of classic C++-style macros around method >>>> declarations/blocks etc. In fact, I think that, apart from partial >>>> abstractness, what's implemented right now is more or less functionally >>>> equivalent to layers (modulo bugs, of course). That said, I think we are >>>> in the search of something that would sit better with the Java >>>> programming model; granted, #ifdefs and friends will take you there, but >>>> I think it will also be overly powerful - and prone to be abused (and >>>> perhaps, as some of you have noted in this mailing list, layers has that >>>> problem); what if there was a nice little construct that, with minimal >>>> footprint could take you all the way there - meaning that you could >>>> retrofit the libraries you care about, w/o really adding a new powerful >>>> hammer to the language? I think that 'something' is the sort of magic we >>>> are after here. >>>> >>>> Maurizio >>>> >>> >>> >> >> >> -- >> >> Best Regards, >> Ali Ebrahimi >> > > -- Best Regards, Ali Ebrahimi From maurizio.cimadamore at oracle.com Tue Jan 13 12:07:33 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 13 Jan 2015 12:07:33 +0000 Subject: caution: bleeding-edge ahead! In-Reply-To: References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> Message-ID: <54B50A85.7060803@oracle.com> Timo, Ali, type aliasing won't address all issues associated with layering - i.e. the fact that we'd like, for instance, to be able to say that a method remove(Object) only exists on reference-parameterized collections. That said, something like type aliasing might be useful when doing wholesale specialization (i.e. custom specialized implementation for i.e. ArrayList). Maurizio On 13/01/15 09:31, Ali Ebrahimi wrote: > I don't see this how solves the problems layering tries to solve. Just > now we can alias IntStream as Stream in User code maybe in import > statements. > > import j.u.IntStream as Stream; > > Stream intStream; // compiles as IntStream intStream; > > > On Tue, Jan 13, 2015 at 10:16 AM, Timo Kinnunen > > wrote: > > How about a new keyword ?__aliasOf?, which would be used in a > class declaration like this: > > public class IntArrayList __aliasOf ArrayList implements > List { > // int layer is implemented here > } > > The effect of __aliasOf would be that user code could call > > new ArrayList() > > but this would actually execute like a call to > > new IntArrayList() > > and user code would then be compiled against the methods and > fields from IntArrayList. > > > (And the name of the keyword is subject to change, of course.) > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > *From:* Maurizio Cimadamore > *Sent:* ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 > *To:* Ali Ebrahimi , Brian > Goetz > *Cc:* valhalla-dev at openjdk.java.net > > > > On 12/01/15 23:32, Ali Ebrahimi wrote: > > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives > > for java with java-like syntax. You can see that in hotspot code > the > > similar problems (OS depends-code) perfectly to be solved by > Directives. > > > > So we can have support for multiple any type vars and nested > layers, > > and compiler can do flow analysis for nested layers (where > clauses or > > what ever you want). > Hi Ali, > I agree that, to some extent, the end goal of layers is to > effectively > enable some form of 'optional' membership/overriding which could > also be > thought of in terms of classic C++-style macros around method > declarations/blocks etc. In fact, I think that, apart from partial > abstractness, what's implemented right now is more or less > functionally > equivalent to layers (modulo bugs, of course). That said, I think > we are > in the search of something that would sit better with the Java > programming model; granted, #ifdefs and friends will take you > there, but > I think it will also be overly powerful - and prone to be abused (and > perhaps, as some of you have noted in this mailing list, layers > has that > problem); what if there was a nice little construct that, with > minimal > footprint could take you all the way there - meaning that you could > retrofit the libraries you care about, w/o really adding a new > powerful > hammer to the language? I think that 'something' is the sort of > magic we > are after here. > > Maurizio > > > > > -- > > Best Regards, > Ali Ebrahimi From vitalyd at gmail.com Tue Jan 13 14:29:27 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 13 Jan 2015 09:29:27 -0500 Subject: Static fields and specialization In-Reply-To: <54B4D801.5040306@gmail.com> References: <54B1A520.2020006@oracle.com> <54B3E918.60408@oracle.com> <54B3F23C.7030606@oracle.com> <54B4D801.5040306@gmail.com> Message-ID: Why not just keep the existing static init block? If you have a static generic field, I think that field should have the same T as the enclosing class, and not some other T (i.e. it's not like generic methods that can take arbitrary type parameters). Sent from my phone On Jan 13, 2015 3:32 AM, "Peter Levart" wrote: > On 01/12/2015 05:11 PM, Brian Goetz wrote: > >> You probably misunderstood me (again) :( >>> >>> Richard suggested that you can keep speed of Optional.empty() by >>> creating separate layers for , , etc... >>> >> >> Remember, for value types (hopefully Optional will be one), instantiation >> is essentially free, and in most cases where this idiom is used, it is the >> allocation that we're trying to optimize away. So the idiom of "cache one >> and return the same instance", in these cases, can be retired. >> >> I realize this still doesn't help us for reference types, like >> Collections.emptyList(); this point got made a while ago (so it didn't >> occur to me we were still talking about it.) >> >> >> > Hi, > > "generic" or "specialized" static fields are a concept I have been > thinking of too, but their initialization presents a lot of challenges and > consequently possible irregularities in language. The example we 1st > imagine is of course the following: > > On 01/12/2015 01:51 PM, Palo Marton wrote: > >> private static final Optional EMPTY = new Optional(); >> >> This will compile initializing expression to generic static method: >> >> private static Optional EMPTY$init() { >> return new Optional(); >> } >> >> > > ...but we know that classic static fields can be initialized in > hand-crafted static initializers too: > > static final int x; > static { > ... > x = ... > } > > So the next logical thing we consider is "generic" or "specialized" static > initialization blocks: > > On 01/12/2015 02:26 PM, MacGregor, Duncan (GE Energy Management) wrote: > >> That sounds fundamentally okay, but probably also requires a typed static >> initialiser syntax for fields that could throw an exception during >> initialisation. >> >> Something like >> >> static final MethodHandle mh; >> >> static { >> try { >> // Do MethodHandle lookup >> } (catch e) { >> Throw new Error(); >> } >> } >> > > ...but we can argue that above syntax is unsound. Currently there's no > construct in Java language that would associate two lexicaly independent > scopes together just by the name of generic type variable. Above two "any > Ts" are distinct type variables. So we might need a top-level static scope > that would allow declaring anything - from static fields to static > initializer blocks and/or static methods. > > We already have all that. Just remove the word "static" - it's the class! > So what Java might need is singleton objects (similar to Scala's): > > public object Singleton { > > final T specializedValue; > > Singleton() { > ... > specializedValue = ... > } > } > > Before you argue that such specializedValue can not be constant, > hold-back... > > The @java.lang.invoke.Stable annotation has shown that VM already has > support for constant-folding such final instance fields. Voila, here's the > syntax to make this functionality "public". > > Serialization can be extended to support such singleton objects in a > special way (like enums), reflection can be special-cased for such classes > (don't allow setting final instance fields), so @Stable attribute can be > apppied to them... > > > Regards, Peter > > From peter.levart at gmail.com Tue Jan 13 15:00:54 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 13 Jan 2015 16:00:54 +0100 Subject: Static fields and specialization In-Reply-To: References: <54B1A520.2020006@oracle.com> <54B3E918.60408@oracle.com> <54B3F23C.7030606@oracle.com> <54B4D801.5040306@gmail.com> Message-ID: <54B53326.70403@gmail.com> On 01/13/2015 03:29 PM, Vitaly Davidovich wrote: > Why not just keep the existing static init block? If you have a static > generic field, I think that field should have the same T as the enclosing > class, and not some other T (i.e. it's not like generic methods that can > take arbitrary type parameters). Huh, so you would propagate type variable from instance scope to static context. So you could write the following: public class Holder { static final List specializedField; static final int normalField; static { specializedField = new SomeList(); ... normalField = specializedField.size(); } ...see where this is going to? You cant "split" the single static initializer block into meaningful parts. Peter > > Sent from my phone > On Jan 13, 2015 3:32 AM, "Peter Levart" wrote: > >> On 01/12/2015 05:11 PM, Brian Goetz wrote: >> >>> You probably misunderstood me (again) :( >>>> Richard suggested that you can keep speed of Optional.empty() by >>>> creating separate layers for , , etc... >>>> >>> Remember, for value types (hopefully Optional will be one), instantiation >>> is essentially free, and in most cases where this idiom is used, it is the >>> allocation that we're trying to optimize away. So the idiom of "cache one >>> and return the same instance", in these cases, can be retired. >>> >>> I realize this still doesn't help us for reference types, like >>> Collections.emptyList(); this point got made a while ago (so it didn't >>> occur to me we were still talking about it.) >>> >>> >>> >> Hi, >> >> "generic" or "specialized" static fields are a concept I have been >> thinking of too, but their initialization presents a lot of challenges and >> consequently possible irregularities in language. The example we 1st >> imagine is of course the following: >> >> On 01/12/2015 01:51 PM, Palo Marton wrote: >> >>> private static final Optional EMPTY = new Optional(); >>> >>> This will compile initializing expression to generic static method: >>> >>> private static Optional EMPTY$init() { >>> return new Optional(); >>> } >>> >>> >> ...but we know that classic static fields can be initialized in >> hand-crafted static initializers too: >> >> static final int x; >> static { >> ... >> x = ... >> } >> >> So the next logical thing we consider is "generic" or "specialized" static >> initialization blocks: >> >> On 01/12/2015 02:26 PM, MacGregor, Duncan (GE Energy Management) wrote: >> >>> That sounds fundamentally okay, but probably also requires a typed static >>> initialiser syntax for fields that could throw an exception during >>> initialisation. >>> >>> Something like >>> >>> static final MethodHandle mh; >>> >>> static { >>> try { >>> // Do MethodHandle lookup >>> } (catch e) { >>> Throw new Error(); >>> } >>> } >>> >> ...but we can argue that above syntax is unsound. Currently there's no >> construct in Java language that would associate two lexicaly independent >> scopes together just by the name of generic type variable. Above two "any >> Ts" are distinct type variables. So we might need a top-level static scope >> that would allow declaring anything - from static fields to static >> initializer blocks and/or static methods. >> >> We already have all that. Just remove the word "static" - it's the class! >> So what Java might need is singleton objects (similar to Scala's): >> >> public object Singleton { >> >> final T specializedValue; >> >> Singleton() { >> ... >> specializedValue = ... >> } >> } >> >> Before you argue that such specializedValue can not be constant, >> hold-back... >> >> The @java.lang.invoke.Stable annotation has shown that VM already has >> support for constant-folding such final instance fields. Voila, here's the >> syntax to make this functionality "public". >> >> Serialization can be extended to support such singleton objects in a >> special way (like enums), reflection can be special-cased for such classes >> (don't allow setting final instance fields), so @Stable attribute can be >> apppied to them... >> >> >> Regards, Peter >> >> From timo.kinnunen at gmail.com Tue Jan 13 14:38:53 2015 From: timo.kinnunen at gmail.com (=?utf-8?Q?Timo_Kinnunen?=) Date: Tue, 13 Jan 2015 14:38:53 +0000 Subject: =?utf-8?Q?Re:_caution:_bleeding-edge_ahead!?= In-Reply-To: <54B50A85.7060803@oracle.com> References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> , <54B50A85.7060803@oracle.com> Message-ID: <54b53346.64c8c20a.405e.338b@mx.google.com> I think it could be used to address that as well. You can?t add layers to an existing class without rewriting its source code, right? So under that allowance, you could rewrite java.util.List by splitting it into two, like this: public interface List extends PackagePrivateBaseList { T remove(int index); boolean remove(Object o); } Then provide another interface for value types: interface PackagePrivateAnyList __aliasOf List extends PackagePrivateBaseList { T removeByIndex(int index); boolean removeByValue(T o); } And finally provide implementations for references and values, respectively: public class ArrayList implements List { /* ? */ } class PackagePrivateArrayAnyList __aliasOf ArrayList implements List { /* ? */ } This is probably not binary-compatible as-is, but maybe that could be addressed as well. -- Have a nice day, Timo. Sent from Windows Mail From: Maurizio Cimadamore Sent: ?Tuesday?, ?January? ?13?, ?2015 ?13?:?07 To: Ali Ebrahimi, Timo Kinnunen Cc: Brian Goetz, valhalla-dev at openjdk.java.net Timo, Ali, type aliasing won't address all issues associated with layering - i.e. the fact that we'd like, for instance, to be able to say that a method remove(Object) only exists on reference-parameterized collections. That said, something like type aliasing might be useful when doing wholesale specialization (i.e. custom specialized implementation for i.e. ArrayList). Maurizio On 13/01/15 09:31, Ali Ebrahimi wrote: I don't see this how solves the problems layering tries to solve. Just now we can alias IntStream as Stream in User code maybe in import statements. import j.u.IntStream as Stream; Stream intStream; // compiles as IntStream intStream; On Tue, Jan 13, 2015 at 10:16 AM, Timo Kinnunen wrote: How about a new keyword ?__aliasOf?, which would be used in a class declaration like this: public class IntArrayList __aliasOf ArrayList implements List { // int layer is implemented here } The effect of __aliasOf would be that user code could call new ArrayList() but this would actually execute like a call to new IntArrayList() and user code would then be compiled against the methods and fields from IntArrayList. (And the name of the keyword is subject to change, of course.) -- Have a nice day, Timo. Sent from Windows Mail From: Maurizio Cimadamore Sent: ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 To: Ali Ebrahimi, Brian Goetz Cc: valhalla-dev at openjdk.java.net On 12/01/15 23:32, Ali Ebrahimi wrote: > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives > for java with java-like syntax. You can see that in hotspot code the > similar problems (OS depends-code) perfectly to be solved by Directives. > > So we can have support for multiple any type vars and nested layers, > and compiler can do flow analysis for nested layers (where clauses or > what ever you want). Hi Ali, I agree that, to some extent, the end goal of layers is to effectively enable some form of 'optional' membership/overriding which could also be thought of in terms of classic C++-style macros around method declarations/blocks etc. In fact, I think that, apart from partial abstractness, what's implemented right now is more or less functionally equivalent to layers (modulo bugs, of course). That said, I think we are in the search of something that would sit better with the Java programming model; granted, #ifdefs and friends will take you there, but I think it will also be overly powerful - and prone to be abused (and perhaps, as some of you have noted in this mailing list, layers has that problem); what if there was a nice little construct that, with minimal footprint could take you all the way there - meaning that you could retrofit the libraries you care about, w/o really adding a new powerful hammer to the language? I think that 'something' is the sort of magic we are after here. Maurizio -- Best Regards, Ali Ebrahimi From vitalyd at gmail.com Tue Jan 13 15:12:21 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 13 Jan 2015 10:12:21 -0500 Subject: Static fields and specialization In-Reply-To: <54B53326.70403@gmail.com> References: <54B1A520.2020006@oracle.com> <54B3E918.60408@oracle.com> <54B3F23C.7030606@oracle.com> <54B4D801.5040306@gmail.com> <54B53326.70403@gmail.com> Message-ID: I don't consider generic type param as being instance scope (or any scope for that matter). In your Holder example, the effect would be that each specialization of the class would have its own normalField static member. The upside here is this doesn't require any special handling of static generic fields and is intuitive (to me at least). Sent from my phone On Jan 13, 2015 10:00 AM, "Peter Levart" wrote: > On 01/13/2015 03:29 PM, Vitaly Davidovich wrote: > >> Why not just keep the existing static init block? If you have a static >> generic field, I think that field should have the same T as the enclosing >> class, and not some other T (i.e. it's not like generic methods that can >> take arbitrary type parameters). >> > > Huh, so you would propagate type variable from instance scope to static > context. So you could write the following: > > public class Holder { > > static final List specializedField; > static final int normalField; > > static { > specializedField = new SomeList(); > ... > normalField = specializedField.size(); > } > > > ...see where this is going to? You cant "split" the single static > initializer block into meaningful parts. > > > Peter > > >> Sent from my phone >> On Jan 13, 2015 3:32 AM, "Peter Levart" wrote: >> >> On 01/12/2015 05:11 PM, Brian Goetz wrote: >>> >>> You probably misunderstood me (again) :( >>>> >>>>> Richard suggested that you can keep speed of Optional.empty() by >>>>> creating separate layers for , , etc... >>>>> >>>>> Remember, for value types (hopefully Optional will be one), >>>> instantiation >>>> is essentially free, and in most cases where this idiom is used, it is >>>> the >>>> allocation that we're trying to optimize away. So the idiom of "cache >>>> one >>>> and return the same instance", in these cases, can be retired. >>>> >>>> I realize this still doesn't help us for reference types, like >>>> Collections.emptyList(); this point got made a while ago (so it didn't >>>> occur to me we were still talking about it.) >>>> >>>> >>>> >>>> Hi, >>> >>> "generic" or "specialized" static fields are a concept I have been >>> thinking of too, but their initialization presents a lot of challenges >>> and >>> consequently possible irregularities in language. The example we 1st >>> imagine is of course the following: >>> >>> On 01/12/2015 01:51 PM, Palo Marton wrote: >>> >>> private static final Optional EMPTY = new Optional(); >>>> >>>> This will compile initializing expression to generic static method: >>>> >>>> private static Optional EMPTY$init() { >>>> return new Optional(); >>>> } >>>> >>>> >>>> ...but we know that classic static fields can be initialized in >>> hand-crafted static initializers too: >>> >>> static final int x; >>> static { >>> ... >>> x = ... >>> } >>> >>> So the next logical thing we consider is "generic" or "specialized" >>> static >>> initialization blocks: >>> >>> On 01/12/2015 02:26 PM, MacGregor, Duncan (GE Energy Management) wrote: >>> >>> That sounds fundamentally okay, but probably also requires a typed >>>> static >>>> initialiser syntax for fields that could throw an exception during >>>> initialisation. >>>> >>>> Something like >>>> >>>> static final MethodHandle mh; >>>> >>>> static { >>>> try { >>>> // Do MethodHandle lookup >>>> } (catch e) { >>>> Throw new Error(); >>>> } >>>> } >>>> >>>> ...but we can argue that above syntax is unsound. Currently there's no >>> construct in Java language that would associate two lexicaly independent >>> scopes together just by the name of generic type variable. Above two "any >>> Ts" are distinct type variables. So we might need a top-level static >>> scope >>> that would allow declaring anything - from static fields to static >>> initializer blocks and/or static methods. >>> >>> We already have all that. Just remove the word "static" - it's the class! >>> So what Java might need is singleton objects (similar to Scala's): >>> >>> public object Singleton { >>> >>> final T specializedValue; >>> >>> Singleton() { >>> ... >>> specializedValue = ... >>> } >>> } >>> >>> Before you argue that such specializedValue can not be constant, >>> hold-back... >>> >>> The @java.lang.invoke.Stable annotation has shown that VM already has >>> support for constant-folding such final instance fields. Voila, here's >>> the >>> syntax to make this functionality "public". >>> >>> Serialization can be extended to support such singleton objects in a >>> special way (like enums), reflection can be special-cased for such >>> classes >>> (don't allow setting final instance fields), so @Stable attribute can be >>> apppied to them... >>> >>> >>> Regards, Peter >>> >>> >>> > From maurizio.cimadamore at oracle.com Tue Jan 13 16:20:00 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 13 Jan 2015 16:20:00 +0000 Subject: hg: valhalla/valhalla/jdk: Add support for (some) Object method calls on 'any' tvars Message-ID: <201501131620.t0DGK0EQ026839@aojmv0008> Changeset: 5bec6f407925 Author: mcimadamore Date: 2015-01-13 16:19 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/5bec6f407925 Add support for (some) Object method calls on 'any' tvars * handle object method calls on values * add helper class for handling primitive comparisons * fix iicmp local var offset for 2-slots specialized values + src/java.base/share/classes/valhalla/runtime/EqualsHelper.java ! src/java.base/share/classes/valhalla/specializer/SignatureSpecializer.java ! src/java.base/share/classes/valhalla/specializer/Specializer.java + test/valhalla/test/valhalla/specializer/ObjectMethodsTest.java From maurizio.cimadamore at oracle.com Tue Jan 13 16:23:09 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 13 Jan 2015 16:23:09 +0000 Subject: hg: valhalla/valhalla/langtools: Add support for (some) Object method calls on 'any' tvars Message-ID: <201501131623.t0DGN9LD027253@aojmv0008> Changeset: ec4dbf00c278 Author: mcimadamore Date: 2015-01-13 16:22 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/ec4dbf00c278 Add support for (some) Object method calls on 'any' tvars * add type-checking support for the following Object methods when called from an 'any' typevar: -toString -hashCode -equals * implement stricter semantics when calling 'equals' on an 'any' tvar: values to be compared should have same static type * added tests ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties - test/tools/javac/diags/examples/TypeVarCantBeDerefAny.java ! test/tools/javac/valhalla/typespec/AnyReference.java ! test/tools/javac/valhalla/typespec/AnyReference.out ! test/tools/javac/valhalla/typespec/Instanceof01.java ! test/tools/javac/valhalla/typespec/Instanceof01.out + test/tools/javac/valhalla/typespec/ObjectMethods.java + test/tools/javac/valhalla/typespec/ObjectMethods.out From maurizio.cimadamore at oracle.com Tue Jan 13 16:35:40 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 13 Jan 2015 16:35:40 +0000 Subject: more band aid coming - Object methods on 'any' tvars Message-ID: <54B5495C.6030509@oracle.com> Hi, in order to further facilitate experiments with the current prototype, I've put together a compiler/specializer patch that adds some restricted support for calling 'harmless' Object methods on 'any' type-variables. Methods currently supported are: * toString * hashCode * equals The remaining Object methods are unsupported (most of them are related to locking ); the only two 'interesting' methods left out are: * clone * getClass While it wouldn't be impossible to support them in the future, they are not as straightforward to get to (esp. getClass) - and we decided to leave them out for now. One caveat about using Object methods on 'any' type-variables is that the signature of 'equals' is stricter; that is, you can imagine that an 'any' type-variable T defines a method equals(T) and not equals(Object) That is: the compared values must have the same static type or the compiler will bark. I hope you will find this useful. Happy hacking! Maurizio From duncan.macgregor at ge.com Thu Jan 15 11:09:43 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Thu, 15 Jan 2015 11:09:43 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54B5495C.6030509@oracle.com> References: <54B5495C.6030509@oracle.com> Message-ID: Okay, I?ve used this to write a little toy hash table. These methods do not work if the type is concrete at compile time, which feels weird. I can call toString() on an int provided the compiler doesn?t know it?s an int. Part of implementing my collection involved making a suitable Optional style type, and this turns out to be slightly more fiddly than I might have hoped as there is no way to specify a default value for any val. Java.util.Optional defines EMPTY as an optional with its final field set to null, but I can?t do that across any T. I?ve worked round the problem by using two subclasses, Empty (which doesn?t have a value field), and Present (which does and always initialises it), but this only works because my Optional is a reference type, and I don?t think we really want Optional type things to be ref types in the long run. Duncan. On 13/01/2015 16:35, "Maurizio Cimadamore" wrote: >Hi, >in order to further facilitate experiments with the current prototype, >I've put together a compiler/specializer patch that adds some restricted >support for calling 'harmless' Object methods on 'any' type-variables. >Methods currently supported are: > >* toString >* hashCode >* equals > >The remaining Object methods are unsupported (most of them are related >to locking ); the only two 'interesting' methods left out are: > >* clone >* getClass > >While it wouldn't be impossible to support them in the future, they are >not as straightforward to get to (esp. getClass) - and we decided to >leave them out for now. > >One caveat about using Object methods on 'any' type-variables is that >the signature of 'equals' is stricter; that is, you can imagine that an >'any' type-variable T defines a method > >equals(T) > >and not > >equals(Object) > >That is: the compared values must have the same static type or the >compiler will bark. > >I hope you will find this useful. > >Happy hacking! > >Maurizio > From maurizio.cimadamore at oracle.com Thu Jan 15 11:14:44 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 15 Jan 2015 11:14:44 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: References: <54B5495C.6030509@oracle.com> Message-ID: <54B7A124.9060700@oracle.com> On 15/01/15 11:09, MacGregor, Duncan (GE Energy Management) wrote: > Okay, I?ve used this to write a little toy hash table. These methods do > not work if the type is concrete at compile time, which feels weird. I can > call toString() on an int provided the compiler doesn?t know it?s an int. Right - I should have included that in the 'limitations' section - i.e. 1.toString() won't work. > > Part of implementing my collection involved making a suitable Optional T> style type, and this turns out to be slightly more fiddly than I might > have hoped as there is no way to specify a default value for any val. There might be some updates soon in that department :-) > Java.util.Optional defines EMPTY as an optional with its final field > set to null, but I can?t do that across any T. I?ve worked round the > problem by using two subclasses, Empty (which doesn?t have a value > field), and Present (which does and always initialises it), but this > only works because my Optional is a reference type, and I don?t think we > really want Optional type things to be ref types in the long run. I think some of the proposals I've seen on the table were about using an extra boolean in order to capture the 'is present' info. Maurizio > > Duncan. > > On 13/01/2015 16:35, "Maurizio Cimadamore" > wrote: > >> Hi, >> in order to further facilitate experiments with the current prototype, >> I've put together a compiler/specializer patch that adds some restricted >> support for calling 'harmless' Object methods on 'any' type-variables. >> Methods currently supported are: >> >> * toString >> * hashCode >> * equals >> >> The remaining Object methods are unsupported (most of them are related >> to locking ); the only two 'interesting' methods left out are: >> >> * clone >> * getClass >> >> While it wouldn't be impossible to support them in the future, they are >> not as straightforward to get to (esp. getClass) - and we decided to >> leave them out for now. >> >> One caveat about using Object methods on 'any' type-variables is that >> the signature of 'equals' is stricter; that is, you can imagine that an >> 'any' type-variable T defines a method >> >> equals(T) >> >> and not >> >> equals(Object) >> >> That is: the compared values must have the same static type or the >> compiler will bark. >> >> I hope you will find this useful. >> >> Happy hacking! >> >> Maurizio >> From duncan.macgregor at ge.com Thu Jan 15 11:19:16 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Thu, 15 Jan 2015 11:19:16 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54B7A124.9060700@oracle.com> References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> Message-ID: On 15/01/2015 11:14, "Maurizio Cimadamore" wrote: >On 15/01/15 11:09, MacGregor, Duncan (GE Energy Management) wrote: >>Part of implementing my collection involved making a suitable >>Optional> T> style type, and this turns out to be slightly more fiddly than I >>might >> have hoped as there is no way to specify a default value for any val. >There might be some updates soon in that department :-) Lovely. >> Java.util.Optional defines EMPTY as an optional with its final field >> set to null, but I can?t do that across any T. I?ve worked round the >> problem by using two subclasses, Empty (which doesn?t have a value >> field), and Present (which does and always initialises it), but this >> only works because my Optional is a reference type, and I don?t think we >> really want Optional type things to be ref types in the long run. >I think some of the proposals I've seen on the table were about using an >extra boolean in order to capture the 'is present' info. That is how I was intending to implement it, but if I don?t want to subclass then I need have a type something like Class Optional { final T value; final boolean hasValue; } And that means I need to set value to something, even for the empty case. Hence the need for defaults. Duncan. From maurizio.cimadamore at oracle.com Thu Jan 15 11:35:05 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 15 Jan 2015 11:35:05 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> Message-ID: <54B7A5E9.3090003@oracle.com> > That is how I was intending to implement it, but if I don?t want to > subclass then I need have a type something like > > Class Optional { > final T value; > final boolean hasValue; > } > > And that means I need to set value to something, even for the empty case. > Hence the need for defaults. Right - I hear you. Maurizio > > Duncan. > From maurizio.cimadamore at oracle.com Fri Jan 16 12:22:59 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 16 Jan 2015 12:22:59 +0000 Subject: hg: valhalla/valhalla/jdk: Add support for statically-typed default literals Message-ID: <201501161222.t0GCMxgb016238@aojmv0008> Changeset: 2193aca0ba68 Author: mcimadamore Date: 2015-01-16 12:22 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/2193aca0ba68 Add support for statically-typed default literals * handle specialization of aconst_null * add test ! src/java.base/share/classes/valhalla/specializer/Specializer.java + test/valhalla/test/valhalla/specializer/DefaultValueTest.java From maurizio.cimadamore at oracle.com Fri Jan 16 12:24:37 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 16 Jan 2015 12:24:37 +0000 Subject: hg: valhalla/valhalla/langtools: Add support for statically typed default literals Message-ID: <201501161224.t0GCOb6A016422@aojmv0008> Changeset: 82892f0cd83a Author: mcimadamore Date: 2015-01-16 12:24 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/82892f0cd83a Add support for statically typed default literals * add parser/type-checking/codegen support T.default * added tests ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/DeferredAttr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Code.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/tree/JCTree.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeInfo.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeMaker.java + test/tools/javac/valhalla/typespec/Default01.java + test/tools/javac/valhalla/typespec/Default01.out + test/tools/javac/valhalla/typespec/Default02.java ! test/tools/javac/valhalla/typespec/items/Opcodes.java + test/tools/javac/valhalla/typespec/items/tests/TestDefault.java From maurizio.cimadamore at oracle.com Fri Jan 16 12:29:37 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 16 Jan 2015 12:29:37 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54B7A5E9.3090003@oracle.com> References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> Message-ID: <54B90431.1090403@oracle.com> I've just pushed a compiler/specializer patch that adds support for T.default. This is internally handled as an 'aconst_null' which has a BytecodeMapping entry attached to it (in case T is an 'any' type-var). The specializer will then replace the aconst_null with the proper XYZconst_0. Again, to keep things simpler at this stage, stuff like 'int.default' will not work (parser will reject that). This should still be enough to get you going with the experiments. Happy coding! Maurizio On 15/01/15 11:35, Maurizio Cimadamore wrote: >> That is how I was intending to implement it, but if I don?t want to >> subclass then I need have a type something like >> >> Class Optional { >> final T value; >> final boolean hasValue; >> } >> >> And that means I need to set value to something, even for the empty case. >> Hence the need for defaults. > Right - I hear you. > > Maurizio >> Duncan. >> From duncan.macgregor at ge.com Fri Jan 16 15:22:48 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Fri, 16 Jan 2015 15:22:48 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54B90431.1090403@oracle.com> References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> Message-ID: Thanks for that, I can now write a sensible optional class and it behaves as expected. One bug I have found is that getField is not always being correctly specialised in all cases. Specifically a final boolean field used in an if statement such as if (!isPresent) { throw... } else { return otherField; } Was not marked for specialisation and caused a byte code verification error, but using a method to get the field value worked perfectly. Duncan. On 16/01/2015 12:29, "Maurizio Cimadamore" wrote: >I've just pushed a compiler/specializer patch that adds support for >T.default. > >This is internally handled as an 'aconst_null' which has a >BytecodeMapping entry attached to it (in case T is an 'any' type-var). >The specializer will then replace the aconst_null with the proper >XYZconst_0. > >Again, to keep things simpler at this stage, stuff like 'int.default' >will not work (parser will reject that). This should still be enough to >get you going with the experiments. > >Happy coding! > >Maurizio > > >On 15/01/15 11:35, Maurizio Cimadamore wrote: >>> That is how I was intending to implement it, but if I don?t want to >>> subclass then I need have a type something like >>> >>> Class Optional { >>> final T value; >>> final boolean hasValue; >>> } >>> >>> And that means I need to set value to something, even for the empty >>>case. >>> Hence the need for defaults. >> Right - I hear you. >> >> Maurizio >>> Duncan. >>> > From maurizio.cimadamore at oracle.com Fri Jan 16 15:37:43 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 16 Jan 2015 15:37:43 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> Message-ID: <54B93047.5050004@oracle.com> On 16/01/15 15:22, MacGregor, Duncan (GE Energy Management) wrote: > Thanks for that, I can now write a sensible optional class and it behaves > as expected. > > One bug I have found is that getField is not always being correctly > specialised in all cases. Specifically a final boolean field used in an if > statement such as > > if (!isPresent) { throw... } else { return otherField; } > > Was not marked for specialisation and caused a byte code verification > error, but using a method to get the field value worked perfectly. Thanks for the report, I'll take a look. Maurizio > > Duncan. > > On 16/01/2015 12:29, "Maurizio Cimadamore" > wrote: > >> I've just pushed a compiler/specializer patch that adds support for >> T.default. >> >> This is internally handled as an 'aconst_null' which has a >> BytecodeMapping entry attached to it (in case T is an 'any' type-var). >> The specializer will then replace the aconst_null with the proper >> XYZconst_0. >> >> Again, to keep things simpler at this stage, stuff like 'int.default' >> will not work (parser will reject that). This should still be enough to >> get you going with the experiments. >> >> Happy coding! >> >> Maurizio >> >> >> On 15/01/15 11:35, Maurizio Cimadamore wrote: >>>> That is how I was intending to implement it, but if I don?t want to >>>> subclass then I need have a type something like >>>> >>>> Class Optional { >>>> final T value; >>>> final boolean hasValue; >>>> } >>>> >>>> And that means I need to set value to something, even for the empty >>>> case. >>>> Hence the need for defaults. >>> Right - I hear you. >>> >>> Maurizio >>>> Duncan. >>>> From simon at ochsenreither.de Fri Jan 16 16:54:35 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Fri, 16 Jan 2015 17:54:35 +0100 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54B90431.1090403@oracle.com> References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> Message-ID: <54B9424B.5020000@ochsenreither.de> > I've just pushed a compiler/specializer patch that adds support for > T.default. I really don't want to start a syntax discussion, but what's the reason to introduce a special syntax where existing syntax would have been sufficient? From brian.goetz at oracle.com Fri Jan 16 16:59:06 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 16 Jan 2015 11:59:06 -0500 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54B9424B.5020000@ochsenreither.de> References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> <54B9424B.5020000@ochsenreither.de> Message-ID: <54B9435A.1050203@oracle.com> If you mean "null" as the existing syntax... that would be semantically confusing. It would creates the idea in the user's head that null and 0 are the same thing. Then they'll expect "x == null" to mean "x == 0" for ints. They'll expect to be able to assign null directly to an int, and have it coerced to zero. And now it's not just syntax, its conceptual, and they have to build a new concept of null. Overloading this syntax essentially pushes us into a different mental model, and hence a different feature. We definitely DO NOT want to just say "null for int means 0". So while this sounds like a syntax issue, it's not really. On 1/16/2015 11:54 AM, Simon Ochsenreither wrote: >> I've just pushed a compiler/specializer patch that adds support for >> T.default. > > I really don't want to start a syntax discussion, but what's the reason > to introduce a special syntax where existing syntax would have been > sufficient? > From simon at ochsenreither.de Fri Jan 16 17:05:00 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Fri, 16 Jan 2015 18:05:00 +0100 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54B9435A.1050203@oracle.com> References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> <54B9424B.5020000@ochsenreither.de> <54B9435A.1050203@oracle.com> Message-ID: <54B944BC.8010504@ochsenreither.de> > If you mean "null" as the existing syntax... No, just something like: T default() From brian.goetz at oracle.com Fri Jan 16 17:18:13 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 16 Jan 2015 12:18:13 -0500 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54B944BC.8010504@ochsenreither.de> References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> <54B9424B.5020000@ochsenreither.de> <54B9435A.1050203@oracle.com> <54B944BC.8010504@ochsenreither.de> Message-ID: <54B947D5.408@oracle.com> Sure, we could have done that -- and we might well end up doing that (and same with "new T[]"). Don't forget, nothing we do right now constitutes any sort of final decision -- especially syntax stuff like this. Right now, we're interested in exploring the impact of specialization on migration of existing generic code. This was a reasonable option; there are other reasonable options too. On 1/16/2015 12:05 PM, Simon Ochsenreither wrote: >> If you mean "null" as the existing syntax... > > No, just something like: > > T default() > From richard.warburton at gmail.com Sat Jan 17 12:36:32 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Sat, 17 Jan 2015 12:36:32 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54B5495C.6030509@oracle.com> References: <54B5495C.6030509@oracle.com> Message-ID: Hi Maurizio, I think I've found a bug here. You can't call the object methods on anyified arrays. Here's a minimal test case and stack trace: public static void fail(T[] values) { values[0].toString(); } Workaround: public static void fail(T[] values) { T value = values[0]; value.toString(); } Javac stacktrace: java.lang.NullPointerException at com.sun.tools.javac.comp.Attr.selectSym(Attr.java:3433) at com.sun.tools.javac.comp.Attr.selectSym(Attr.java:3420) at com.sun.tools.javac.comp.Attr.visitSelect(Attr.java:3258) at com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2082) at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) at com.sun.tools.javac.comp.Attr.visitApply(Attr.java:1800) at com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1608) at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) at com.sun.tools.javac.comp.Attr.attribExpr(Attr.java:589) at com.sun.tools.javac.comp.Attr.visitExec(Attr.java:1568) at com.sun.tools.javac.tree.JCTree$JCExpressionStatement.accept(JCTree.java:1426) at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) at com.sun.tools.javac.comp.Attr.attribStats(Attr.java:625) at com.sun.tools.javac.comp.Attr.visitBlock(Attr.java:1112) at com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:994) at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) at com.sun.tools.javac.comp.Attr.visitMethodDef(Attr.java:997) at com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:843) at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) at com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:4404) at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:4312) at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:4234) at com.sun.tools.javac.comp.Attr.attrib(Attr.java:4209) at com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1230) at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:872) at com.sun.tools.javac.main.Main.compile(Main.java:249) at com.sun.tools.javac.main.Main.compile(Main.java:140) at com.sun.tools.javac.Main.compile(Main.java:56) at com.sun.tools.javac.Main.main(Main.java:42) regards, Richard Warburton http://insightfullogic.com @RichardWarburto From richard.warburton at gmail.com Sat Jan 17 13:00:26 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Sat, 17 Jan 2015 13:00:26 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: References: <54B5495C.6030509@oracle.com> Message-ID: Hi, Actually it appears to be that you can't call the object method on any expression that's more complex that referencing a variable. Eg: public static void fail(T[] values) { get(values).toString(); } public static T get(T[] values) { return values[0]; } Also NPEs. On Sat, Jan 17, 2015 at 12:36 PM, Richard Warburton < richard.warburton at gmail.com> wrote: > Hi Maurizio, > > I think I've found a bug here. You can't call the object methods on > anyified arrays. Here's a minimal test case and stack trace: > > public static void fail(T[] values) { > values[0].toString(); > } > > Workaround: > > public static void fail(T[] values) { > T value = values[0]; > value.toString(); > } > > Javac stacktrace: > > java.lang.NullPointerException > at com.sun.tools.javac.comp.Attr.selectSym(Attr.java:3433) > at com.sun.tools.javac.comp.Attr.selectSym(Attr.java:3420) > at com.sun.tools.javac.comp.Attr.visitSelect(Attr.java:3258) > at > com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2082) > at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) > at com.sun.tools.javac.comp.Attr.visitApply(Attr.java:1800) > at > com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1608) > at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) > at com.sun.tools.javac.comp.Attr.attribExpr(Attr.java:589) > at com.sun.tools.javac.comp.Attr.visitExec(Attr.java:1568) > at > com.sun.tools.javac.tree.JCTree$JCExpressionStatement.accept(JCTree.java:1426) > at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) > at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) > at com.sun.tools.javac.comp.Attr.attribStats(Attr.java:625) > at com.sun.tools.javac.comp.Attr.visitBlock(Attr.java:1112) > at com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:994) > at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) > at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) > at com.sun.tools.javac.comp.Attr.visitMethodDef(Attr.java:997) > at > com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:843) > at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) > at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) > at com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:4404) > at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:4312) > at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:4234) > at com.sun.tools.javac.comp.Attr.attrib(Attr.java:4209) > at > com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1230) > at > com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:872) > at com.sun.tools.javac.main.Main.compile(Main.java:249) > at com.sun.tools.javac.main.Main.compile(Main.java:140) > at com.sun.tools.javac.Main.compile(Main.java:56) > at com.sun.tools.javac.Main.main(Main.java:42) > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > -- regards, Richard Warburton http://insightfullogic.com @RichardWarburto From aka.bash0r at gmail.com Sat Jan 17 19:06:21 2015 From: aka.bash0r at gmail.com (Nils Jonsson) Date: Sat, 17 Jan 2015 20:06:21 +0100 Subject: Guide for building OpenJDK Valhalla Message-ID: Hey everyone, I wanted to start helping testing the Valhalla project after reading on the mailing list recently. Is there any guide or something on the web where I can get started building the project? Thanks in advance! From ivan.st.ivanov at gmail.com Sat Jan 17 20:06:29 2015 From: ivan.st.ivanov at gmail.com (Ivan St. Ivanov) Date: Sat, 17 Jan 2015 22:06:29 +0200 Subject: Guide for building OpenJDK Valhalla In-Reply-To: References: Message-ID: Hi Nils, Have you built openjdk so far? If not, there are the build instructions: https://java.net/projects/adoptopenjdk/pages/WhatToWorkOnForOpenJDK#Build_OpenJDK You may notice that there are instructions for building OpenJDK on your own machine or on VM. I would recommend to use Linux. I've done it once on Windows, but it was really hard. So, if you are not running Linux on your computer, then choose virtual machine. Now, you can follow the same instructions for building Valhalla. The only difference is the repository where you get the sources. So, at the GetSource step (https://java.net/projects/adoptopenjdk/pages/GetSource) instead of doing: hg clone http://hg.openjdk.java.net/jdk9/dev jdk9 ...for Valhalla you should do something along these lines: hg clone http://hg.openjdk.java.net/valhalla/valhalla valhalla The rest should be the same. I hope this helps! Ivan On Sat, Jan 17, 2015 at 9:06 PM, Nils Jonsson wrote: > Hey everyone, > > I wanted to start helping testing the Valhalla project after reading on the > mailing list recently. Is there any guide or something on the web where I > can get started building the project? > > Thanks in advance! > From martijnverburg at gmail.com Sat Jan 17 20:32:57 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Sat, 17 Jan 2015 20:32:57 +0000 Subject: Guide for building OpenJDK Valhalla In-Reply-To: References: Message-ID: I should be adding the instructions soon on the Valhalla home page - apologies for the delays! On Saturday, 17 January 2015, Ivan St. Ivanov wrote: > Hi Nils, > > Have you built openjdk so far? If not, there are the build instructions: > > > https://java.net/projects/adoptopenjdk/pages/WhatToWorkOnForOpenJDK#Build_OpenJDK > > You may notice that there are instructions for building OpenJDK on your own > machine or on VM. I would recommend to use Linux. I've done it once on > Windows, but it was really hard. So, if you are not running Linux on your > computer, then choose virtual machine. > > Now, you can follow the same instructions for building Valhalla. The only > difference is the repository where you get the sources. So, at the > GetSource step (https://java.net/projects/adoptopenjdk/pages/GetSource) > instead of doing: > > hg clone http://hg.openjdk.java.net/jdk9/dev jdk9 > > ...for Valhalla you should do something along these lines: > > hg clone http://hg.openjdk.java.net/valhalla/valhalla valhalla > > The rest should be the same. > > I hope this helps! > Ivan > > > On Sat, Jan 17, 2015 at 9:06 PM, Nils Jonsson > wrote: > > > Hey everyone, > > > > I wanted to start helping testing the Valhalla project after reading on > the > > mailing list recently. Is there any guide or something on the web where I > > can get started building the project? > > > > Thanks in advance! > > > -- Cheers, Martijn From aka.bash0r at gmail.com Sat Jan 17 20:34:28 2015 From: aka.bash0r at gmail.com (Nils Jonsson) Date: Sat, 17 Jan 2015 21:34:28 +0100 Subject: Guide for building OpenJDK Valhalla In-Reply-To: References: Message-ID: Thanks a lot! I'll try to build it the next few days. Am 17.01.2015 21:32 schrieb "Martijn Verburg" : > I should be adding the instructions soon on the Valhalla home page - > apologies for the delays! > > On Saturday, 17 January 2015, Ivan St. Ivanov > wrote: > >> Hi Nils, >> >> Have you built openjdk so far? If not, there are the build instructions: >> >> >> https://java.net/projects/adoptopenjdk/pages/WhatToWorkOnForOpenJDK#Build_OpenJDK >> >> You may notice that there are instructions for building OpenJDK on your >> own >> machine or on VM. I would recommend to use Linux. I've done it once on >> Windows, but it was really hard. So, if you are not running Linux on your >> computer, then choose virtual machine. >> >> Now, you can follow the same instructions for building Valhalla. The only >> difference is the repository where you get the sources. So, at the >> GetSource step (https://java.net/projects/adoptopenjdk/pages/GetSource) >> instead of doing: >> >> hg clone http://hg.openjdk.java.net/jdk9/dev jdk9 >> >> ...for Valhalla you should do something along these lines: >> >> hg clone http://hg.openjdk.java.net/valhalla/valhalla valhalla >> >> The rest should be the same. >> >> I hope this helps! >> Ivan >> >> >> On Sat, Jan 17, 2015 at 9:06 PM, Nils Jonsson >> wrote: >> >> > Hey everyone, >> > >> > I wanted to start helping testing the Valhalla project after reading on >> the >> > mailing list recently. Is there any guide or something on the web where >> I >> > can get started building the project? >> > >> > Thanks in advance! >> > >> > > > -- > Cheers, > Martijn > From maurizio.cimadamore at oracle.com Sat Jan 17 21:34:13 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Sat, 17 Jan 2015 21:34:13 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: References: <54B5495C.6030509@oracle.com> Message-ID: <54BAD555.8040401@oracle.com> Thanks for the report; I will fix this. Maurizio On 17/01/15 13:00, Richard Warburton wrote: > Hi, > > Actually it appears to be that you can't call the object method on any > expression that's more complex that referencing a variable. Eg: > > public static void fail(T[] values) { > get(values).toString(); > } > > public static T get(T[] values) { > return values[0]; > } > > Also NPEs. > > > On Sat, Jan 17, 2015 at 12:36 PM, Richard Warburton > > wrote: > > Hi Maurizio, > > I think I've found a bug here. You can't call the object methods > on anyified arrays. Here's a minimal test case and stack trace: > > public static void fail(T[] values) { > values[0].toString(); > } > > Workaround: > > public static void fail(T[] values) { > T value = values[0]; > value.toString(); > } > > Javac stacktrace: > > java.lang.NullPointerException > at com.sun.tools.javac.comp.Attr.selectSym(Attr.java:3433) > at com.sun.tools.javac.comp.Attr.selectSym(Attr.java:3420) > at com.sun.tools.javac.comp.Attr.visitSelect(Attr.java:3258) > at > com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2082) > at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) > at com.sun.tools.javac.comp.Attr.visitApply(Attr.java:1800) > at > com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1608) > at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) > at com.sun.tools.javac.comp.Attr.attribExpr(Attr.java:589) > at com.sun.tools.javac.comp.Attr.visitExec(Attr.java:1568) > at > com.sun.tools.javac.tree.JCTree$JCExpressionStatement.accept(JCTree.java:1426) > at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) > at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) > at com.sun.tools.javac.comp.Attr.attribStats(Attr.java:625) > at com.sun.tools.javac.comp.Attr.visitBlock(Attr.java:1112) > at > com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:994) > at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) > at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) > at com.sun.tools.javac.comp.Attr.visitMethodDef(Attr.java:997) > at > com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:843) > at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) > at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) > at > com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:4404) > at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:4312) > at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:4234) > at com.sun.tools.javac.comp.Attr.attrib(Attr.java:4209) > at > com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1230) > at > com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:872) > at com.sun.tools.javac.main.Main.compile(Main.java:249) > at com.sun.tools.javac.main.Main.compile(Main.java:140) > at com.sun.tools.javac.Main.compile(Main.java:56) > at com.sun.tools.javac.Main.main(Main.java:42) > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > > > > > -- > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto From twhitmore.nz at gmail.com Sun Jan 18 04:41:34 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Sun, 18 Jan 2015 17:41:34 +1300 Subject: __WhereRef/WhereVal peeling, #ifdef and related proposals Message-ID: Maurizio's "peeling" patch is interesting. As he noted, his proposal is not a real syntax.. just a hack to take experimentation further. "Peeling" -- as he has titled it -- is programmatic manual specialization. This is method specific, in that different methods may depend on a different subset of the class's type arguments. We should be thinking about this in terms of methods having a "multi-dimensional space" in which they are specialized; with 1 dimension for each type on which the method is specialized. Note that some method codes may not be manually specialized at all, some may only depend on only one of the generic type arguments, others may depend on more.. A signature-based illustration (in reality, implementations might tend to suck in V but we need to start sketching somewhere): public class CountMap { public void clear(); // not manually specialized public void increment (K key); // manually specialize on K; possibly also on V? public void add (K key, V value); // manually specialize on K, V public V get (K key); // manually specialize on K, V static private V valueInc (V value); // manually specialize just on V } Correct output bytecode should completely cover (either by explicit code sections, by defaulting mechanism, or by automatically specializable code) the entire space of specializations which could be requested. Since we required a well-structured output, completely covering a well-defined output space, I would tend to avoid low-level constructs as #ifdef. These allow arbitrary spatial patchworks to be created, of high complexity but not great for covering the output space in a simple & reliable way. Case statement, as standard, is only one-dimensional. A multi-dimensional "specialization case" statement could, however, be ideal; it can be defined to have useful rules of matching (most-specific wins?) and can fairly easily be checked to prove it correctly covers all output space. Let's have a look at CountMap.increment(); specializing on K. public void increment (K key) { int hash = specialize (K) { case (ref) : (key != null) ? key.hashCode() : 0; case (val) : key.hashCode(); case (int) : key; }; //.. etc And to specialize on two dimensions: public void someMethod (K key, V value) { specialize (K, V) { case (ref, any) : etc1(); case (val, any) : etc2(); case (val, int) : etc3(); } //.. etc Taking this further, such blocks could be nested; and/or mixed with normal code. The outermost case would not need to contain all dimensions; dimensionality of the method overall would be the product of dimensionalities of all such 'specialization case' statements within it. Syntactically, it might possibly (probably?) help to be able to have a _result value_ and for this 'specialization case' to actually be an expression. I've exemplified this above. -------------- I'm not actually arguing for the "manual specialization of methods" concept, here -- I've yet to reach a decision on that. In some ways, the increment() code I've written above illustrates the same point I was making with "EQ" -- how being able to code that is portable & efficient, at the logical level, between valtype/ reftype/ primitives would be mostly superior to manual specialization. HashCode above is an example of this issue. In the one counter-example I've heard -- packing of booleans to bits, eg List -- I think that *classes* are exactly the mechanism we should use! I would much prefer to have a manually written BooleanArrayList than "magic" specialization of storage layouts below the byte-level. However, in terms of 'manual specialization' syntax I would like us to have a high-level view of the problem & express it at this level. C's #ifdef syntax is far too ugly & dangerous -- most possible permutations are wrong, so I'm dead against it. Regards, Thomas From mikeb01 at gmail.com Sun Jan 18 09:45:59 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Sun, 18 Jan 2015 22:45:59 +1300 Subject: caution: bleeding-edge ahead! In-Reply-To: <54b53346.64c8c20a.405e.338b@mx.google.com> References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> <54B50A85.7060803@oracle.com> <54b53346.64c8c20a.405e.338b@mx.google.com> Message-ID: Hi, The following compiles, but crashes at runtime. public class C { public void foo() { __WhereVal(T) { ; } __WhereRef(T) { ; } } public static void main(String[] args) { C c = new C(); c.foo(); } } [barkerm at localhost SpecialisedMap]$ $JAVA_HOME/bin/javac -d bin src/github/mikeb01/C.java && $JAVA_HOME/bin/java -cp bin github.mikeb01.C Specializing github.mikeb01.C${0=I}; searching for github/mikeb01/C.class (not found) Specializing github.mikeb01.C${0=I}; searching for github/mikeb01/C.class (found) Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 256 at jdk.internal.org.objectweb.asm.ClassReader.readUTF8(ClassReader.java:2445) at valhalla.specializer.WhereAttribute.read(WhereAttribute.java:62) at jdk.internal.org.objectweb.asm.ClassReader.readAttribute(ClassReader.java:2312) at jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:965) at jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729) at valhalla.specializer.Specializer.specialize(Specializer.java:77) at java.net.URLClassLoader$1.run(URLClassLoader.java:409) at java.net.URLClassLoader$1.run(URLClassLoader.java:386) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:385) at java.lang.ClassLoader.loadClass(ClassLoader.java:426) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317) at java.lang.ClassLoader.loadClass(ClassLoader.java:359) at github.mikeb01.C.main(C.java:14) Mike. On 14 January 2015 at 03:38, Timo Kinnunen wrote: > > I think it could be used to address that as well. You can?t add layers to > an existing class without rewriting its source code, right? So under that > allowance, you could rewrite java.util.List by splitting it into two, like > this: > > > public interface List extends PackagePrivateBaseList { > > > T remove(int index); > > boolean remove(Object o); > > > } > > > > Then provide another interface for value types: > > > > > interface PackagePrivateAnyList __aliasOf List extends > PackagePrivateBaseList { > > > T removeByIndex(int index); > > boolean removeByValue(T o); > > > } > > > > And finally provide implementations for references and values, > respectively: > > > public class ArrayList implements List { /* ? */ } > > > class PackagePrivateArrayAnyList __aliasOf ArrayList > implements List { /* ? */ } > > > > > This is probably not binary-compatible as-is, but maybe that could be > addressed as well. > > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > > > > > From: Maurizio Cimadamore > Sent: ?Tuesday?, ?January? ?13?, ?2015 ?13?:?07 > To: Ali Ebrahimi, Timo Kinnunen > Cc: Brian Goetz, valhalla-dev at openjdk.java.net > > > > > Timo, Ali, > type aliasing won't address all issues associated with layering - i.e. the > fact that we'd like, for instance, to be able to say that a method > remove(Object) only exists on reference-parameterized collections. > > That said, something like type aliasing might be useful when doing > wholesale specialization (i.e. custom specialized implementation for i.e. > ArrayList). > > Maurizio > > > On 13/01/15 09:31, Ali Ebrahimi wrote: > > > > > I don't see this how solves the problems layering tries to solve. Just now > we can alias IntStream as Stream in User code maybe in import > statements. > > > import j.u.IntStream as Stream; > > Stream intStream; // compiles as IntStream intStream; > > > > > On Tue, Jan 13, 2015 at 10:16 AM, Timo Kinnunen > wrote: > > > > > How about a new keyword ?__aliasOf?, which would be used in a class > declaration like this: > > > > > public class IntArrayList __aliasOf ArrayList implements List { > > // int layer is implemented here > > } > > > > > > > The effect of __aliasOf would be that user code could call > > > > > new ArrayList() > > > > > but this would actually execute like a call to > > > > > new IntArrayList() > > > > > and user code would then be compiled against the methods and fields from > IntArrayList. > > > > > > > > (And the name of the keyword is subject to change, of course.) > > > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > > > > > From: Maurizio Cimadamore > Sent: ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 > To: Ali Ebrahimi, Brian Goetz > Cc: valhalla-dev at openjdk.java.net > > > > > > > > > On 12/01/15 23:32, Ali Ebrahimi wrote: > > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives > > for java with java-like syntax. You can see that in hotspot code the > > similar problems (OS depends-code) perfectly to be solved by Directives. > > > > So we can have support for multiple any type vars and nested layers, > > and compiler can do flow analysis for nested layers (where clauses or > > what ever you want). > Hi Ali, > I agree that, to some extent, the end goal of layers is to effectively > enable some form of 'optional' membership/overriding which could also be > thought of in terms of classic C++-style macros around method > declarations/blocks etc. In fact, I think that, apart from partial > abstractness, what's implemented right now is more or less functionally > equivalent to layers (modulo bugs, of course). That said, I think we are > in the search of something that would sit better with the Java > programming model; granted, #ifdefs and friends will take you there, but > I think it will also be overly powerful - and prone to be abused (and > perhaps, as some of you have noted in this mailing list, layers has that > problem); what if there was a nice little construct that, with minimal > footprint could take you all the way there - meaning that you could > retrofit the libraries you care about, w/o really adding a new powerful > hammer to the language? I think that 'something' is the sort of magic we > are after here. > > Maurizio > > > > > -- > > > > > Best Regards, > > Ali Ebrahimi > From ali.ebrahimi1781 at gmail.com Sun Jan 18 11:17:42 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Sun, 18 Jan 2015 14:47:42 +0330 Subject: caution: bleeding-edge ahead! In-Reply-To: References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> <54B50A85.7060803@oracle.com> <54b53346.64c8c20a.405e.338b@mx.google.com> Message-ID: This works for me with latest code. On Sun, Jan 18, 2015 at 1:15 PM, Michael Barker wrote: > Hi, > > The following compiles, but crashes at runtime. > > public class C { > public void foo() { > __WhereVal(T) { > ; > } > __WhereRef(T) { > ; > } > } > > public static void main(String[] args) { > C c = new C(); > c.foo(); > } > } > > [barkerm at localhost SpecialisedMap]$ $JAVA_HOME/bin/javac -d bin > src/github/mikeb01/C.java && $JAVA_HOME/bin/java -cp bin github.mikeb01.C > Specializing github.mikeb01.C${0=I}; searching for github/mikeb01/C.class > (not found) > Specializing github.mikeb01.C${0=I}; searching for github/mikeb01/C.class > (found) > Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 256 > at > jdk.internal.org.objectweb.asm.ClassReader.readUTF8(ClassReader.java:2445) > at valhalla.specializer.WhereAttribute.read(WhereAttribute.java:62) > at > jdk.internal.org.objectweb.asm.ClassReader.readAttribute(ClassReader.java:2312) > at > jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:965) > at jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729) > at valhalla.specializer.Specializer.specialize(Specializer.java:77) > at java.net.URLClassLoader$1.run(URLClassLoader.java:409) > at java.net.URLClassLoader$1.run(URLClassLoader.java:386) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:385) > at java.lang.ClassLoader.loadClass(ClassLoader.java:426) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317) > at java.lang.ClassLoader.loadClass(ClassLoader.java:359) > at github.mikeb01.C.main(C.java:14) > > Mike. > > On 14 January 2015 at 03:38, Timo Kinnunen > wrote: > >> >> I think it could be used to address that as well. You can?t add layers to >> an existing class without rewriting its source code, right? So under that >> allowance, you could rewrite java.util.List by splitting it into two, like >> this: >> >> >> public interface List extends PackagePrivateBaseList { >> >> >> T remove(int index); >> >> boolean remove(Object o); >> >> >> } >> >> >> >> Then provide another interface for value types: >> >> >> >> >> interface PackagePrivateAnyList __aliasOf List extends >> PackagePrivateBaseList { >> >> >> T removeByIndex(int index); >> >> boolean removeByValue(T o); >> >> >> } >> >> >> >> And finally provide implementations for references and values, >> respectively: >> >> >> public class ArrayList implements List { /* ? */ } >> >> >> class PackagePrivateArrayAnyList __aliasOf ArrayList >> implements List { /* ? */ } >> >> >> >> >> This is probably not binary-compatible as-is, but maybe that could be >> addressed as well. >> >> >> >> >> -- >> Have a nice day, >> Timo. >> >> Sent from Windows Mail >> >> >> >> >> >> From: Maurizio Cimadamore >> Sent: ?Tuesday?, ?January? ?13?, ?2015 ?13?:?07 >> To: Ali Ebrahimi, Timo Kinnunen >> Cc: Brian Goetz, valhalla-dev at openjdk.java.net >> >> >> >> >> Timo, Ali, >> type aliasing won't address all issues associated with layering - i.e. >> the fact that we'd like, for instance, to be able to say that a method >> remove(Object) only exists on reference-parameterized collections. >> >> That said, something like type aliasing might be useful when doing >> wholesale specialization (i.e. custom specialized implementation for i.e. >> ArrayList). >> >> Maurizio >> >> >> On 13/01/15 09:31, Ali Ebrahimi wrote: >> >> >> >> >> I don't see this how solves the problems layering tries to solve. Just >> now we can alias IntStream as Stream in User code maybe in import >> statements. >> >> >> import j.u.IntStream as Stream; >> >> Stream intStream; // compiles as IntStream intStream; >> >> >> >> >> On Tue, Jan 13, 2015 at 10:16 AM, Timo Kinnunen >> wrote: >> >> >> >> >> How about a new keyword ?__aliasOf?, which would be used in a class >> declaration like this: >> >> >> >> >> public class IntArrayList __aliasOf ArrayList implements List { >> >> // int layer is implemented here >> >> } >> >> >> >> >> >> >> The effect of __aliasOf would be that user code could call >> >> >> >> >> new ArrayList() >> >> >> >> >> but this would actually execute like a call to >> >> >> >> >> new IntArrayList() >> >> >> >> >> and user code would then be compiled against the methods and fields from >> IntArrayList. >> >> >> >> >> >> >> >> (And the name of the keyword is subject to change, of course.) >> >> >> >> >> >> -- >> Have a nice day, >> Timo. >> >> Sent from Windows Mail >> >> >> >> >> >> From: Maurizio Cimadamore >> Sent: ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 >> To: Ali Ebrahimi, Brian Goetz >> Cc: valhalla-dev at openjdk.java.net >> >> >> >> >> >> >> >> >> On 12/01/15 23:32, Ali Ebrahimi wrote: >> > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives >> > for java with java-like syntax. You can see that in hotspot code the >> > similar problems (OS depends-code) perfectly to be solved by Directives. >> > >> > So we can have support for multiple any type vars and nested layers, >> > and compiler can do flow analysis for nested layers (where clauses or >> > what ever you want). >> Hi Ali, >> I agree that, to some extent, the end goal of layers is to effectively >> enable some form of 'optional' membership/overriding which could also be >> thought of in terms of classic C++-style macros around method >> declarations/blocks etc. In fact, I think that, apart from partial >> abstractness, what's implemented right now is more or less functionally >> equivalent to layers (modulo bugs, of course). That said, I think we are >> in the search of something that would sit better with the Java >> programming model; granted, #ifdefs and friends will take you there, but >> I think it will also be overly powerful - and prone to be abused (and >> perhaps, as some of you have noted in this mailing list, layers has that >> problem); what if there was a nice little construct that, with minimal >> footprint could take you all the way there - meaning that you could >> retrofit the libraries you care about, w/o really adding a new powerful >> hammer to the language? I think that 'something' is the sort of magic we >> are after here. >> >> Maurizio >> >> >> >> >> -- >> >> >> >> >> Best Regards, >> >> Ali Ebrahimi >> > > -- Best Regards, Ali Ebrahimi From palo.marton at gmail.com Sun Jan 18 12:04:15 2015 From: palo.marton at gmail.com (Palo Marton) Date: Sun, 18 Jan 2015 13:04:15 +0100 Subject: caution: bleeding-edge ahead! In-Reply-To: <54b53346.64c8c20a.405e.338b@mx.google.com> References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> <54B50A85.7060803@oracle.com> <54b53346.64c8c20a.405e.338b@mx.google.com> Message-ID: In some previous post in this thread there was a suggestion to use "class aliasing" as a solution to manual specialization of a class. I don't agree with that. But thinking of aliases, they might be useful for something else: Quote from Brian's "State of specialization": a successful conversion would be to end up with IntStream extends Stream and be able to jettison the large amount of hand-specialized code that the streams library contains. If this will be achieved (and that's a big IF), then may be it can be pushed even further to: IntStream.class == Stream.class On the surface it seems that all that is needed for this is to have some ability to specify class names for some specializations, something like: "class IntStream aliasof Stream; (no class body here)". On the JVM/bytecode level this will mean that specialized class should get specified name instead of some unspecified JVM dependent name. But this depends on that big IF and that is probably what the JVM guys are working on at the moment. On Tue, Jan 13, 2015 at 3:38 PM, Timo Kinnunen wrote: > > I think it could be used to address that as well. You can?t add layers to > an existing class without rewriting its source code, right? So under that > allowance, you could rewrite java.util.List by splitting it into two, like > this: > > > public interface List extends PackagePrivateBaseList { > > > T remove(int index); > > boolean remove(Object o); > > > } > > > > Then provide another interface for value types: > > > > > interface PackagePrivateAnyList __aliasOf List extends > PackagePrivateBaseList { > > > T removeByIndex(int index); > > boolean removeByValue(T o); > > > } > > > > And finally provide implementations for references and values, > respectively: > > > public class ArrayList implements List { /* ? */ } > > > class PackagePrivateArrayAnyList __aliasOf ArrayList > implements List { /* ? */ } > > > > > This is probably not binary-compatible as-is, but maybe that could be > addressed as well. > > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > > > > > From: Maurizio Cimadamore > Sent: ?Tuesday?, ?January? ?13?, ?2015 ?13?:?07 > To: Ali Ebrahimi, Timo Kinnunen > Cc: Brian Goetz, valhalla-dev at openjdk.java.net > > > > > Timo, Ali, > type aliasing won't address all issues associated with layering - i.e. the > fact that we'd like, for instance, to be able to say that a method > remove(Object) only exists on reference-parameterized collections. > > That said, something like type aliasing might be useful when doing > wholesale specialization (i.e. custom specialized implementation for i.e. > ArrayList). > > Maurizio > > > On 13/01/15 09:31, Ali Ebrahimi wrote: > > > > > I don't see this how solves the problems layering tries to solve. Just now > we can alias IntStream as Stream in User code maybe in import > statements. > > > import j.u.IntStream as Stream; > > Stream intStream; // compiles as IntStream intStream; > > > > > On Tue, Jan 13, 2015 at 10:16 AM, Timo Kinnunen > wrote: > > > > > How about a new keyword ?__aliasOf?, which would be used in a class > declaration like this: > > > > > public class IntArrayList __aliasOf ArrayList implements List { > > // int layer is implemented here > > } > > > > > > > The effect of __aliasOf would be that user code could call > > > > > new ArrayList() > > > > > but this would actually execute like a call to > > > > > new IntArrayList() > > > > > and user code would then be compiled against the methods and fields from > IntArrayList. > > > > > > > > (And the name of the keyword is subject to change, of course.) > > > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > > > > > From: Maurizio Cimadamore > Sent: ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 > To: Ali Ebrahimi, Brian Goetz > Cc: valhalla-dev at openjdk.java.net > > > > > > > > > On 12/01/15 23:32, Ali Ebrahimi wrote: > > Why we can not adapt C++'s #if, #elif, #else, and #endif Directives > > for java with java-like syntax. You can see that in hotspot code the > > similar problems (OS depends-code) perfectly to be solved by Directives. > > > > So we can have support for multiple any type vars and nested layers, > > and compiler can do flow analysis for nested layers (where clauses or > > what ever you want). > Hi Ali, > I agree that, to some extent, the end goal of layers is to effectively > enable some form of 'optional' membership/overriding which could also be > thought of in terms of classic C++-style macros around method > declarations/blocks etc. In fact, I think that, apart from partial > abstractness, what's implemented right now is more or less functionally > equivalent to layers (modulo bugs, of course). That said, I think we are > in the search of something that would sit better with the Java > programming model; granted, #ifdefs and friends will take you there, but > I think it will also be overly powerful - and prone to be abused (and > perhaps, as some of you have noted in this mailing list, layers has that > problem); what if there was a nice little construct that, with minimal > footprint could take you all the way there - meaning that you could > retrofit the libraries you care about, w/o really adding a new powerful > hammer to the language? I think that 'something' is the sort of magic we > are after here. > > Maurizio > > > > > -- > > > > > Best Regards, > > Ali Ebrahimi > From brian.goetz at oracle.com Sun Jan 18 15:29:43 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 18 Jan 2015 10:29:43 -0500 Subject: caution: bleeding-edge ahead! In-Reply-To: References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> <54B50A85.7060803@oracle.com> <54b53346.64c8c20a.405e.338b@mx.google.com> Message-ID: <54BBD167.9060505@oracle.com> > If this will be achieved (and that's a big IF), then may be it can be > pushed even further to: > > IntStream.class == Stream.class Note that IntStream is not simply the result of "take Stream and replace T with int." It has methods not in Stream (e.g., sum()), and some of the specialized methods not only specialize T -> int but things like Supplier -> IntSupplier. So the above equality is even a bigger "if" than you are probably imagining. From brian.goetz at oracle.com Sun Jan 18 16:35:45 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 18 Jan 2015 11:35:45 -0500 Subject: __WhereRef/WhereVal peeling, #ifdef and related proposals In-Reply-To: References: Message-ID: <54BBE0E1.2050605@oracle.com> Please do not characterize the strawman __WhereRef syntax as any sort of "proposal" (as you do in your message title.) Such things have a tendency to get out of hand, and before you know it, you get news article like this one: http://sdtimes.com/javas-generics-getting-generic/ which seriously thought we were "proposing" to make Java developers append "*T" to class names to get specialization. (You can't make this stuff up, but you reduce the degree to which you feed it.) If your point was "OMG I hope they're not considering a lexical preprocessor", hopefully you'll sleep better knowing that this has always been off the table. There are many ways to slice type-casing; your example shows one (though you need a "meet rule" when you get to the multiple-tvar case), but there are others. We'll consider syntax when we have a clearer grasp on the model we've settled on. On 1/17/2015 11:41 PM, Thomas W wrote: > Maurizio's "peeling" patch is interesting. As he noted, his proposal is not > a real syntax.. just a hack to take experimentation further. > > "Peeling" -- as he has titled it -- is programmatic manual specialization. > This is method specific, in that different methods may depend on a > different subset of the class's type arguments. > > We should be thinking about this in terms of methods having a > "multi-dimensional space" in which they are specialized; with 1 dimension > for each type on which the method is specialized. > > Note that some method codes may not be manually specialized at all, some > may only depend on only one of the generic type arguments, others may > depend on more.. > > A signature-based illustration (in reality, implementations might tend to > suck in V but we need to start sketching somewhere): > > public class CountMap { > public void clear(); // not manually specialized > public void increment (K key); // manually specialize on K; > possibly also on V? > public void add (K key, V value); // manually specialize on K, V > public V get (K key); // manually specialize on K, V > > static private V valueInc (V value); // manually specialize just > on V > } > > > Correct output bytecode should completely cover (either by explicit code > sections, by defaulting mechanism, or by automatically specializable code) > the entire space of specializations which could be requested. > > Since we required a well-structured output, completely covering a > well-defined output space, I would tend to avoid low-level constructs as > #ifdef. These allow arbitrary spatial patchworks to be created, of high > complexity but not great for covering the output space in a simple & > reliable way. > > Case statement, as standard, is only one-dimensional. > > A multi-dimensional "specialization case" statement could, however, be > ideal; it can be defined to have useful rules of matching (most-specific > wins?) and can fairly easily be checked to prove it correctly covers all > output space. > > Let's have a look at CountMap.increment(); specializing on K. > > public void increment (K key) { > int hash = specialize (K) { > case (ref) : (key != null) ? key.hashCode() : 0; > case (val) : key.hashCode(); > case (int) : key; > }; > //.. etc > > And to specialize on two dimensions: > > public void someMethod (K key, V value) { > specialize (K, V) { > case (ref, any) : etc1(); > case (val, any) : etc2(); > case (val, int) : etc3(); > } > //.. etc > > Taking this further, such blocks could be nested; and/or mixed with normal > code. The outermost case would not need to contain all dimensions; > dimensionality of the method overall would be the product of > dimensionalities of all such 'specialization case' statements within it. > > Syntactically, it might possibly (probably?) help to be able to have a > _result value_ and for this 'specialization case' to actually be an > expression. I've exemplified this above. > -------------- > > I'm not actually arguing for the "manual specialization of methods" > concept, here -- I've yet to reach a decision on that. > > In some ways, the increment() code I've written above illustrates the same > point I was making with "EQ" -- how being able to code that is portable & > efficient, at the logical level, between valtype/ reftype/ primitives would > be mostly superior to manual specialization. HashCode above is an example > of this issue. > > In the one counter-example I've heard -- packing of booleans to bits, eg > List -- I think that *classes* are exactly the mechanism we should > use! I would much prefer to have a manually written BooleanArrayList than > "magic" specialization of storage layouts below the byte-level. > > However, in terms of 'manual specialization' syntax I would like us to have > a high-level view of the problem & express it at this level. C's #ifdef > syntax is far too ugly & dangerous -- most possible permutations are wrong, > so I'm dead against it. > > > Regards, > Thomas > From duncan.macgregor at ge.com Sun Jan 18 21:10:20 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Sun, 18 Jan 2015 21:10:20 +0000 Subject: Guide for building OpenJDK Valhalla In-Reply-To: References: , Message-ID: <978D194CF4618446926EDAD23C949D9939BE078C@LONURLNA08.e2k.ad.ge.com> One quick note, bug 8062816 (https://bugs.openjdk.java.net/browse/JDK-8062816) will prevent building on OS X Yosemite unless you manually do a graft from the main jdk9-dev branch. The fix should be merged next time changed are puled in from jdk9. Just a heads up if you're building on OS X. Duncan. ________________________________________ From: valhalla-dev [valhalla-dev-bounces at openjdk.java.net] on behalf of Nils Jonsson [aka.bash0r at gmail.com] Sent: 17 January 2015 20:34 To: valhalla-dev at openjdk.java.net Subject: Re: Guide for building OpenJDK Valhalla Thanks a lot! I'll try to build it the next few days. Am 17.01.2015 21:32 schrieb "Martijn Verburg" : > I should be adding the instructions soon on the Valhalla home page - > apologies for the delays! > > On Saturday, 17 January 2015, Ivan St. Ivanov > wrote: > >> Hi Nils, >> >> Have you built openjdk so far? If not, there are the build instructions: >> >> >> https://java.net/projects/adoptopenjdk/pages/WhatToWorkOnForOpenJDK#Build_OpenJDK >> >> You may notice that there are instructions for building OpenJDK on your >> own >> machine or on VM. I would recommend to use Linux. I've done it once on >> Windows, but it was really hard. So, if you are not running Linux on your >> computer, then choose virtual machine. >> >> Now, you can follow the same instructions for building Valhalla. The only >> difference is the repository where you get the sources. So, at the >> GetSource step (https://java.net/projects/adoptopenjdk/pages/GetSource) >> instead of doing: >> >> hg clone http://hg.openjdk.java.net/jdk9/dev jdk9 >> >> ...for Valhalla you should do something along these lines: >> >> hg clone http://hg.openjdk.java.net/valhalla/valhalla valhalla >> >> The rest should be the same. >> >> I hope this helps! >> Ivan >> >> >> On Sat, Jan 17, 2015 at 9:06 PM, Nils Jonsson >> wrote: >> >> > Hey everyone, >> > >> > I wanted to start helping testing the Valhalla project after reading on >> the >> > mailing list recently. Is there any guide or something on the web where >> I >> > can get started building the project? >> > >> > Thanks in advance! >> > >> > > > -- > Cheers, > Martijn > From maurizio.cimadamore at oracle.com Sun Jan 18 21:41:46 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Sun, 18 Jan 2015 21:41:46 +0000 Subject: caution: bleeding-edge ahead! In-Reply-To: References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> <54B50A85.7060803@oracle.com> <54b53346.64c8c20a.405e.338b@mx.google.com> Message-ID: <54BC289A.6020403@oracle.com> On 18/01/15 11:17, Ali Ebrahimi wrote: > This works for me with latest code. Thanks - I will take a look anyway to see what's going on. Maurizio > > On Sun, Jan 18, 2015 at 1:15 PM, Michael Barker > wrote: > > Hi, > > The following compiles, but crashes at runtime. > > public class C { > public void foo() { > __WhereVal(T) { > ; > } > __WhereRef(T) { > ; > } > } > > public static void main(String[] args) { > C c = new C(); > c.foo(); > } > } > > [barkerm at localhost SpecialisedMap]$ $JAVA_HOME/bin/javac -d bin > src/github/mikeb01/C.java && $JAVA_HOME/bin/java -cp bin > github.mikeb01.C > Specializing github.mikeb01.C${0=I}; searching for > github/mikeb01/C.class (not found) > Specializing github.mikeb01.C${0=I}; searching for > github/mikeb01/C.class (found) > Exception in thread "main" > java.lang.ArrayIndexOutOfBoundsException: 256 > at > jdk.internal.org.objectweb.asm.ClassReader.readUTF8(ClassReader.java:2445) > at valhalla.specializer.WhereAttribute.read(WhereAttribute.java:62) > at > jdk.internal.org.objectweb.asm.ClassReader.readAttribute(ClassReader.java:2312) > at > jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:965) > at > jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729) > at valhalla.specializer.Specializer.specialize(Specializer.java:77) > at java.net.URLClassLoader$1.run(URLClassLoader.java:409) > at java.net.URLClassLoader$1.run(URLClassLoader.java:386) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:385) > at java.lang.ClassLoader.loadClass(ClassLoader.java:426) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317) > at java.lang.ClassLoader.loadClass(ClassLoader.java:359) > at github.mikeb01.C.main(C.java:14) > > Mike. > > On 14 January 2015 at 03:38, Timo Kinnunen > > wrote: > > > I think it could be used to address that as well. You can?t > add layers to an existing class without rewriting its source > code, right? So under that allowance, you could rewrite > java.util.List by splitting it into two, like this: > > > public interface List extends PackagePrivateBaseList { > > > T remove(int index); > > boolean remove(Object o); > > > } > > > > Then provide another interface for value types: > > > > > interface PackagePrivateAnyList __aliasOf List > extends PackagePrivateBaseList { > > > T removeByIndex(int index); > > boolean removeByValue(T o); > > > } > > > > And finally provide implementations for references and values, > respectively: > > > public class ArrayList implements List { /* ? */ } > > > class PackagePrivateArrayAnyList __aliasOf > ArrayList implements List { /* ? */ } > > > > > This is probably not binary-compatible as-is, but maybe that > could be addressed as well. > > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > > > > > From: Maurizio Cimadamore > Sent: ?Tuesday?, ?January? ?13?, ?2015 ?13?:?07 > To: Ali Ebrahimi, Timo Kinnunen > Cc: Brian Goetz, valhalla-dev at openjdk.java.net > > > > > > Timo, Ali, > type aliasing won't address all issues associated with > layering - i.e. the fact that we'd like, for instance, to be > able to say that a method remove(Object) only exists on > reference-parameterized collections. > > That said, something like type aliasing might be useful when > doing wholesale specialization (i.e. custom specialized > implementation for i.e. ArrayList). > > Maurizio > > > On 13/01/15 09:31, Ali Ebrahimi wrote: > > > > > I don't see this how solves the problems layering tries to > solve. Just now we can alias IntStream as Stream in User > code maybe in import statements. > > > import j.u.IntStream as Stream; > > Stream intStream; // compiles as IntStream intStream; > > > > > On Tue, Jan 13, 2015 at 10:16 AM, Timo Kinnunen > > wrote: > > > > > How about a new keyword ?__aliasOf?, which would be used in a > class declaration like this: > > > > > public class IntArrayList __aliasOf ArrayList implements > List { > > // int layer is implemented here > > } > > > > > > > The effect of __aliasOf would be that user code could call > > > > > new ArrayList() > > > > > but this would actually execute like a call to > > > > > new IntArrayList() > > > > > and user code would then be compiled against the methods and > fields from IntArrayList. > > > > > > > > (And the name of the keyword is subject to change, of course.) > > > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > > > > > From: Maurizio Cimadamore > Sent: ?Tuesday?, ?January? ?13?, ?2015 ?1?:?21 > To: Ali Ebrahimi, Brian Goetz > Cc: valhalla-dev at openjdk.java.net > > > > > > > > > > On 12/01/15 23:32, Ali Ebrahimi wrote: > > Why we can not adapt C++'s #if, #elif, #else, and #endif > Directives > > for java with java-like syntax. You can see that in hotspot > code the > > similar problems (OS depends-code) perfectly to be solved by > Directives. > > > > So we can have support for multiple any type vars and nested > layers, > > and compiler can do flow analysis for nested layers (where > clauses or > > what ever you want). > Hi Ali, > I agree that, to some extent, the end goal of layers is to > effectively > enable some form of 'optional' membership/overriding which > could also be > thought of in terms of classic C++-style macros around method > declarations/blocks etc. In fact, I think that, apart from partial > abstractness, what's implemented right now is more or less > functionally > equivalent to layers (modulo bugs, of course). That said, I > think we are > in the search of something that would sit better with the Java > programming model; granted, #ifdefs and friends will take you > there, but > I think it will also be overly powerful - and prone to be > abused (and > perhaps, as some of you have noted in this mailing list, > layers has that > problem); what if there was a nice little construct that, with > minimal > footprint could take you all the way there - meaning that you > could > retrofit the libraries you care about, w/o really adding a new > powerful > hammer to the language? I think that 'something' is the sort > of magic we > are after here. > > Maurizio > > > > > -- > > > > > Best Regards, > > Ali Ebrahimi > > > > > > -- > > Best Regards, > Ali Ebrahimi From mikeb01 at gmail.com Sun Jan 18 22:03:38 2015 From: mikeb01 at gmail.com (Michael Barker) Date: Mon, 19 Jan 2015 11:03:38 +1300 Subject: caution: bleeding-edge ahead! In-Reply-To: <54BC289A.6020403@oracle.com> References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> <54B50A85.7060803@oracle.com> <54b53346.64c8c20a.405e.338b@mx.google.com> <54BC289A.6020403@oracle.com> Message-ID: > > This works for me with latest code. > > Thanks - I will take a look anyway to see what's going on > Something funny was going on with my checkout (didn't follow the README correctly). Clean checkout and build has fixed it. Thanks. Mike. From twhitmore.nz at gmail.com Mon Jan 19 01:27:28 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Mon, 19 Jan 2015 14:27:28 +1300 Subject: __WhereRef/WhereVal peeling, #ifdef and related experiments Message-ID: Thanks Brian. As I noted, Maurizio's patch is not a real syntax.. just a hack to take experimentation further. Brian doesn't want to discuss syntaxes, nor was my intent to suggest such -- just to move the concepts away from preprocessors, #ifdef and single-dimensionality. There are many ways to slice type-casing; your example shows one (though > you need a "meet rule" when you get to the multiple-tvar case), but there > are others. My post specifically raised the possibility of multiple-tvar cases, and suggested a simple rectilinear approach in multi-dimensional space. There would need to be order of precedence in match rules, but no interaction would exist between separate clauses. I'm having difficulty Googling an exact definition for "meet rule" -- if you want to email a definition privately, I'd appreciate -- but apart from most-specific L-to-R order of precedence, no such rule would be needed. Let me know if anything is unclear. Regards, Thomas From twhitmore.nz at gmail.com Mon Jan 19 01:52:14 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Mon, 19 Jan 2015 14:52:14 +1300 Subject: __WhereRef/WhereVal peeling, #ifdef and related proposals In-Reply-To: References: Message-ID: Hi Palo, people My suggestion was really to avoid #ifdef and case(single Tvar) suggestions, and move the discussion to encompass multi-dimensional space; ie more than one Tvar. Logically at the highest level, however, I feel "portable code" that specializes correctly across domains would be better still. > it does not handle "optional" methods that should be available in just some specializations Good point. However, one fundamental I want to raise -- I don't feel that capabilities (questions/ or verbs) can correctly vary between different specializations of the same class. To me, a _class_ as written in one .java file should be the product of "logical algorithm" and a "degree of type knowledge". Capabilities -- and abstractness -- should logically be symmetrical between different specializations of such a class. I see variance in methods, only as a technical issue -- to avoid signature collisions & adapt APIs for optionality -- and I do not believe variance should exist in logical capabilities. I discussed this with Brian and feel the above paragraph to be loosely a proof. > - (big one) - supporting manual specialization on the level of code blocks within the function will make process of method specialization in JVM much more complicated. It's a tree walk & one matching rule. Generally I'd prefer portable code that specialized to all domains, over any "manual coding" approach; that's why I proposed EQ. However if we really were going to manually specialize, reuse of most code (with small type-switched blocks) would avoid repeating structurally similar method bodies with only slight adaptations. However, the sense I get is that this is all "just prototypical" to enable us to proceed with experimentation. Probably my hope & preference is, that we have *no* such manual code specialization features -- maybe just method switching, to get around the signature collisions. Here's what I'd actually want to see: - portable code across specialization domains; eg. equals, hashCode, default, etc - no manual specialization at all - methods need some kind of switching/ annotation/ layering to avoid collisions and adapt APIs for optionality. - write your own class (BitList implements List) if you want to pack boolean into bits; that's what we have classes for. So at a conceptual level, not a syntax level, that's really my preferred outcome. Just one ArrayList.java code body that specializes cleanly, elegantly & correctly to any required domain. Regards, Thomas From aka.bash0r at gmail.com Mon Jan 19 09:19:15 2015 From: aka.bash0r at gmail.com (Nils Jonsson) Date: Mon, 19 Jan 2015 10:19:15 +0100 Subject: __WhereRef/WhereVal peeling, #ifdef and related experiments In-Reply-To: References: Message-ID: There is a problem with this kind of syntax anyway. It should be a semantic failure to match on different type variables. Matching different type variables produces cases that are racing against each other. So a syntax similar to yours suggested is quite desirable to make semantics clear. But anyway, it should not be possible to match cases on an any X and any Y in the same "switch-case". By this I suggest a syntax more related to switch-case Statements and to allow matching only one type variable at a time. Then code refactoring would not be much of a concern, because the related problems would not exist. Regards, Nils On Mon, Jan 19, 2015 at 2:27 AM, Thomas W wrote: > Thanks Brian. As I noted, Maurizio's patch is not a real syntax.. just a > hack to take experimentation further. > > Brian doesn't want to discuss syntaxes, nor was my intent to suggest such > -- just to move the concepts away from preprocessors, #ifdef and > single-dimensionality. > > There are many ways to slice type-casing; your example shows one (though > > you need a "meet rule" when you get to the multiple-tvar case), but there > > are others. > > > My post specifically raised the possibility of multiple-tvar cases, and > suggested a simple rectilinear approach in multi-dimensional space. There > would need to be order of precedence in match rules, but no interaction > would exist between separate clauses. > > I'm having difficulty Googling an exact definition for "meet rule" -- if > you want to email a definition privately, I'd appreciate -- but apart from > most-specific L-to-R order of precedence, no such rule would be needed. > > Let me know if anything is unclear. > > Regards, > Thomas > From maurizio.cimadamore at oracle.com Mon Jan 19 10:57:03 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 19 Jan 2015 10:57:03 +0000 Subject: __WhereRef/WhereVal peeling, #ifdef and related experiments In-Reply-To: References: Message-ID: <54BCE2FF.8070501@oracle.com> On 19/01/15 01:27, Thomas W wrote: > I'm having difficulty Googling an exact definition for "meet rule" -- if > you want to email a definition privately, I'd appreciate -- but apart from > most-specific L-to-R order of precedence, no such rule would be needed. What Brian means by 'meet rule' is, at its heart, a rule that prevents you from defining ambiguous specializations - think of the following case (using existing syntax): class Pair { void someOperation() { __WhereRef(X) { /* ref X implementation */ } __WhereVal(Y) { /* val Y implementation */ } __WhereVal(X) __WhereRef(X) { /* val x, ref Y implementation */ } } } Now, let's say you have something like this: Pair psi = new Pair(); How should the body of 'someOperation' be specialized? Should it match the first 'ref X' block, or the second 'val Y' block? We think this should be an hard compile-time error - in fact, the current compiler already gives you two kinds of errors when writing peeled methods: * errors for ambiguous specializations (i.e. where two or more blocks match a given type-parameterization) * errors for missing specializations (i.e. if a given parameterization has _no_ matching where blocks). So, I hope you can appreciate that, despite this being essentially an hack, is also quite far from being a postprocessor-like feature; the compiler has full knowledge of such blocks and is able to reason about them in order to prove properties about the programs you are typing. Of course, eventually, all this will have to be leveraged in a more readable, Java-like way. Maurizio From asviraspossible at gmail.com Mon Jan 19 11:10:52 2015 From: asviraspossible at gmail.com (Victor Nazarov) Date: Mon, 19 Jan 2015 15:10:52 +0400 Subject: caution: bleeding-edge ahead! In-Reply-To: <54BBD167.9060505@oracle.com> References: <54B3F3A2.8050403@oracle.com> <54B464ED.7080007@oracle.com> <54b4c456.475cc20a.1a99.ffff9cf6@mx.google.com> <54B50A85.7060803@oracle.com> <54b53346.64c8c20a.405e.338b@mx.google.com> <54BBD167.9060505@oracle.com> Message-ID: I still think that class aliasing is the clearest approach proposed. And I'd like to express my support for this direction. If we can really make IntStream.class to be the same as Stream.class and new IntStream() to be the same as new Stream() We can easily and cleanly implement peeling on top of it. List example will look like this. List.java interface List extends Collection specializations __WhereRef(T) ObjectList, __WhereInt(T) IntList { /* ... */ T removeByIndex(int index); T removeByValue(T value); } ObjectList.java interface ObjectList extends List { T remove(T value); T remove(int index); default T removeByIndex(int index) { return remove(index); } default T removeByValue(T value) { return remove(index); } } IntList.java interface IntList extends List { int max(); int min(); } When interface is used in class declarations, specialized interface version is used: class OldList implements List { } Since T is defined as reference above declaration actually the same as : class OldList implements ObjectList { } This seems clean enough but it spawns some complexity that manifests itself as follows. I don't know if it this complexity worth it. class ArrayList implements List { } It seems that ArrayList should implement IntList and ArrayList should implement ObjectList ArrayList should somehow inherit set of specializations and implement different set of interfaces with different type variables. ArrayList a = new ArrayList<> (); a instanceof IntList; // ===> true ArrayList b = new ArrayList<> (); b instanceof ObjectList; // ===> true ArrayList definition should be checked to correctly implement all specialized interfaces. Since missing interface method implementation should better be a compile-time error, that run-time (specialization-time probably) one. This compile time check is only feasible if we put some constraints on specialization interfaces, like: "interface used as a specialization should provide default implementation for all of it's methods" Usage of List interface should use specialized version too: class Test { List list; void method1(T value) { // Since T is reference type, list should be instance of ObjectList interface and have remove method list.remove(value); } } Victor Nazarov On Sun, Jan 18, 2015 at 6:29 PM, Brian Goetz wrote: > If this will be achieved (and that's a big IF), then may be it can be >> pushed even further to: >> >> IntStream.class == Stream.class >> > > Note that IntStream is not simply the result of "take Stream and > replace T with int." It has methods not in Stream (e.g., sum()), and some > of the specialized methods not only specialize T -> int but things like > Supplier -> IntSupplier. > > So the above equality is even a bigger "if" than you are probably > imagining. > From gavin.bierman at oracle.com Mon Jan 19 11:15:53 2015 From: gavin.bierman at oracle.com (Gavin Bierman) Date: Mon, 19 Jan 2015 11:15:53 +0000 Subject: __WhereRef/WhereVal peeling, #ifdef and related experiments In-Reply-To: References: Message-ID: The ?meet? terminology comes from maths: given a partially ordered set S, the meet of a subset of S is the greatest lower bound (if it exists). In OO type systems, it is typically used in the context of overloading: e.g. if you have overloads whose argument types have a common lower bound (i.e. a common subtype) then you must have an overload with argument types of that common lower bound. For example, Fortress had such a restriction (see section 1): http://www.mpi-sws.org/~skilpat/papers/multipoly.pdf Hope this helps, Gavin > On 19 Jan 2015, at 01:27, Thomas W wrote: > > Thanks Brian. As I noted, Maurizio's patch is not a real syntax.. just a > hack to take experimentation further. > > Brian doesn't want to discuss syntaxes, nor was my intent to suggest such > -- just to move the concepts away from preprocessors, #ifdef and > single-dimensionality. > > There are many ways to slice type-casing; your example shows one (though >> you need a "meet rule" when you get to the multiple-tvar case), but there >> are others. > > > My post specifically raised the possibility of multiple-tvar cases, and > suggested a simple rectilinear approach in multi-dimensional space. There > would need to be order of precedence in match rules, but no interaction > would exist between separate clauses. > > I'm having difficulty Googling an exact definition for "meet rule" -- if > you want to email a definition privately, I'd appreciate -- but apart from > most-specific L-to-R order of precedence, no such rule would be needed. > > Let me know if anything is unclear. > > Regards, > Thomas From maurizio.cimadamore at oracle.com Mon Jan 19 11:26:15 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 19 Jan 2015 11:26:15 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> Message-ID: <54BCE9D7.60902@oracle.com> Hi Duncan, I've come back to this, and don't seem to be able to reproduce the problem; can you please post a bigger snippet that reproduces the issue? Thanks Maurizio On 16/01/15 15:22, MacGregor, Duncan (GE Energy Management) wrote: > Thanks for that, I can now write a sensible optional class and it behaves > as expected. > > One bug I have found is that getField is not always being correctly > specialised in all cases. Specifically a final boolean field used in an if > statement such as > > if (!isPresent) { throw... } else { return otherField; } > > Was not marked for specialisation and caused a byte code verification > error, but using a method to get the field value worked perfectly. > > Duncan. > > On 16/01/2015 12:29, "Maurizio Cimadamore" > wrote: > >> I've just pushed a compiler/specializer patch that adds support for >> T.default. >> >> This is internally handled as an 'aconst_null' which has a >> BytecodeMapping entry attached to it (in case T is an 'any' type-var). >> The specializer will then replace the aconst_null with the proper >> XYZconst_0. >> >> Again, to keep things simpler at this stage, stuff like 'int.default' >> will not work (parser will reject that). This should still be enough to >> get you going with the experiments. >> >> Happy coding! >> >> Maurizio >> >> >> On 15/01/15 11:35, Maurizio Cimadamore wrote: >>>> That is how I was intending to implement it, but if I don?t want to >>>> subclass then I need have a type something like >>>> >>>> Class Optional { >>>> final T value; >>>> final boolean hasValue; >>>> } >>>> >>>> And that means I need to set value to something, even for the empty >>>> case. >>>> Hence the need for defaults. >>> Right - I hear you. >>> >>> Maurizio >>>> Duncan. >>>> From duncan.macgregor at ge.com Mon Jan 19 13:45:53 2015 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Mon, 19 Jan 2015 13:45:53 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54BCE9D7.60902@oracle.com> References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> <54BCE9D7.60902@oracle.com> Message-ID: Here?s my little optional class and the output from javap. As you can see there is no annotation for the getField at offset 1 in public T get(), but there is for the getField() in public boolean isPresent(). Duncan. On 19/01/2015 11:26, "Maurizio Cimadamore" wrote: >Hi Duncan, >I've come back to this, and don't seem to be able to reproduce the >problem; can you please post a bigger snippet that reproduces the issue? > >Thanks >Maurizio > >On 16/01/15 15:22, MacGregor, Duncan (GE Energy Management) wrote: >> Thanks for that, I can now write a sensible optional class and it >>behaves >> as expected. >> >> One bug I have found is that getField is not always being correctly >> specialised in all cases. Specifically a final boolean field used in an >>if >> statement such as >> >> if (!isPresent) { throw... } else { return otherField; } >> >> Was not marked for specialisation and caused a byte code verification >> error, but using a method to get the field value worked perfectly. >> >> Duncan. >> >> On 16/01/2015 12:29, "Maurizio Cimadamore" >> wrote: >> >>> I've just pushed a compiler/specializer patch that adds support for >>> T.default. >>> >>> This is internally handled as an 'aconst_null' which has a >>> BytecodeMapping entry attached to it (in case T is an 'any' type-var). >>> The specializer will then replace the aconst_null with the proper >>> XYZconst_0. >>> >>> Again, to keep things simpler at this stage, stuff like 'int.default' >>> will not work (parser will reject that). This should still be enough to >>> get you going with the experiments. >>> >>> Happy coding! >>> >>> Maurizio >>> >>> >>> On 15/01/15 11:35, Maurizio Cimadamore wrote: >>>>> That is how I was intending to implement it, but if I don?t want to >>>>> subclass then I need have a type something like >>>>> >>>>> Class Optional { >>>>> final T value; >>>>> final boolean hasValue; >>>>> } >>>>> >>>>> And that means I need to set value to something, even for the empty >>>>> case. >>>>> Hence the need for defaults. >>>> Right - I hear you. >>>> >>>> Maurizio >>>>> Duncan. >>>>> > From maurizio.cimadamore at oracle.com Mon Jan 19 13:50:16 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 19 Jan 2015 13:50:16 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> <54BCE9D7.60902@oracle.com> Message-ID: <54BD0B98.8010403@oracle.com> Hi Duncan thanks for the example - the field access at offset 1 is the one for 'isPresent' which is a boolean (regardless of how the class is specialized) - why should there be an annotation on it? Maurizio On 19/01/15 13:45, MacGregor, Duncan (GE Energy Management) wrote: > Here?s my little optional class and the output from javap. As you can see > there is no annotation for the getField at offset 1 in public T get(), but > there is for the getField() in public boolean isPresent(). > > Duncan. > > On 19/01/2015 11:26, "Maurizio Cimadamore" > wrote: > >> Hi Duncan, >> I've come back to this, and don't seem to be able to reproduce the >> problem; can you please post a bigger snippet that reproduces the issue? >> >> Thanks >> Maurizio >> >> On 16/01/15 15:22, MacGregor, Duncan (GE Energy Management) wrote: >>> Thanks for that, I can now write a sensible optional class and it >>> behaves >>> as expected. >>> >>> One bug I have found is that getField is not always being correctly >>> specialised in all cases. Specifically a final boolean field used in an >>> if >>> statement such as >>> >>> if (!isPresent) { throw... } else { return otherField; } >>> >>> Was not marked for specialisation and caused a byte code verification >>> error, but using a method to get the field value worked perfectly. >>> >>> Duncan. >>> >>> On 16/01/2015 12:29, "Maurizio Cimadamore" >>> wrote: >>> >>>> I've just pushed a compiler/specializer patch that adds support for >>>> T.default. >>>> >>>> This is internally handled as an 'aconst_null' which has a >>>> BytecodeMapping entry attached to it (in case T is an 'any' type-var). >>>> The specializer will then replace the aconst_null with the proper >>>> XYZconst_0. >>>> >>>> Again, to keep things simpler at this stage, stuff like 'int.default' >>>> will not work (parser will reject that). This should still be enough to >>>> get you going with the experiments. >>>> >>>> Happy coding! >>>> >>>> Maurizio >>>> >>>> >>>> On 15/01/15 11:35, Maurizio Cimadamore wrote: >>>>>> That is how I was intending to implement it, but if I don?t want to >>>>>> subclass then I need have a type something like >>>>>> >>>>>> Class Optional { >>>>>> final T value; >>>>>> final boolean hasValue; >>>>>> } >>>>>> >>>>>> And that means I need to set value to something, even for the empty >>>>>> case. >>>>>> Hence the need for defaults. >>>>> Right - I hear you. >>>>> >>>>> Maurizio >>>>>> Duncan. >>>>>> From maurizio.cimadamore at oracle.com Mon Jan 19 13:52:01 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 19 Jan 2015 13:52:01 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54BD0B98.8010403@oracle.com> References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> <54BCE9D7.60902@oracle.com> <54BD0B98.8010403@oracle.com> Message-ID: <54BD0C01.1020003@oracle.com> On 19/01/15 13:50, Maurizio Cimadamore wrote: > Hi Duncan > thanks for the example - the field access at offset 1 is the one for > 'isPresent' which is a boolean (regardless of how the class is > specialized) - why should there be an annotation on it? Sorry, you are correct, as the field is being accessed on an Optional, you do need the attribute. Thanks for the report Maurizio > > Maurizio > > On 19/01/15 13:45, MacGregor, Duncan (GE Energy Management) wrote: >> Here?s my little optional class and the output from javap. As you can see >> there is no annotation for the getField at offset 1 in public T get(), but >> there is for the getField() in public boolean isPresent(). >> >> Duncan. >> >> On 19/01/2015 11:26, "Maurizio Cimadamore" >> wrote: >> >>> Hi Duncan, >>> I've come back to this, and don't seem to be able to reproduce the >>> problem; can you please post a bigger snippet that reproduces the issue? >>> >>> Thanks >>> Maurizio >>> >>> On 16/01/15 15:22, MacGregor, Duncan (GE Energy Management) wrote: >>>> Thanks for that, I can now write a sensible optional class and it >>>> behaves >>>> as expected. >>>> >>>> One bug I have found is that getField is not always being correctly >>>> specialised in all cases. Specifically a final boolean field used in an >>>> if >>>> statement such as >>>> >>>> if (!isPresent) { throw... } else { return otherField; } >>>> >>>> Was not marked for specialisation and caused a byte code verification >>>> error, but using a method to get the field value worked perfectly. >>>> >>>> Duncan. >>>> >>>> On 16/01/2015 12:29, "Maurizio Cimadamore" >>>> wrote: >>>> >>>>> I've just pushed a compiler/specializer patch that adds support for >>>>> T.default. >>>>> >>>>> This is internally handled as an 'aconst_null' which has a >>>>> BytecodeMapping entry attached to it (in case T is an 'any' type-var). >>>>> The specializer will then replace the aconst_null with the proper >>>>> XYZconst_0. >>>>> >>>>> Again, to keep things simpler at this stage, stuff like 'int.default' >>>>> will not work (parser will reject that). This should still be enough to >>>>> get you going with the experiments. >>>>> >>>>> Happy coding! >>>>> >>>>> Maurizio >>>>> >>>>> >>>>> On 15/01/15 11:35, Maurizio Cimadamore wrote: >>>>>>> That is how I was intending to implement it, but if I don?t want to >>>>>>> subclass then I need have a type something like >>>>>>> >>>>>>> Class Optional { >>>>>>> final T value; >>>>>>> final boolean hasValue; >>>>>>> } >>>>>>> >>>>>>> And that means I need to set value to something, even for the empty >>>>>>> case. >>>>>>> Hence the need for defaults. >>>>>> Right - I hear you. >>>>>> >>>>>> Maurizio >>>>>>> Duncan. >>>>>>> From maurizio.cimadamore at oracle.com Mon Jan 19 15:09:55 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 19 Jan 2015 15:09:55 +0000 Subject: hg: valhalla/valhalla/langtools: Bug fixes: Message-ID: <201501191509.t0JF9tkT021052@aojmv0008> Changeset: f306606b8b5e Author: mcimadamore Date: 2015-01-19 15:09 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/f306606b8b5e Bug fixes: * compiler crashes when Object methods are called on receivers whose static types is an 'any' tvar array * missing field bytecode mapping in unary conditions ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java ! test/tools/javac/valhalla/typespec/ObjectMethods.java ! test/tools/javac/valhalla/typespec/ObjectMethods.out ! test/tools/javac/valhalla/typespec/items/tests/TestCmp.java From maurizio.cimadamore at oracle.com Mon Jan 19 15:11:57 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 19 Jan 2015 15:11:57 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54BAD555.8040401@oracle.com> References: <54B5495C.6030509@oracle.com> <54BAD555.8040401@oracle.com> Message-ID: <54BD1EBD.4040807@oracle.com> On 17/01/15 21:34, Maurizio Cimadamore wrote: > Thanks for the report; I will fix this. Should be fixed by latest push. Thanks Maurizio > > Maurizio > > On 17/01/15 13:00, Richard Warburton wrote: >> Hi, >> >> Actually it appears to be that you can't call the object method on >> any expression that's more complex that referencing a variable. Eg: >> >> public static void fail(T[] values) { >> get(values).toString(); >> } >> >> public static T get(T[] values) { >> return values[0]; >> } >> >> Also NPEs. >> >> >> On Sat, Jan 17, 2015 at 12:36 PM, Richard Warburton >> > >> wrote: >> >> Hi Maurizio, >> >> I think I've found a bug here. You can't call the object methods >> on anyified arrays. Here's a minimal test case and stack trace: >> >> public static void fail(T[] values) { >> values[0].toString(); >> } >> >> Workaround: >> >> public static void fail(T[] values) { >> T value = values[0]; >> value.toString(); >> } >> >> Javac stacktrace: >> >> java.lang.NullPointerException >> at com.sun.tools.javac.comp.Attr.selectSym(Attr.java:3433) >> at com.sun.tools.javac.comp.Attr.selectSym(Attr.java:3420) >> at com.sun.tools.javac.comp.Attr.visitSelect(Attr.java:3258) >> at >> com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2082) >> at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) >> at com.sun.tools.javac.comp.Attr.visitApply(Attr.java:1800) >> at >> com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1608) >> at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) >> at com.sun.tools.javac.comp.Attr.attribExpr(Attr.java:589) >> at com.sun.tools.javac.comp.Attr.visitExec(Attr.java:1568) >> at >> com.sun.tools.javac.tree.JCTree$JCExpressionStatement.accept(JCTree.java:1426) >> at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) >> at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) >> at com.sun.tools.javac.comp.Attr.attribStats(Attr.java:625) >> at com.sun.tools.javac.comp.Attr.visitBlock(Attr.java:1112) >> at >> com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:994) >> at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) >> at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) >> at >> com.sun.tools.javac.comp.Attr.visitMethodDef(Attr.java:997) >> at >> com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:843) >> at com.sun.tools.javac.comp.Attr.attribTree(Attr.java:543) >> at com.sun.tools.javac.comp.Attr.attribStat(Attr.java:609) >> at >> com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:4404) >> at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:4312) >> at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:4234) >> at com.sun.tools.javac.comp.Attr.attrib(Attr.java:4209) >> at >> com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1230) >> at >> com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:872) >> at com.sun.tools.javac.main.Main.compile(Main.java:249) >> at com.sun.tools.javac.main.Main.compile(Main.java:140) >> at com.sun.tools.javac.Main.compile(Main.java:56) >> at com.sun.tools.javac.Main.main(Main.java:42) >> >> regards, >> >> Richard Warburton >> >> http://insightfullogic.com >> @RichardWarburto >> >> >> >> >> -- >> regards, >> >> Richard Warburton >> >> http://insightfullogic.com >> @RichardWarburto > From maurizio.cimadamore at oracle.com Mon Jan 19 15:12:21 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 19 Jan 2015 15:12:21 +0000 Subject: more band aid coming - Object methods on 'any' tvars In-Reply-To: <54BD0C01.1020003@oracle.com> References: <54B5495C.6030509@oracle.com> <54B7A124.9060700@oracle.com> <54B7A5E9.3090003@oracle.com> <54B90431.1090403@oracle.com> <54BCE9D7.60902@oracle.com> <54BD0B98.8010403@oracle.com> <54BD0C01.1020003@oracle.com> Message-ID: <54BD1ED5.8050208@oracle.com> This is now fixed with latest push. Thanks Maurizio On 19/01/15 13:52, Maurizio Cimadamore wrote: > On 19/01/15 13:50, Maurizio Cimadamore wrote: >> Hi Duncan >> thanks for the example - the field access at offset 1 is the one for >> 'isPresent' which is a boolean (regardless of how the class is >> specialized) - why should there be an annotation on it? > Sorry, you are correct, as the field is being accessed on an > Optional, you do need the attribute. > > Thanks for the report > Maurizio >> Maurizio >> >> On 19/01/15 13:45, MacGregor, Duncan (GE Energy Management) wrote: >>> Here?s my little optional class and the output from javap. As you can see >>> there is no annotation for the getField at offset 1 in public T get(), but >>> there is for the getField() in public boolean isPresent(). >>> >>> Duncan. >>> >>> On 19/01/2015 11:26, "Maurizio Cimadamore" >>> wrote: >>> >>>> Hi Duncan, >>>> I've come back to this, and don't seem to be able to reproduce the >>>> problem; can you please post a bigger snippet that reproduces the issue? >>>> >>>> Thanks >>>> Maurizio >>>> >>>> On 16/01/15 15:22, MacGregor, Duncan (GE Energy Management) wrote: >>>>> Thanks for that, I can now write a sensible optional class and it >>>>> behaves >>>>> as expected. >>>>> >>>>> One bug I have found is that getField is not always being correctly >>>>> specialised in all cases. Specifically a final boolean field used in an >>>>> if >>>>> statement such as >>>>> >>>>> if (!isPresent) { throw... } else { return otherField; } >>>>> >>>>> Was not marked for specialisation and caused a byte code verification >>>>> error, but using a method to get the field value worked perfectly. >>>>> >>>>> Duncan. >>>>> >>>>> On 16/01/2015 12:29, "Maurizio Cimadamore" >>>>> wrote: >>>>> >>>>>> I've just pushed a compiler/specializer patch that adds support for >>>>>> T.default. >>>>>> >>>>>> This is internally handled as an 'aconst_null' which has a >>>>>> BytecodeMapping entry attached to it (in case T is an 'any' type-var). >>>>>> The specializer will then replace the aconst_null with the proper >>>>>> XYZconst_0. >>>>>> >>>>>> Again, to keep things simpler at this stage, stuff like 'int.default' >>>>>> will not work (parser will reject that). This should still be enough to >>>>>> get you going with the experiments. >>>>>> >>>>>> Happy coding! >>>>>> >>>>>> Maurizio >>>>>> >>>>>> >>>>>> On 15/01/15 11:35, Maurizio Cimadamore wrote: >>>>>>>> That is how I was intending to implement it, but if I don?t want to >>>>>>>> subclass then I need have a type something like >>>>>>>> >>>>>>>> Class Optional { >>>>>>>> final T value; >>>>>>>> final boolean hasValue; >>>>>>>> } >>>>>>>> >>>>>>>> And that means I need to set value to something, even for the empty >>>>>>>> case. >>>>>>>> Hence the need for defaults. >>>>>>> Right - I hear you. >>>>>>> >>>>>>> Maurizio >>>>>>>> Duncan. >>>>>>>> From twhitmore.nz at gmail.com Mon Jan 19 23:26:05 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Tue, 20 Jan 2015 12:26:05 +1300 Subject: valhalla-dev Digest, Vol 7, Issue 66 In-Reply-To: References: Message-ID: Hi Maurizio, Gavin, Brian, Thanks for the clarification -- that was what I thought "meet rule" meant. And no, what I had suggested (in terms of multi-dimensional matching on >=1 Tvars) was specifically formulated to *exclude* the need for meet rules. I was discussing using a "precedence of matching" based L-to-R on specificity, with a sole winning clause. Multi-dimensional matching is not well-served by any loose or ambiguous matching, which is why I had specified a system not needing such. Rather than free-floating blocks which can specify any matching condition on any TVar -- logically the equivalent of #ifdef blocks with arbitrary conditions -- I proposed declaring a well-structured dimensionality around the blocks to be matched. Contained clauses would exactly match this dimensionality. A clear "most specific wins" rule, left-to-right, would apply. // implicit dimensionality: could be anything, multiple blocks can appear applicable __WhereRef(X) { /* ref X implementation */ } __WhereVal(Y) { /* val Y implementation */ } __WhereVal(X) __WhereRef(X) { /* val x, ref Y implementation */ } // explicit format -- incomplete spatial coverage of (X, Y) will not compile! specializing on (X, Y) where (X is ref, Y is any) where (X is any, Y is val) where (X is val, Y is ref) // explicit & correct -- would compile. specializing on (X, Y) where (X is ref, Y is any) where (X is val, Y is any) where (X is val, Y is ref) This identifies the error in Maurizio's ugly example very clearly -- by producing a compile-time error, as (all possible X, all possible Y) are not defined. [In the error example above, (X is val, *) would not be not fully covered.] The corollary, with such a scheme, is a requirement to check at compile-time for "complete coverage" of the output space. ie, that a method body could be produced for any possible (X, Y) specialization requested. Some interesting points from that: 1) "complete coverage" is most easily met by writing clauses which apply to (any, any..) etc for all Tvars; - these requires only one clause to fill the space & correctly fulfill the method; effectively a default. 2) the alternative of (ref | val) bifurcation, requires 2^number(Tvar) blocks to correctly fulfill a method. - (ref, ref) - (ref, val) - (val, ref) - (val, val) etc. Longwinded & prone to error, and that's for one method with only 2 Tvars. 3) just on numerical order, there appears to be a strong preference to code for 'any' as a default & specialize occasionally/ as needed. I've been pushing the desirability of coding for 'any' generally, and the desirability of generic Eq/ HashCode support specifically. But it appears that exploration of "specialized blocks" also would support that preference. > We think this should be an hard compile-time error - in fact, the current compiler already gives you two kinds of errors when writing peeled methods: > * errors for ambiguous specializations (i.e. where two or more blocks match a given type-parameterization) > * errors for missing specializations (i.e. if a given parameterization has _no_ matching where blocks). Great, I strongly agree with this. There should be as little as possible (or no) ambiguity about what code a method is going to run. I guess what I'm floating, is just a more formal declaration of which typevars a given method/ section of code is going to be specialized on -- and a strict & clear rule to select only one of these as applicable. Overall, I still feel I prefer "coding for ANY" as a language paradigm. Thanks for your feedback, Regards, Thomas From brian.goetz at oracle.com Mon Jan 19 23:33:32 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 19 Jan 2015 18:33:32 -0500 Subject: valhalla-dev Digest, Vol 7, Issue 66 In-Reply-To: References: Message-ID: <54BD944C.8080200@oracle.com> > Multi-dimensional matching is not well-served by any loose or ambiguous > matching, which is why I had specified a system not needing such. Except that's exactly how method overload selection works in Java today [1]; by contrast, there are no left-to-right precedence mechanisms. So while you might consider the left-to-right rule "better", when taken in the context of how the language already works (and how developers think), I think this has to be rejected. [1] The lack of a meet rule actually causes problems here as well, such as overloads like foo(Object, String) vs foo(String, Object) (which do you call with foo("", "")?) From twhitmore.nz at gmail.com Tue Jan 20 01:15:19 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Tue, 20 Jan 2015 14:15:19 +1300 Subject: valhalla-dev Digest, Vol 7, Issue 66 In-Reply-To: <54BD944C.8080200@oracle.com> References: <54BD944C.8080200@oracle.com> Message-ID: Hi Gavin, Maurizio, Brian, [Resent from private, thanks Maurizio:] "Meet rule" is potentially required for this idea of OO dispatch due to the "open" nature of the subtype/ method declaration universe. By contrast, I am assuming that specialization space must be completely defined & fulfilled in each & every method implementation. No interaction between separate partial spatial coverages, therefore no need for a meet rule. [Gavin said:] > The ?meet? terminology comes from maths: given a partially ordered set S, the meet of a subset of S is the greatest lower bound (if it exists). > > In OO type systems, it is typically used in the context of overloading: e.g. if you have overloads whose argument types have a common lower > bound (i.e. a common subtype) then you must have an overload with argument types of that common lower bound. > For example, Fortress: http://www.mpi-sws.org/~skilpat/papers/multipoly.pdf [Brian said:] > Except that's exactly how method overload selection works in Java today [1]; by contrast, there are no left-to-right precedence mechanisms. > So while you might consider the left-to-right rule "better", when taken in the context of how the language already works (and how developers think), > I think this has to be rejected. This may be conflating two separate issues. My focus would be on circumscribing matching into a well-defined & explicitly dimensioned space, which can be done -- making it a well-structured construct with a "closed space" which can thus be evaluated/determined independently, rather than an "open space" where interaction with other declarations/ constructs becomes necessary. The single-dimensional example would be 'switch' statements -- 'case' clauses live within their 'switch' container, and do not interact. The other issue is to use a precedence rule within a single system, rather than a "meet rule" between systems. L-to-R most specific was merely the first suggestion off the top of my head. Other matching precedence rules (such as the current Java method resolution rules) would obviously be worth considering. Regards, Thomas From peter.levart at gmail.com Tue Jan 20 10:11:56 2015 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 20 Jan 2015 11:11:56 +0100 Subject: valhalla-dev Digest, Vol 7, Issue 66 In-Reply-To: <54BD944C.8080200@oracle.com> References: <54BD944C.8080200@oracle.com> Message-ID: <54BE29EC.3080601@gmail.com> On 01/20/2015 12:33 AM, Brian Goetz wrote: >> Multi-dimensional matching is not well-served by any loose or ambiguous >> matching, which is why I had specified a system not needing such. > > Except that's exactly how method overload selection works in Java > today [1]; by contrast, there are no left-to-right precedence > mechanisms. So while you might consider the left-to-right rule > "better", when taken in the context of how the language already works > (and how developers think), I think this has to be rejected. > > > > [1] The lack of a meet rule actually causes problems here as well, > such as overloads like foo(Object, String) vs foo(String, Object) > (which do you call with foo("", "")?) Could the meet rule be added to overloads in Java? I mean only in the form of a compile-time warning of course. It seems it could be useful for API designers. Peter From forax at univ-mlv.fr Tue Jan 20 11:25:55 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 20 Jan 2015 12:25:55 +0100 Subject: valhalla-dev Digest, Vol 7, Issue 66 In-Reply-To: <54BE29EC.3080601@gmail.com> References: <54BD944C.8080200@oracle.com> <54BE29EC.3080601@gmail.com> Message-ID: <54BE3B43.4020405@univ-mlv.fr> On 01/20/2015 11:11 AM, Peter Levart wrote: > On 01/20/2015 12:33 AM, Brian Goetz wrote: >>> Multi-dimensional matching is not well-served by any loose or ambiguous >>> matching, which is why I had specified a system not needing such. >> >> Except that's exactly how method overload selection works in Java >> today [1]; by contrast, there are no left-to-right precedence >> mechanisms. So while you might consider the left-to-right rule >> "better", when taken in the context of how the language already works >> (and how developers think), I think this has to be rejected. >> >> >> >> [1] The lack of a meet rule actually causes problems here as well, >> such as overloads like foo(Object, String) vs foo(String, Object) >> (which do you call with foo("", "")?) > > Could the meet rule be added to overloads in Java? I mean only in the > form of a compile-time warning of course. It seems it could be useful > for API designers. You can not do that because by design you don't know all the subtypes of a supertype in Java, so by example, what is the meet rule for class Foo { void bar(Runnable r) { ... } void bar(Serializable s) { ... } } As an API designer, the best rule about overloads is to avoid them :) > > Peter > R?mi From brian.goetz at oracle.com Tue Jan 20 15:43:08 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 20 Jan 2015 10:43:08 -0500 Subject: valhalla-dev Digest, Vol 7, Issue 66 In-Reply-To: <54BE29EC.3080601@gmail.com> References: <54BD944C.8080200@oracle.com> <54BE29EC.3080601@gmail.com> Message-ID: <54BE778C.6020504@oracle.com> > Could the meet rule be added to overloads in Java? I mean only in the > form of a compile-time warning of course. It seems it could be useful > for API designers. Yes, this would be a useful warning to add (and we've added similar warnings in 8, when methods taking sufficiently similar functional interfaces are overloaded.) Obviously we can't make them errors because it would break existing code. From maurizio.cimadamore at oracle.com Wed Jan 21 17:01:32 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 21 Jan 2015 17:01:32 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: BMA mentions sometimes uninstantiated signatures Message-ID: <201501211701.t0LH1Wti007209@aojmv0008> Changeset: ea0693f563da Author: mcimadamore Date: 2015-01-21 17:01 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/ea0693f563da Fix: BMA mentions sometimes uninstantiated signatures ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java + test/tools/javac/valhalla/typespec/items/tests/TestInheritedAnyMembers.java From stef at epardaud.fr Wed Jan 21 18:07:28 2015 From: stef at epardaud.fr (Stef Epardaud) Date: Wed, 21 Jan 2015 19:07:28 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54B3F6A3.8050500@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> Message-ID: <20150121180727.GA10038@inforealm.org> No answer? -- St?phane Epardaud From maurizio.cimadamore at oracle.com Wed Jan 21 18:39:52 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 21 Jan 2015 18:39:52 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54B3F6A3.8050500@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> Message-ID: <54BFF278.9030500@oracle.com> Thanks, there are some useful thoughts in here, we're still thinking about directions. To some extent, this whole specialization business is like a knob - on one end of it there's a 'performance' label, on the opposite end there's a 'language uniformity' label. I think it's fair to point out (as you and others have done) that the current proposal has the knob turned all the way up to the performance end (which was kinda the point!) and that if, somehow the knob was turned back a little, a slightly different compromise could be achieved. Does different means better? Jury's still out :-) Maurizio On 12/01/15 16:30, Stephane Epardaud wrote: > On 01/12/2015 05:05 PM, Brian Goetz wrote: >> Part of what you may be missing is: code that was compiled with older >> compilers. While we have the opportunity to change the translation >> of newly compiled code, there will be existing code out there that we >> also need to be compatible with. > > Well, these would call the boxed methods, that would be _more_ > compatible since they can traverse collections of value types by > boxing them. > >> >>> To me the compromise that we can't write code that iterates say, a >>> list, without knowing at compile-time what type of value-generics it >>> may >>> hold is a compromise that will haunt Java forever. >> >> This is just incorrect. You can write a generic method: >> >> forEach(Collection c, Consumer action) > > But there's no instantiation of "any T" that would let me not know the > generic type at compile-time, no? Frameworks that need to traverse > collections without having to know what type of thing they're > traversing can't do that with value types ATM. Obviously with my > proposal they'd box, but at least they could traverse any collection. > >> >>> This clearly doesn't work with fields, but so what? Add a language >>> limitation that >>> fields of Any-generic types must be private. >> >> Which means that any class that had public fields can't ever be >> anyfied. Also, as people become more aware of the evils of >> mutability, more people are (correctly) writing classes with public >> final fields rather than intermediating with unnecessary getters. > > Well, do we have numbers on how many classes expose their state like > this via public fields? Does the Java collection API do that? Perhaps > you're right and it's unacceptable, but it's not really clear. I > suppose that one way to deal with this if this was an unacceptable > limitation would be to generate boxing specialised methods that could > provide boxed values for virtual fields backed by actual value fields. > >> (Hint: when you're inclined to say "but so what", that's the warning >> sign that you're spiraling off into wishful-thinking territory.) > > It's not about "so what" it's about the cost of the compromise. I > happen to find that tradeoff to be much much more acceptable than > treating List and List<@ValueType Date> disjoined. I do > appreciate that it's a matter of personal opinion and I may be in the > minority, but I've no evidence of that ;) From stef at epardaud.fr Thu Jan 22 08:24:56 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 09:24:56 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54BFF278.9030500@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> Message-ID: <54C0B3D8.6080902@epardaud.fr> On 01/21/2015 07:39 PM, Maurizio Cimadamore wrote: > Thanks, there are some useful thoughts in here, we're still thinking > about directions. > > To some extent, this whole specialization business is like a knob - on > one end of it there's a 'performance' label, on the opposite end > there's a 'language uniformity' label. I think it's fair to point out > (as you and others have done) that the current proposal has the knob > turned all the way up to the performance end (which was kinda the > point!) and that if, somehow the knob was turned back a little, a > slightly different compromise could be achieved. Does different means > better? Jury's still out :-) I don't think my proposal affects performance _at all_ compared to the current implementation. It does allow for auto-boxing due to equivalence of boxed/unboxed value types, which is more flexible since it allows a uniform view of "everything's an object" (at the cost of boxing) but at the same time the current performance of specialisation for unboxed value types exactly as the current implementation allows. What it does limit though is what sort of generics can be used in the type of public fields. That's more a knob of language feature, where the current implementation does not limit this but limits what we can do with value type generics, whereas my proposal makes this more regular but with the limitation of what you can declare as public fields in such an any-generic type. Note that what I'm really looking for here is a whole in my proposal. I haven't found one yet (except for the limitation on public fields) but that's probably because I'm not as used to the problem as you guys are. If there's really no hole and it's really a plausible implementation I'm willing to help test it, possibly by implementing it. I've some knowledge of the internals of javac and the bytecode, thought nothing on the VM front I'm afraid. From palo.marton at gmail.com Thu Jan 22 08:34:40 2015 From: palo.marton at gmail.com (Palo Marton) Date: Thu, 22 Jan 2015 09:34:40 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54B3F6A3.8050500@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> Message-ID: > > >> To me the compromise that we can't write code that iterates say, a >>> list, without knowing at compile-time what type of value-generics it may >>> hold is a compromise that will haunt Java forever. >>> >> >> This is just incorrect. You can write a generic method: >> >> forEach(Collection c, Consumer action) >> > > But there's no instantiation of "any T" that would let me not know the > generic type at compile-time, no? Frameworks that need to traverse > collections without having to know what type of thing they're traversing > can't do that with value types ATM. Obviously with my proposal they'd box, > but at least they could traverse any collection. In java.util.Collection you have static methods like: public static List synchronizedList(List list) public static List unmodifiableList(List list) I think that in the end there will be also method like this: public static List<__Boxed T> boxedList(List list) This will return "boxed" view of value type list. From what you write it seems that such method will do exactly what you need - you can cast value type list into boxed list and traverse it as boxed objects. From stef at epardaud.fr Thu Jan 22 09:17:33 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 10:17:33 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> Message-ID: <54C0C02D.7030003@epardaud.fr> On 01/22/2015 09:34 AM, Palo Marton wrote: > > > But there's no instantiation of "any T" that would let me not know > the generic type at compile-time, no? Frameworks that need to > traverse collections without having to know what type of thing > they're traversing can't do that with value types ATM. Obviously > with my proposal they'd box, but at least they could traverse any > collection. > > > > I think that in the end there will be also method like this: > > public static List<__Boxed T> boxedList(List list) > > This will return "boxed" view of value type list. From what you write > it seems that such method will do exactly what you need - you can cast > value type list into boxed list and traverse it as boxed objects. This would indeed allow frameworks to traverse List as List without requiring an instantiation of the value type T which they cannot possibly know about, but this is more expensive than making List<__Boxed T> === List because it requires wrappers and is frankly inelegant. From maurizio.cimadamore at oracle.com Thu Jan 22 11:18:48 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 11:18:48 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0B3D8.6080902@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> Message-ID: <54C0DC98.3020204@oracle.com> On 22/01/15 08:24, St?phane ?pardaud wrote: > Note that what I'm really looking for here is a whole in my proposal. > I haven't found one yet (except for the limitation on public fields) > but that's probably because I'm not as used to the problem as you guys > are. If there's really no hole and it's really a plausible > implementation I'm willing to help test it, possibly by implementing > it. I've some knowledge of the internals of javac and the bytecode, > thought nothing on the VM front I'm afraid. Hi Stephane, I don't think there's an 'hole' as such in your proposal - but there are 'unknowns'. Performance-wise, I think the 'risk' is that if we keep the API the way they are today (i.e. removeAll(Collection) and friends), the boxing path will pretty much be the norm. Now, in a perfect world, as value types are non-polymorphic (or their polymorphism is very restricted) and immutable, that would suggest that VM should have enough of an hint to perform boxing elimination and such, so that the cost you end up paying is negligible. How much of this is reality? As you, I'm not a VM guru - but I think it's a question worth asking. It seems a likely scenario that the JVM will do great in most cases, and have some bad performance cliffs in others - is that something we are willing to sign up for? On the language-side unknowns, how much code out there is relying on being able to access fields on Foo or raw Foo types? This is perhaps not common, but I think we need to gather data points on this i.e. by looking at existing open source projects (help welcome here!). Other possible weak points are that this doesn't necessarily address all the issue w.r.t. language uniformity - i.e. how is an ArrayList supposed to answer to a question of the kind 'is instance of List' ? I'm not denying there's something there worth exploring (as I have in fact already said), but it seems to me that, while you can go a long way with bridges, there are still questions that bridges alone simply do not have an answer for. Maurizio From stef at epardaud.fr Thu Jan 22 13:11:54 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:11:54 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0DC98.3020204@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> Message-ID: <54C0F71A.2010406@epardaud.fr> On 01/22/2015 12:18 PM, Maurizio Cimadamore wrote: > I don't think there's an 'hole' as such in your proposal - but there > are 'unknowns'. Well, that's already good news. I prefer unknowns to known holes ;) > Performance-wise, I think the 'risk' is that if we keep the API the > way they are today (i.e. removeAll(Collection) and friends), the > boxing path will pretty much be the norm. Now, in a perfect world, as > value types are non-polymorphic (or their polymorphism is very > restricted) and immutable, that would suggest that VM should have > enough of an hint to perform boxing elimination and such, so that the > cost you end up paying is negligible. How much of this is reality? As > you, I'm not a VM guru - but I think it's a question worth asking. It > seems a likely scenario that the JVM will do great in most cases, and > have some bad performance cliffs in others - is that something we are > willing to sign up for? OK, that's fair. By allowing people to use value types boxed we allow them to shoot themselves in the foot perf-wise (assuming the VM can't help), because we make value types much more accessible. This is mostly due to `any T` being opt-in rather than the new default for `T` where `T extends Object` is not specified. If we also changed `T` to mean `any T` (source only, so for newly compiled code) then all generics code would be specializable by default and that means that users of generic code will always be able to use the specialised code as long as they can instantiate the generic type argument at compile-time (as is already the limitation). If they can't, well they already have to use `Object` and boxing and can't traverse collections because they're not `List` (ATM). Making it the default has down sides, probably in larger class files, but as long as it's not the default, there will always be incompatibilities in libraries that will forget to opt-in, which will mean that either (ATM) they can't be used with containers of value types, or that the containers have to be wrapped to box the value types (unless we fix that as I suggest). IMO that's already going to cause compatibility issues and there may arise a "coding guideline" (remember them from C++ sore points?) that people will be strongly encourage to use `val T` everywhere just in case someone wants to use an unboxed value type. The rift between primitives and Object is already a famous sore point in Java (not criticising, this was a choice made a long time ago, for valid reasons, we just have to deal with it) which has been "fixed" in most non-Java JVM-languages due to popular demand. I don't think the new default with value types should be that generics don't accept value types. > On the language-side unknowns, how much code out there is relying on > being able to access fields on Foo or raw Foo types? This is > perhaps not common, but I think we need to gather data points on this > i.e. by looking at existing open source projects (help welcome here!). > Other possible weak points are that this doesn't necessarily address > all the issue w.r.t. language uniformity - i.e. how is an > ArrayList supposed to answer to a question of the kind 'is > instance of List' ? Fields that don't involve the `any T` are fine. Fields that do may have to revert to autoboxing _if_ we feel we _must_ accomodate that to autobox not just value types but their containers. `ArrayList` is special, mostly due to the fact that I'm not sure we can retrofit `Integer` to be a value type, so existing primitives may not get the `List === List` that I suggest for value types (though I'd be very interested in making this work). If we take an easier example assuming a value type named `Date`, then yes I expect that: - `List instanceof List` == `true` - `List<__Boxed Date> instanceof List` == `true` > I'm not denying there's something there worth exploring (as I have in > fact already said), but it seems to me that, while you can go a long > way with bridges, there are still questions that bridges alone simply > do not have an answer for. Thanks, and yes I agree there are still questions remaining, and more importantly an implementation lacking, but hopefully if the proposal sounds decent I can help with the proto. Thanks for your answers BTW :) From vitalyd at gmail.com Thu Jan 22 13:19:52 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 08:19:52 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0F71A.2010406@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> Message-ID: Can you expand a bit on the part where you say frameworks can't iterate a Collection without knowing the instantiation? Do you mean existing methods that take Collection won't work without change or something else? sent from my phone On Jan 22, 2015 8:12 AM, "St?phane ?pardaud" wrote: > On 01/22/2015 12:18 PM, Maurizio Cimadamore wrote: > >> I don't think there's an 'hole' as such in your proposal - but there are >> 'unknowns'. >> > > Well, that's already good news. I prefer unknowns to known holes ;) > > Performance-wise, I think the 'risk' is that if we keep the API the way >> they are today (i.e. removeAll(Collection) and friends), the boxing path >> will pretty much be the norm. Now, in a perfect world, as value types are >> non-polymorphic (or their polymorphism is very restricted) and immutable, >> that would suggest that VM should have enough of an hint to perform boxing >> elimination and such, so that the cost you end up paying is negligible. How >> much of this is reality? As you, I'm not a VM guru - but I think it's a >> question worth asking. It seems a likely scenario that the JVM will do >> great in most cases, and have some bad performance cliffs in others - is >> that something we are willing to sign up for? >> > > OK, that's fair. By allowing people to use value types boxed we allow them > to shoot themselves in the foot perf-wise (assuming the VM can't help), > because we make value types much more accessible. This is mostly due to > `any T` being opt-in rather than the new default for `T` where `T extends > Object` is not specified. If we also changed `T` to mean `any T` (source > only, so for newly compiled code) then all generics code would be > specializable by default and that means that users of generic code will > always be able to use the specialised code as long as they can instantiate > the generic type argument at compile-time (as is already the limitation). > If they can't, well they already have to use `Object` and boxing and can't > traverse collections because they're not `List` (ATM). > > Making it the default has down sides, probably in larger class files, but > as long as it's not the default, there will always be incompatibilities in > libraries that will forget to opt-in, which will mean that either (ATM) > they can't be used with containers of value types, or that the containers > have to be wrapped to box the value types (unless we fix that as I > suggest). IMO that's already going to cause compatibility issues and there > may arise a "coding guideline" (remember them from C++ sore points?) that > people will be strongly encourage to use `val T` everywhere just in case > someone wants to use an unboxed value type. > > The rift between primitives and Object is already a famous sore point in > Java (not criticising, this was a choice made a long time ago, for valid > reasons, we just have to deal with it) which has been "fixed" in most > non-Java JVM-languages due to popular demand. I don't think the new default > with value types should be that generics don't accept value types. > > On the language-side unknowns, how much code out there is relying on >> being able to access fields on Foo or raw Foo types? This is perhaps not >> common, but I think we need to gather data points on this i.e. by looking >> at existing open source projects (help welcome here!). Other possible weak >> points are that this doesn't necessarily address all the issue w.r.t. >> language uniformity - i.e. how is an ArrayList supposed to answer to a >> question of the kind 'is instance of List' ? >> > > Fields that don't involve the `any T` are fine. Fields that do may have to > revert to autoboxing _if_ we feel we _must_ accomodate that to autobox not > just value types but their containers. > > `ArrayList` is special, mostly due to the fact that I'm not sure we > can retrofit `Integer` to be a value type, so existing primitives may not > get the `List === List` that I suggest for value types > (though I'd be very interested in making this work). > > If we take an easier example assuming a value type named `Date`, then yes > I expect that: > > - `List instanceof List` == `true` > - `List<__Boxed Date> instanceof List` == `true` > > I'm not denying there's something there worth exploring (as I have in >> fact already said), but it seems to me that, while you can go a long way >> with bridges, there are still questions that bridges alone simply do not >> have an answer for. >> > > Thanks, and yes I agree there are still questions remaining, and more > importantly an implementation lacking, but hopefully if the proposal sounds > decent I can help with the proto. > > Thanks for your answers BTW :) > From maurizio.cimadamore at oracle.com Thu Jan 22 13:21:19 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 13:21:19 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0F71A.2010406@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> Message-ID: <54C0F94F.3020404@oracle.com> On 22/01/15 13:11, St?phane ?pardaud wrote: > `ArrayList` is special, mostly due to the fact that I'm not sure > we can retrofit `Integer` to be a value type, so existing primitives > may not get the `List === List` that I suggest for value > types (though I'd be very interested in making this work). Sorry - didn't want fold two problems into one - let's keep things simple and think about: Object o = new ArrayList(); where Point is a value class. Now, o instanceof ArrayList //? Or, as translated by our current prototype: o instanceof ArrayList${0=QPoint;} //yes! But how about: o instanceof ArrayList //? Maurizio From stef at epardaud.fr Thu Jan 22 13:26:33 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:26:33 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> Message-ID: <54C0FA89.2000906@epardaud.fr> On 01/22/2015 02:19 PM, Vitaly Davidovich wrote: > > Can you expand a bit on the part where you say frameworks can't > iterate a Collection without knowing the instantiation? Do you > mean existing methods that take Collection won't work without > change or something else? > > If you're writing a framework and you want to traverse instances, for example to implement JSON serialisation, you're going to have code that goes: private void traverseFields(Object instance){ for(Field f : instance.getClass().getFields()) traverseField(instance, f); } private void traverseField(Object instance, Field f){ if(f.getDeclaringClass() == Collection.class){ traverseCollection(f.get(instance)); } } private void traverseCollection(Collection collection){ for(Object val : collection) traverseFields(val); } You can see that at compile-time I don't have any idea about which value types I could encounter, so the only instantiation of `T` I will have is `Object`, never `value Date` which may pop up there, and since (unboxed) value types don't have any common superclass I can't traverse collections of them without knowing at compile-time what type they are. Which is not possible for frameworks. From maurizio.cimadamore at oracle.com Thu Jan 22 13:26:59 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 13:26:59 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> Message-ID: <54C0FAA3.900@oracle.com> On 22/01/15 13:19, Vitaly Davidovich wrote: > Can you expand a bit on the part where you say frameworks can't iterate a > Collection without knowing the instantiation? Do you mean existing > methods that take Collection won't work without change or something else? In the current prototype, such methods will work, but they will work with all reference-parameterizations - example: void m(Collection c) { ... } List ls = ... List lo = ... List li = ... List lmpv = ... m(ls); //ok m(lo); //ok m(li); //fail, List is not subtype of Collection m(lmpv); //fail, List is not subtype of Collection In other words, Collection means Collection. Therefore, primitives and values are outside the domain supported by the unbounded wildcard. To fix that, you need to change 'm' as follows: void m2(Collection c) { ... } m(ls); //ok m(lo); //ok m(li); //ok m(lmpv); //ok Is this what you were asking? Maurizio > > sent from my phone > On Jan 22, 2015 8:12 AM, "St?phane ?pardaud" wrote: > >> On 01/22/2015 12:18 PM, Maurizio Cimadamore wrote: >> >>> I don't think there's an 'hole' as such in your proposal - but there are >>> 'unknowns'. >>> >> Well, that's already good news. I prefer unknowns to known holes ;) >> >> Performance-wise, I think the 'risk' is that if we keep the API the way >>> they are today (i.e. removeAll(Collection) and friends), the boxing path >>> will pretty much be the norm. Now, in a perfect world, as value types are >>> non-polymorphic (or their polymorphism is very restricted) and immutable, >>> that would suggest that VM should have enough of an hint to perform boxing >>> elimination and such, so that the cost you end up paying is negligible. How >>> much of this is reality? As you, I'm not a VM guru - but I think it's a >>> question worth asking. It seems a likely scenario that the JVM will do >>> great in most cases, and have some bad performance cliffs in others - is >>> that something we are willing to sign up for? >>> >> OK, that's fair. By allowing people to use value types boxed we allow them >> to shoot themselves in the foot perf-wise (assuming the VM can't help), >> because we make value types much more accessible. This is mostly due to >> `any T` being opt-in rather than the new default for `T` where `T extends >> Object` is not specified. If we also changed `T` to mean `any T` (source >> only, so for newly compiled code) then all generics code would be >> specializable by default and that means that users of generic code will >> always be able to use the specialised code as long as they can instantiate >> the generic type argument at compile-time (as is already the limitation). >> If they can't, well they already have to use `Object` and boxing and can't >> traverse collections because they're not `List` (ATM). >> >> Making it the default has down sides, probably in larger class files, but >> as long as it's not the default, there will always be incompatibilities in >> libraries that will forget to opt-in, which will mean that either (ATM) >> they can't be used with containers of value types, or that the containers >> have to be wrapped to box the value types (unless we fix that as I >> suggest). IMO that's already going to cause compatibility issues and there >> may arise a "coding guideline" (remember them from C++ sore points?) that >> people will be strongly encourage to use `val T` everywhere just in case >> someone wants to use an unboxed value type. >> >> The rift between primitives and Object is already a famous sore point in >> Java (not criticising, this was a choice made a long time ago, for valid >> reasons, we just have to deal with it) which has been "fixed" in most >> non-Java JVM-languages due to popular demand. I don't think the new default >> with value types should be that generics don't accept value types. >> >> On the language-side unknowns, how much code out there is relying on >>> being able to access fields on Foo or raw Foo types? This is perhaps not >>> common, but I think we need to gather data points on this i.e. by looking >>> at existing open source projects (help welcome here!). Other possible weak >>> points are that this doesn't necessarily address all the issue w.r.t. >>> language uniformity - i.e. how is an ArrayList supposed to answer to a >>> question of the kind 'is instance of List' ? >>> >> Fields that don't involve the `any T` are fine. Fields that do may have to >> revert to autoboxing _if_ we feel we _must_ accomodate that to autobox not >> just value types but their containers. >> >> `ArrayList` is special, mostly due to the fact that I'm not sure we >> can retrofit `Integer` to be a value type, so existing primitives may not >> get the `List === List` that I suggest for value types >> (though I'd be very interested in making this work). >> >> If we take an easier example assuming a value type named `Date`, then yes >> I expect that: >> >> - `List instanceof List` == `true` >> - `List<__Boxed Date> instanceof List` == `true` >> >> I'm not denying there's something there worth exploring (as I have in >>> fact already said), but it seems to me that, while you can go a long way >>> with bridges, there are still questions that bridges alone simply do not >>> have an answer for. >>> >> Thanks, and yes I agree there are still questions remaining, and more >> importantly an implementation lacking, but hopefully if the proposal sounds >> decent I can help with the proto. >> >> Thanks for your answers BTW :) >> From vitalyd at gmail.com Thu Jan 22 13:34:04 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 08:34:04 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FA89.2000906@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> Message-ID: Ok, so the issue is really reflection/late binding here and not per say. In .NET for example, there's reflection support for generics and what one typically does in such a situation is invoke a generic method via reflection by specifying the type parameter reflectively. This all works because generics are reified in the VM, of course. Perhaps proper time to ask what the current thoughts are on reflection support for value types and any T methods. sent from my phone On Jan 22, 2015 8:26 AM, "St?phane ?pardaud" wrote: > On 01/22/2015 02:19 PM, Vitaly Davidovich wrote: > >> >> Can you expand a bit on the part where you say frameworks can't iterate a >> Collection without knowing the instantiation? Do you mean existing >> methods that take Collection won't work without change or something else? >> >> >> If you're writing a framework and you want to traverse instances, for > example to implement JSON serialisation, you're going to have code that > goes: > > private void traverseFields(Object instance){ > for(Field f : instance.getClass().getFields()) > traverseField(instance, f); > } > > private void traverseField(Object instance, Field f){ > if(f.getDeclaringClass() == Collection.class){ > traverseCollection(f.get(instance)); > } > } > > private void traverseCollection(Collection collection){ > for(Object val : collection) > traverseFields(val); > } > > You can see that at compile-time I don't have any idea about which value > types I could encounter, so the only instantiation of `T` I will have is > `Object`, never `value Date` which may pop up there, and since (unboxed) > value types don't have any common superclass I can't traverse collections > of them without knowing at compile-time what type they are. Which is not > possible for frameworks. > From vitalyd at gmail.com Thu Jan 22 13:35:57 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 08:35:57 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FAA3.900@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FAA3.900@oracle.com> Message-ID: Yes, I was asking why any T methods wouldn't work in a framework to iterate over "opaque" types, but I see Stephane's example now to which I replied. sent from my phone On Jan 22, 2015 8:27 AM, "Maurizio Cimadamore" < maurizio.cimadamore at oracle.com> wrote: > > On 22/01/15 13:19, Vitaly Davidovich wrote: > >> Can you expand a bit on the part where you say frameworks can't iterate a >> Collection without knowing the instantiation? Do you mean existing >> methods that take Collection won't work without change or something >> else? >> > In the current prototype, such methods will work, but they will work with > all reference-parameterizations - example: > > void m(Collection c) { ... } > > List ls = ... > List lo = ... > List li = ... > List lmpv = ... > > m(ls); //ok > m(lo); //ok > m(li); //fail, List is not subtype of Collection > m(lmpv); //fail, List is not subtype of Collection > > In other words, Collection means Collection. > Therefore, primitives and values are outside the domain supported by the > unbounded wildcard. > > To fix that, you need to change 'm' as follows: > > void m2(Collection c) { ... } > > m(ls); //ok > m(lo); //ok > m(li); //ok > m(lmpv); //ok > > Is this what you were asking? > > Maurizio > > >> sent from my phone >> On Jan 22, 2015 8:12 AM, "St?phane ?pardaud" wrote: >> >> On 01/22/2015 12:18 PM, Maurizio Cimadamore wrote: >>> >>> I don't think there's an 'hole' as such in your proposal - but there are >>>> 'unknowns'. >>>> >>>> Well, that's already good news. I prefer unknowns to known holes ;) >>> >>> Performance-wise, I think the 'risk' is that if we keep the API the way >>> >>>> they are today (i.e. removeAll(Collection) and friends), the boxing >>>> path >>>> will pretty much be the norm. Now, in a perfect world, as value types >>>> are >>>> non-polymorphic (or their polymorphism is very restricted) and >>>> immutable, >>>> that would suggest that VM should have enough of an hint to perform >>>> boxing >>>> elimination and such, so that the cost you end up paying is negligible. >>>> How >>>> much of this is reality? As you, I'm not a VM guru - but I think it's a >>>> question worth asking. It seems a likely scenario that the JVM will do >>>> great in most cases, and have some bad performance cliffs in others - is >>>> that something we are willing to sign up for? >>>> >>>> OK, that's fair. By allowing people to use value types boxed we allow >>> them >>> to shoot themselves in the foot perf-wise (assuming the VM can't help), >>> because we make value types much more accessible. This is mostly due to >>> `any T` being opt-in rather than the new default for `T` where `T extends >>> Object` is not specified. If we also changed `T` to mean `any T` (source >>> only, so for newly compiled code) then all generics code would be >>> specializable by default and that means that users of generic code will >>> always be able to use the specialised code as long as they can >>> instantiate >>> the generic type argument at compile-time (as is already the limitation). >>> If they can't, well they already have to use `Object` and boxing and >>> can't >>> traverse collections because they're not `List` (ATM). >>> >>> Making it the default has down sides, probably in larger class files, but >>> as long as it's not the default, there will always be incompatibilities >>> in >>> libraries that will forget to opt-in, which will mean that either (ATM) >>> they can't be used with containers of value types, or that the containers >>> have to be wrapped to box the value types (unless we fix that as I >>> suggest). IMO that's already going to cause compatibility issues and >>> there >>> may arise a "coding guideline" (remember them from C++ sore points?) that >>> people will be strongly encourage to use `val T` everywhere just in case >>> someone wants to use an unboxed value type. >>> >>> The rift between primitives and Object is already a famous sore point in >>> Java (not criticising, this was a choice made a long time ago, for valid >>> reasons, we just have to deal with it) which has been "fixed" in most >>> non-Java JVM-languages due to popular demand. I don't think the new >>> default >>> with value types should be that generics don't accept value types. >>> >>> On the language-side unknowns, how much code out there is relying on >>> >>>> being able to access fields on Foo or raw Foo types? This is perhaps >>>> not >>>> common, but I think we need to gather data points on this i.e. by >>>> looking >>>> at existing open source projects (help welcome here!). Other possible >>>> weak >>>> points are that this doesn't necessarily address all the issue w.r.t. >>>> language uniformity - i.e. how is an ArrayList supposed to answer >>>> to a >>>> question of the kind 'is instance of List' ? >>>> >>>> Fields that don't involve the `any T` are fine. Fields that do may >>> have to >>> revert to autoboxing _if_ we feel we _must_ accomodate that to autobox >>> not >>> just value types but their containers. >>> >>> `ArrayList` is special, mostly due to the fact that I'm not sure we >>> can retrofit `Integer` to be a value type, so existing primitives may not >>> get the `List === List` that I suggest for value types >>> (though I'd be very interested in making this work). >>> >>> If we take an easier example assuming a value type named `Date`, then yes >>> I expect that: >>> >>> - `List instanceof List` == `true` >>> - `List<__Boxed Date> instanceof List` == `true` >>> >>> I'm not denying there's something there worth exploring (as I have in >>> >>>> fact already said), but it seems to me that, while you can go a long way >>>> with bridges, there are still questions that bridges alone simply do not >>>> have an answer for. >>>> >>>> Thanks, and yes I agree there are still questions remaining, and more >>> importantly an implementation lacking, but hopefully if the proposal >>> sounds >>> decent I can help with the proto. >>> >>> Thanks for your answers BTW :) >>> >>> > From stef at epardaud.fr Thu Jan 22 13:36:10 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:36:10 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0F94F.3020404@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> Message-ID: <54C0FCCA.6000100@epardaud.fr> On 01/22/2015 02:21 PM, Maurizio Cimadamore wrote: > Object o = new ArrayList(); > > where Point is a value class. > > Now, > > o instanceof ArrayList //? > > Or, as translated by our current prototype: > > o instanceof ArrayList${0=QPoint;} //yes! And that's good! > But how about: > > o instanceof ArrayList //? Yes, because it would be also an `instanceof ArrayList` with my proposal. I know at runtime we end up with two separate classes for `ArrayList` and `ArrayList`, but if `ArrayList` has all the bridge methods to satisfy `ArrayList` we can make `instanceof ArrayList` return true for any instance of `ArrayList` regardless of value type arguments. So yeah, I agree it's not entirely limited to compiler support and we need some support from the VM, but that's already the case since we expect specialisation to be handled by the VM to generate pseudo-classes. BTW, aren't you afraid of the proliferation of specialised classes at runtime for the VM? I know PermGen moved away, but still, classes are expensive to load? From stef at epardaud.fr Thu Jan 22 13:38:41 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:38:41 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FAA3.900@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FAA3.900@oracle.com> Message-ID: <54C0FD61.1050807@epardaud.fr> On 01/22/2015 02:26 PM, Maurizio Cimadamore wrote: > > In other words, Collection means Collection. > Therefore, primitives and values are outside the domain supported by > the unbounded wildcard. > > To fix that, you need to change 'm' as follows: > > void m2(Collection c) { ... } > > m(ls); //ok > m(lo); //ok > m(li); //ok > m(lmpv); //ok > > Is this what you were asking? Except you're again relying on compile-time available instantiations of `T`. Frameworks don't have those, they work by discovery as in my example. From vitalyd at gmail.com Thu Jan 22 13:41:18 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 08:41:18 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FD61.1050807@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FAA3.900@oracle.com> <54C0FD61.1050807@epardaud.fr> Message-ID: So this type of use case would be a problem if generics were there from day 1 but ? didn't exist. What you want here is be able to invoke generic methods at runtime /late bound, and not necessarily coercing everything to extend ?. sent from my phone On Jan 22, 2015 8:39 AM, "St?phane ?pardaud" wrote: > On 01/22/2015 02:26 PM, Maurizio Cimadamore wrote: > >> >> In other words, Collection means Collection. >> Therefore, primitives and values are outside the domain supported by the >> unbounded wildcard. >> >> To fix that, you need to change 'm' as follows: >> >> void m2(Collection c) { ... } >> >> m(ls); //ok >> m(lo); //ok >> m(li); //ok >> m(lmpv); //ok >> >> Is this what you were asking? >> > > Except you're again relying on compile-time available instantiations of > `T`. Frameworks don't have those, they work by discovery as in my example. > From stef at epardaud.fr Thu Jan 22 13:42:01 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:42:01 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> Message-ID: <54C0FE29.3010009@epardaud.fr> On 01/22/2015 02:34 PM, Vitaly Davidovich wrote: > > Ok, so the issue is really reflection/late binding here and not T> per say. In .NET for example, there's reflection support for > generics and what one typically does in such a situation is invoke a > generic method via reflection by specifying the type parameter > reflectively. This all works because generics are reified in the VM, > of course. > Well, yes and no: it's related to not being able to instantiate `any T` when you don't know `T` at compile-time. Currently you can handle every type in Java without having to know it by handling `Object` and every primitive. We can't expect people to handle every value type when it's an open-ended set, and there's no common superclass of value types, and they're not `Object` (unless they're autoboxed which doesn't happen when they're in containers like collections). So we can't handle them unless you can instantiate their type argument at compile-time. Using `any T` on methods only defers the problem but doesn't solve it for blind usages. From maurizio.cimadamore at oracle.com Thu Jan 22 13:42:16 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 13:42:16 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FCCA.6000100@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> Message-ID: <54C0FE38.7090905@oracle.com> On 22/01/15 13:36, St?phane ?pardaud wrote: > On 01/22/2015 02:21 PM, Maurizio Cimadamore wrote: >> Object o = new ArrayList(); >> >> where Point is a value class. >> >> Now, >> >> o instanceof ArrayList //? >> >> Or, as translated by our current prototype: >> >> o instanceof ArrayList${0=QPoint;} //yes! > And that's good! >> But how about: >> >> o instanceof ArrayList //? > Yes, because it would be also an `instanceof ArrayList` with > my proposal. I know at runtime we end up with two separate classes for > `ArrayList` and `ArrayList`, but if `ArrayList` > has all the bridge methods to satisfy `ArrayList` we can make > `instanceof ArrayList` return true for any instance of `ArrayList` > regardless of value type arguments. So yeah, I agree it's not entirely > limited to compiler support and we need some support from the VM, but > that's already the case since we expect specialisation to be handled > by the VM to generate pseudo-classes. That's mostly where I was trying to get at - bridges are a part of your approach - but it seems like some other part is missing in order to make the approach consistent. > > BTW, aren't you afraid of the proliferation of specialised classes at > runtime for the VM? I know PermGen moved away, but still, classes are > expensive to load? As you might know, I did some work on reification in my early days [1] - among the things I found when implementing a JVM with support for reification was that in real-world cases, the number of different instantiation of the same generic class tends to be quite low. Of course this is another area where we need more evidence. [1] - https://apice.unibo.it/xwiki/bin/view/Theses/MaurizioCimadamorePhd Maurizio From stef at epardaud.fr Thu Jan 22 13:43:59 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:43:59 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FAA3.900@oracle.com> <54C0FD61.1050807@epardaud.fr> Message-ID: <54C0FE9F.9000306@epardaud.fr> On 01/22/2015 02:41 PM, Vitaly Davidovich wrote: > > So this type of use case would be a problem if generics were there > from day 1 but ? didn't exist. What you want here is be able to > invoke generic methods at runtime /late bound, and not necessarily > coercing everything to extend ?. > Well, I don't think we can invoke generic methods of value types unless we can either know the value type statically or unless we can box them. From vitalyd at gmail.com Thu Jan 22 13:46:18 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 08:46:18 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FE29.3010009@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> <54C0FE29.3010009@epardaud.fr> Message-ID: Yes, but I'm not proposing to handle every value type (which is impossible anyway since you don't know the set). Instead, what would be nice is to allow calling methods at runtime by providing the T parameter at that point, after getting it via reflection. This requires some form of reification/VM support though. sent from my phone On Jan 22, 2015 8:42 AM, "St?phane ?pardaud" wrote: > On 01/22/2015 02:34 PM, Vitaly Davidovich wrote: > >> >> Ok, so the issue is really reflection/late binding here and not >> per say. In .NET for example, there's reflection support for generics and >> what one typically does in such a situation is invoke a generic method via >> reflection by specifying the type parameter reflectively. This all works >> because generics are reified in the VM, of course. >> >> Well, yes and no: it's related to not being able to instantiate `any T` > when you don't know `T` at compile-time. Currently you can handle every > type in Java without having to know it by handling `Object` and every > primitive. We can't expect people to handle every value type when it's an > open-ended set, and there's no common superclass of value types, and > they're not `Object` (unless they're autoboxed which doesn't happen when > they're in containers like collections). So we can't handle them unless you > can instantiate their type argument at compile-time. Using `any T` on > methods only defers the problem but doesn't solve it for blind usages. > From stef at epardaud.fr Thu Jan 22 13:47:22 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:47:22 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FE38.7090905@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> Message-ID: <54C0FF6A.3080407@epardaud.fr> On 01/22/2015 02:42 PM, Maurizio Cimadamore wrote: > That's mostly where I was trying to get at - bridges are a part of > your approach - but it seems like some other part is missing in order > to make the approach consistent. Well, sure, it requires _some_ support for `instanceof` and the reflection API to handle this. But the reflection API will already have to be tweaked for value types, so it's not like it's out of the question. >> BTW, aren't you afraid of the proliferation of specialised classes at >> runtime for the VM? I know PermGen moved away, but still, classes are >> expensive to load? > As you might know, I did some work on reification in my early days [1] > - among the things I found when implementing a JVM with support for > reification was that in real-world cases, the number of different > instantiation of the same generic class tends to be quite low. Of > course this is another area where we need more evidence. I didn't know, but I would lean towards agreeing with you. At least in the end usage where you will only have N sorts of `ArrayList`, but that multiplies because each is also a `List` and an `Iterable` and a `Collection` and will access `Stream`? so even though you will only instantiate N `T` values, you will get more classes than just N. From vitalyd at gmail.com Thu Jan 22 13:48:46 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 08:48:46 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FE9F.9000306@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FAA3.900@oracle.com> <54C0FD61.1050807@epardaud.fr> <54C0FE9F.9000306@epardaud.fr> Message-ID: Can't invoke them statically, yes, but could reflectively. Again, .NET allows this and that's how people bridge late bound code with generics. sent from my phone On Jan 22, 2015 8:44 AM, "St?phane ?pardaud" wrote: > On 01/22/2015 02:41 PM, Vitaly Davidovich wrote: > >> >> So this type of use case would be a problem if generics were there from >> day 1 but ? didn't exist. What you want here is be able to invoke generic >> methods at runtime /late bound, and not necessarily coercing everything to >> extend ?. >> >> Well, I don't think we can invoke generic methods of value types unless > we can either know the value type statically or unless we can box them. > From maurizio.cimadamore at oracle.com Thu Jan 22 13:49:15 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 13:49:15 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FF6A.3080407@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> Message-ID: <54C0FFDB.6010205@oracle.com> On 22/01/15 13:47, St?phane ?pardaud wrote: > On 01/22/2015 02:42 PM, Maurizio Cimadamore wrote: >> That's mostly where I was trying to get at - bridges are a part of >> your approach - but it seems like some other part is missing in order >> to make the approach consistent. > > Well, sure, it requires _some_ support for `instanceof` and the > reflection API to handle this. But the reflection API will already > have to be tweaked for value types, so it's not like it's out of the > question. And checkcast. And arraystore (because of dynamic covariance checks). ... Maurizio > >>> BTW, aren't you afraid of the proliferation of specialised classes >>> at runtime for the VM? I know PermGen moved away, but still, classes >>> are expensive to load? >> As you might know, I did some work on reification in my early days >> [1] - among the things I found when implementing a JVM with support >> for reification was that in real-world cases, the number of different >> instantiation of the same generic class tends to be quite low. Of >> course this is another area where we need more evidence. > > I didn't know, but I would lean towards agreeing with you. At least in > the end usage where you will only have N sorts of `ArrayList`, but > that multiplies because each is also a `List` and an `Iterable` > and a `Collection` and will access `Stream`? so even though you > will only instantiate N `T` values, you will get more classes than > just N. From stef at epardaud.fr Thu Jan 22 13:51:49 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:51:49 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> <54C0FE29.3010009@epardaud.fr> Message-ID: <54C10075.1060107@epardaud.fr> On 01/22/2015 02:46 PM, Vitaly Davidovich wrote: > > Yes, but I'm not proposing to handle every value type (which is > impossible anyway since you don't know the set). Instead, what would > be nice is to allow calling methods at runtime by providing > the T parameter at that point, after getting it via reflection. This > requires some form of reification/VM support though. > > Well, if I understand it correctly, value type arguments will be reified necessarily because they end up specialising the type they parameterise. So given a `List` instance you would get a `List${0=QDate;}` class, from which it _must surely_ be possible to extract the `Date` type argument at runtime via reflection. So while that information is lost in an instance of `List` (erasure), it should not be erased from instances of generic types whose type arguments are value types. So yeah, that's an option, but it would _still_ require autoboxing of any value type you put in or extract via reflection. And be less elegant because it forces you to use reflection for every method invocation of `Collection`. From vitalyd at gmail.com Thu Jan 22 13:53:05 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 08:53:05 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FE38.7090905@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> Message-ID: Maurizio, Have you seen this blog post before: http://joeduffyblog.com/2011/10/23/on-generics-and-some-of-the-associated-overheads/ This is from .NET perspective but I imagine java would be similar. sent from my phone On Jan 22, 2015 8:42 AM, "Maurizio Cimadamore" < maurizio.cimadamore at oracle.com> wrote: > > On 22/01/15 13:36, St?phane ?pardaud wrote: > >> On 01/22/2015 02:21 PM, Maurizio Cimadamore wrote: >> >>> Object o = new ArrayList(); >>> >>> where Point is a value class. >>> >>> Now, >>> >>> o instanceof ArrayList //? >>> >>> Or, as translated by our current prototype: >>> >>> o instanceof ArrayList${0=QPoint;} //yes! >>> >> And that's good! >> >>> But how about: >>> >>> o instanceof ArrayList //? >>> >> Yes, because it would be also an `instanceof ArrayList` with my >> proposal. I know at runtime we end up with two separate classes for >> `ArrayList` and `ArrayList`, but if `ArrayList` has >> all the bridge methods to satisfy `ArrayList` we can make >> `instanceof ArrayList` return true for any instance of `ArrayList` >> regardless of value type arguments. So yeah, I agree it's not entirely >> limited to compiler support and we need some support from the VM, but >> that's already the case since we expect specialisation to be handled by the >> VM to generate pseudo-classes. >> > That's mostly where I was trying to get at - bridges are a part of your > approach - but it seems like some other part is missing in order to make > the approach consistent. > >> >> BTW, aren't you afraid of the proliferation of specialised classes at >> runtime for the VM? I know PermGen moved away, but still, classes are >> expensive to load? >> > As you might know, I did some work on reification in my early days [1] - > among the things I found when implementing a JVM with support for > reification was that in real-world cases, the number of different > instantiation of the same generic class tends to be quite low. Of course > this is another area where we need more evidence. > > [1] - https://apice.unibo.it/xwiki/bin/view/Theses/MaurizioCimadamorePhd > > Maurizio > > From stef at epardaud.fr Thu Jan 22 13:53:22 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:53:22 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FFDB.6010205@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> Message-ID: <54C100D2.9000506@epardaud.fr> On 01/22/2015 02:49 PM, Maurizio Cimadamore wrote: > > On 22/01/15 13:47, St?phane ?pardaud wrote: >> On 01/22/2015 02:42 PM, Maurizio Cimadamore wrote: >>> That's mostly where I was trying to get at - bridges are a part of >>> your approach - but it seems like some other part is missing in >>> order to make the approach consistent. >> >> Well, sure, it requires _some_ support for `instanceof` and the >> reflection API to handle this. But the reflection API will already >> have to be tweaked for value types, so it's not like it's out of the >> question. > And checkcast. > > And arraystore (because of dynamic covariance checks). OK, that's true. I already knew I wasn't aware of all the consequences of this approach, but we're making progress finding them :) Any others you can think of? From vitalyd at gmail.com Thu Jan 22 13:56:08 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 08:56:08 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C10075.1060107@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> <54C0FE29.3010009@epardaud.fr> <54C10075.1060107@epardaud.fr> Message-ID: You don't need to box anything if the generic method takes Collection - you just call it once you've obtained the type parameter. If you've already boxed the value type you may as well call methods taking Object. The side benefit to this is you don't need to box the items in the collection. sent from my phone On Jan 22, 2015 8:52 AM, "St?phane ?pardaud" wrote: > On 01/22/2015 02:46 PM, Vitaly Davidovich wrote: > >> >> Yes, but I'm not proposing to handle every value type (which is >> impossible anyway since you don't know the set). Instead, what would be >> nice is to allow calling methods at runtime by providing the T >> parameter at that point, after getting it via reflection. This requires >> some form of reification/VM support though. >> >> >> Well, if I understand it correctly, value type arguments will be reified > necessarily because they end up specialising the type they parameterise. So > given a `List` instance you would get a `List${0=QDate;}` class, from > which it _must surely_ be possible to extract the `Date` type argument at > runtime via reflection. So while that information is lost in an instance of > `List` (erasure), it should not be erased from instances of generic > types whose type arguments are value types. > > So yeah, that's an option, but it would _still_ require autoboxing of any > value type you put in or extract via reflection. And be less elegant > because it forces you to use reflection for every method invocation of > `Collection`. > From maurizio.cimadamore at oracle.com Thu Jan 22 13:56:35 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 13:56:35 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C100D2.9000506@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> Message-ID: <54C10193.40900@oracle.com> On 22/01/15 13:53, St?phane ?pardaud wrote: > On 01/22/2015 02:49 PM, Maurizio Cimadamore wrote: >> >> On 22/01/15 13:47, St?phane ?pardaud wrote: >>> On 01/22/2015 02:42 PM, Maurizio Cimadamore wrote: >>>> That's mostly where I was trying to get at - bridges are a part of >>>> your approach - but it seems like some other part is missing in >>>> order to make the approach consistent. >>> >>> Well, sure, it requires _some_ support for `instanceof` and the >>> reflection API to handle this. But the reflection API will already >>> have to be tweaked for value types, so it's not like it's out of the >>> question. >> And checkcast. >> >> And arraystore (because of dynamic covariance checks). > OK, that's true. I already knew I wasn't aware of all the consequences > of this approach, but we're making progress finding them :) Any others > you can think of? You asked for 'holes' - I gave you some :-) Maurizio From stef at epardaud.fr Thu Jan 22 13:57:21 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:57:21 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> <54C0FE29.3010009@epardaud.fr> <54C10075.1060107@epardaud.fr> Message-ID: <54C101C1.6030007@epardaud.fr> On 01/22/2015 02:56 PM, Vitaly Davidovich wrote: > > You don't need to box anything if the generic method takes Collection > - you just call it once you've obtained the type parameter. > If you've already boxed the value type you may as well call methods > taking Object. The side benefit to this is you don't need to box the > items in the collection. > > Well you will still need to box every item out of the collection because a `val Date` is not an Object unless boxed. And your traversal still works with Objects. From vitalyd at gmail.com Thu Jan 22 13:57:56 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 08:57:56 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C10075.1060107@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> <54C0FE29.3010009@epardaud.fr> <54C10075.1060107@epardaud.fr> Message-ID: As for elegance, IMHO, these types of frameworks are already neck deep in reflection. Vast majority of code would use generics with static binding, I imagine. sent from my phone On Jan 22, 2015 8:52 AM, "St?phane ?pardaud" wrote: > On 01/22/2015 02:46 PM, Vitaly Davidovich wrote: > >> >> Yes, but I'm not proposing to handle every value type (which is >> impossible anyway since you don't know the set). Instead, what would be >> nice is to allow calling methods at runtime by providing the T >> parameter at that point, after getting it via reflection. This requires >> some form of reification/VM support though. >> >> >> Well, if I understand it correctly, value type arguments will be reified > necessarily because they end up specialising the type they parameterise. So > given a `List` instance you would get a `List${0=QDate;}` class, from > which it _must surely_ be possible to extract the `Date` type argument at > runtime via reflection. So while that information is lost in an instance of > `List` (erasure), it should not be erased from instances of generic > types whose type arguments are value types. > > So yeah, that's an option, but it would _still_ require autoboxing of any > value type you put in or extract via reflection. And be less elegant > because it forces you to use reflection for every method invocation of > `Collection`. > From stef at epardaud.fr Thu Jan 22 13:58:51 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 14:58:51 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C10193.40900@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C10193.40900@oracle.com> Message-ID: <54C1021B.3090605@epardaud.fr> On 01/22/2015 02:56 PM, Maurizio Cimadamore wrote: > > You asked for 'holes' - I gave you some :-) And I'm glad you did, this is constructive! But these aren't holes yet, they're things to implement ;) I was genuinely asking for more of such information: that's useful. From scolebourne at joda.org Thu Jan 22 13:58:54 2015 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 22 Jan 2015 13:58:54 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FA89.2000906@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> Message-ID: On 22 January 2015 at 13:26, St?phane ?pardaud wrote: > You can see that at compile-time I don't have any idea about which value > types I could encounter, so the only instantiation of `T` I will have is > `Object`, never `value Date` which may pop up there, and since (unboxed) > value types don't have any common superclass I can't traverse collections of > them without knowing at compile-time what type they are. Which is not > possible for frameworks. Stephane is making a key point that is critical to the success of value types as a project. Almost every piece of framework logic that navigates around an arbitrary object structure will have code that queries if an object is a Collection/Map and performs specific processing, for example: https://github.com/JodaOrg/joda-beans/blob/master/src/main/java/org/joda/beans/ser/SerIteratorFactory.java#L77 https://github.com/JodaOrg/joda-beans/blob/master/src/main/java/org/joda/beans/ser/xml/JodaBeanXmlWriter.java#L176 I'm sure its easy to find other examples in similar projects. Note that this is a key problem to the Beans v2.0 MetaModel stuff I've been looking at. It seems necessary to make List is in some way related to List, just like a value can be boxed to an object. Doing so should take existing libraries that accept Collection and loop over it, and allow them to accept Collection. It seems to me that anything else is token backward compatibility, rather than real backwards compatibility. As indicated in other messages, you don't always know what T is at compile time. Stephen From maurizio.cimadamore at oracle.com Thu Jan 22 14:02:46 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 14:02:46 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> Message-ID: <54C10306.1030209@oracle.com> On 22/01/15 13:53, Vitaly Davidovich wrote: > > Maurizio, > > Have you seen this blog post before: > http://joeduffyblog.com/2011/10/23/on-generics-and-some-of-the-associated-overheads/ > Haven't seen that post exactly - but was generally aware of the problems described there (thx for sharing!). One key difference between the reification approach I implemented and the C# one was that I was mainly using an homogenoeus translation - i.e. only one class and a type token attached to the object header. Now, constructing type tokens is rather quick, i.e. no need to really load a class - so in my approach the proliferation of generic types was kind of less of an issue. I remember that when running javac codebase against the reified VM I was getting more or less 400 distinct type tokens, which is not a lot considering the size of the javac codebase. If those numbers are still true today, then I think we should be fine - note that javac codebase already an extreme case of a biggie codebase that is heavily generified. Another interesting experiment which didn't exist back then would be the Stream API, which is also heavily generified (and performance critical). Maurizio > > This is from .NET perspective but I imagine java would be similar. > > sent from my phone > > On Jan 22, 2015 8:42 AM, "Maurizio Cimadamore" > > wrote: > > > On 22/01/15 13:36, St?phane ?pardaud wrote: > > On 01/22/2015 02:21 PM, Maurizio Cimadamore wrote: > > Object o = new ArrayList(); > > where Point is a value class. > > Now, > > o instanceof ArrayList //? > > Or, as translated by our current prototype: > > o instanceof ArrayList${0=QPoint;} //yes! > > And that's good! > > But how about: > > o instanceof ArrayList //? > > Yes, because it would be also an `instanceof > ArrayList` with my proposal. I know at runtime we end > up with two separate classes for `ArrayList` and > `ArrayList`, but if `ArrayList` has all the > bridge methods to satisfy `ArrayList` we can make > `instanceof ArrayList` return true for any instance of > `ArrayList` regardless of value type arguments. So yeah, I > agree it's not entirely limited to compiler support and we > need some support from the VM, but that's already the case > since we expect specialisation to be handled by the VM to > generate pseudo-classes. > > That's mostly where I was trying to get at - bridges are a part of > your approach - but it seems like some other part is missing in > order to make the approach consistent. > > > BTW, aren't you afraid of the proliferation of specialised > classes at runtime for the VM? I know PermGen moved away, but > still, classes are expensive to load? > > As you might know, I did some work on reification in my early days > [1] - among the things I found when implementing a JVM with > support for reification was that in real-world cases, the number > of different instantiation of the same generic class tends to be > quite low. Of course this is another area where we need more evidence. > > [1] - > https://apice.unibo.it/xwiki/bin/view/Theses/MaurizioCimadamorePhd > > Maurizio > From vitalyd at gmail.com Thu Jan 22 14:05:52 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 09:05:52 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> Message-ID: But List is not and cannot be List - I'm guessing you mean List. sent from my phone On Jan 22, 2015 8:59 AM, "Stephen Colebourne" wrote: > On 22 January 2015 at 13:26, St?phane ?pardaud wrote: > > You can see that at compile-time I don't have any idea about which value > > types I could encounter, so the only instantiation of `T` I will have is > > `Object`, never `value Date` which may pop up there, and since (unboxed) > > value types don't have any common superclass I can't traverse > collections of > > them without knowing at compile-time what type they are. Which is not > > possible for frameworks. > > Stephane is making a key point that is critical to the success of > value types as a project. Almost every piece of framework logic that > navigates around an arbitrary object structure will have code that > queries if an object is a Collection/Map and performs specific > processing, for example: > > https://github.com/JodaOrg/joda-beans/blob/master/src/main/java/org/joda/beans/ser/SerIteratorFactory.java#L77 > > https://github.com/JodaOrg/joda-beans/blob/master/src/main/java/org/joda/beans/ser/xml/JodaBeanXmlWriter.java#L176 > > I'm sure its easy to find other examples in similar projects. Note > that this is a key problem to the Beans v2.0 MetaModel stuff I've been > looking at. > > It seems necessary to make List is in some way related to > List, just like a value can be boxed to an object. Doing so > should take existing libraries that accept Collection and loop over > it, and allow them to accept Collection. It seems to me that > anything else is token backward compatibility, rather than real > backwards compatibility. As indicated in other messages, you don't > always know what T is at compile time. > > Stephen > From palo.marton at gmail.com Thu Jan 22 14:08:54 2015 From: palo.marton at gmail.com (Palo Marton) Date: Thu, 22 Jan 2015 15:08:54 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C100D2.9000506@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> Message-ID: What Stephane probably needs is something like this: I java.lang.reflect.Proxy.specializedToBoxed(Class interface, Object specialized_object); Which will be just generalized form of: public static List<__Boxed T> java.util.Collections.boxedList(List list) (But I think that it is very premature to discuss reflection of specialized generics). On Thu, Jan 22, 2015 at 2:53 PM, St?phane ?pardaud wrote: > On 01/22/2015 02:49 PM, Maurizio Cimadamore wrote: > >> >> On 22/01/15 13:47, St?phane ?pardaud wrote: >> >>> On 01/22/2015 02:42 PM, Maurizio Cimadamore wrote: >>> >>>> That's mostly where I was trying to get at - bridges are a part of your >>>> approach - but it seems like some other part is missing in order to make >>>> the approach consistent. >>>> >>> >>> Well, sure, it requires _some_ support for `instanceof` and the >>> reflection API to handle this. But the reflection API will already have to >>> be tweaked for value types, so it's not like it's out of the question. >>> >> And checkcast. >> >> And arraystore (because of dynamic covariance checks). >> > OK, that's true. I already knew I wasn't aware of all the consequences of > this approach, but we're making progress finding them :) Any others you can > think of? > From maurizio.cimadamore at oracle.com Thu Jan 22 14:09:25 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 14:09:25 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C1021B.3090605@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C10193.40900@oracle.com> <54C1021B.3090605@epardaud.fr> Message-ID: <54C10495.9020309@oracle.com> On 22/01/15 13:58, St?phane ?pardaud wrote: > On 01/22/2015 02:56 PM, Maurizio Cimadamore wrote: >> >> You asked for 'holes' - I gave you some :-) > And I'm glad you did, this is constructive! But these aren't holes > yet, they're things to implement ;) I was genuinely asking for more of > such information: that's useful. Well, to some extent, everything falls in the category of 'things to implement'. To be honest, it's not very clear from your emails how you plan to tackle those issues, and I don't mean in terms of C++ Hotspot code, but merely in terms of how things should be wired up. Saying there should be _some kind_ of instanceof support sounds a bit vague-ish? Backing up a bit, there are two dimensions to the problem you are trying to solve; one is to have specialized(A) and erased(A) to respond to the same sets of messages; I think your strategy there ticks the boxes; yes, it leaves field access a bit behind, but let's say that's collateral damage (although you need to prove that this won't come up so frequently that it would represent a real source compatibility threat). The second dimension is to have specialized(A) 'is-a' erased(A) - and this is the part of the story that looks less clear. Maurizio From stef at epardaud.fr Thu Jan 22 14:11:13 2015 From: stef at epardaud.fr (Stef Epardaud) Date: Thu, 22 Jan 2015 15:11:13 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> Message-ID: <20150122141113.GA12874@inforealm.org> It is with my proposal -- St?phane Epardaud From maurizio.cimadamore at oracle.com Thu Jan 22 14:14:13 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 14:14:13 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> Message-ID: <54C105B5.6050708@oracle.com> I think both Stephane (and you) and Vitaly have a point; such dynamic frameworks have two alternatives: * keep working the way they do now - which implies List is related to List/List * let them be generified, and then use reflection (on steroids) to let the right thing happen at runtime Maurizio On 22/01/15 13:58, Stephen Colebourne wrote: > On 22 January 2015 at 13:26, St?phane ?pardaud wrote: >> You can see that at compile-time I don't have any idea about which value >> types I could encounter, so the only instantiation of `T` I will have is >> `Object`, never `value Date` which may pop up there, and since (unboxed) >> value types don't have any common superclass I can't traverse collections of >> them without knowing at compile-time what type they are. Which is not >> possible for frameworks. > Stephane is making a key point that is critical to the success of > value types as a project. Almost every piece of framework logic that > navigates around an arbitrary object structure will have code that > queries if an object is a Collection/Map and performs specific > processing, for example: > https://github.com/JodaOrg/joda-beans/blob/master/src/main/java/org/joda/beans/ser/SerIteratorFactory.java#L77 > https://github.com/JodaOrg/joda-beans/blob/master/src/main/java/org/joda/beans/ser/xml/JodaBeanXmlWriter.java#L176 > > I'm sure its easy to find other examples in similar projects. Note > that this is a key problem to the Beans v2.0 MetaModel stuff I've been > looking at. > > It seems necessary to make List is in some way related to > List, just like a value can be boxed to an object. Doing so > should take existing libraries that accept Collection and loop over > it, and allow them to accept Collection. It seems to me that > anything else is token backward compatibility, rather than real > backwards compatibility. As indicated in other messages, you don't > always know what T is at compile time. > > Stephen From stef at epardaud.fr Thu Jan 22 14:16:46 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 15:16:46 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C10495.9020309@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C10193.40900@oracle.com> <54C1021B.3090605@epardaud.fr> <54C10495.9020309@oracle.com> Message-ID: <54C1064E.4020908@epardaud.fr> On 01/22/2015 03:09 PM, Maurizio Cimadamore wrote: > Well, to some extent, everything falls in the category of 'things to > implement'. To be honest, it's not very clear from your emails how you > plan to tackle those issues, and I don't mean in terms of C++ Hotspot > code, but merely in terms of how things should be wired up. Saying > there should be _some kind_ of instanceof support sounds a bit vague-ish? Well, I assume there's a relation point back from specialised ArrayList classes to the ArrayList class, no? > Backing up a bit, there are two dimensions to the problem you are > trying to solve; one is to have specialized(A) and erased(A) to > respond to the same sets of messages; I think your strategy there > ticks the boxes; yes, it leaves field access a bit behind, but let's > say that's collateral damage (although you need to prove that this > won't come up so frequently that it would represent a real source > compatibility threat). > > The second dimension is to have specialized(A) 'is-a' erased(A) - and > this is the part of the story that looks less clear. And that's where you may be aware of technical issues that would make this hard that I'm not aware of yet. If it's just about being able to find back erased(A) from and instance of specialized(A) then it doesn't sound too hard, I suppose that information will be available in the runtime class for specialized(A), so `checkcast` and `instanceof` and friends can use that for checks. Pretty much the same way we'd allow these types to be interchangeable at compile-time from javac. From stef at epardaud.fr Thu Jan 22 14:18:48 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 15:18:48 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> Message-ID: <54C106C8.3030801@epardaud.fr> On 01/22/2015 03:08 PM, Palo Marton wrote: > What Stephane probably needs is something like this: > > I java.lang.reflect.Proxy.specializedToBoxed(Class interface, > Object specialized_object); > Well, yes again this can be solved by reflection less elegantly. But note that if we have such a method, then we can provide autoboxing in public fields whose type includes `any T` which either solves _that_ problem of my proposal, or at least means that both approaches will need to solve this issue. From vitalyd at gmail.com Thu Jan 22 14:21:56 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 09:21:56 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C105B5.6050708@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> <54C105B5.6050708@oracle.com> Message-ID: I guess my overarching point is this use of ? and coding to erasure, although works, is just a byproduct of current erasure. Let's think of it this way: when java 20 is out, do you want it to still be carrying baggage and be crippled in some way because it needed to support the language as it existed < java 9/10? I get backcompat is important and a "feature" of java, but it would be unfortunate if that keeps the language from evolving to higher levels. sent from my phone On Jan 22, 2015 9:14 AM, "Maurizio Cimadamore" < maurizio.cimadamore at oracle.com> wrote: > I think both Stephane (and you) and Vitaly have a point; such dynamic > frameworks have two alternatives: > > * keep working the way they do now - which implies List is related > to List/List > * let them be generified, and then use reflection (on steroids) to let the > right thing happen at runtime > > Maurizio > > On 22/01/15 13:58, Stephen Colebourne wrote: > >> On 22 January 2015 at 13:26, St?phane ?pardaud wrote: >> >>> You can see that at compile-time I don't have any idea about which value >>> types I could encounter, so the only instantiation of `T` I will have is >>> `Object`, never `value Date` which may pop up there, and since (unboxed) >>> value types don't have any common superclass I can't traverse >>> collections of >>> them without knowing at compile-time what type they are. Which is not >>> possible for frameworks. >>> >> Stephane is making a key point that is critical to the success of >> value types as a project. Almost every piece of framework logic that >> navigates around an arbitrary object structure will have code that >> queries if an object is a Collection/Map and performs specific >> processing, for example: >> https://github.com/JodaOrg/joda-beans/blob/master/src/ >> main/java/org/joda/beans/ser/SerIteratorFactory.java#L77 >> https://github.com/JodaOrg/joda-beans/blob/master/src/ >> main/java/org/joda/beans/ser/xml/JodaBeanXmlWriter.java#L176 >> >> I'm sure its easy to find other examples in similar projects. Note >> that this is a key problem to the Beans v2.0 MetaModel stuff I've been >> looking at. >> >> It seems necessary to make List is in some way related to >> List, just like a value can be boxed to an object. Doing so >> should take existing libraries that accept Collection and loop over >> it, and allow them to accept Collection. It seems to me that >> anything else is token backward compatibility, rather than real >> backwards compatibility. As indicated in other messages, you don't >> always know what T is at compile time. >> >> Stephen >> > > From vitalyd at gmail.com Thu Jan 22 14:30:23 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 09:30:23 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C106C8.3030801@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> Message-ID: I don't quite get the elegance comment - there's already nothing elegant about heavy use of reflection. Yes, it gets the job done in some cases, but it's a minority use case (I hope!). Also, keep in mind that calling a Collection taking method with value type collections may not be safe as the code in that method may be relying on Object semantics (e.g. locks on objects, compares reference identity, etc); in some ways it's safer to call any T methods because presumably person writing those is conscious of value types being a possibility and also compiler may warn if any assumptions are made about it being an Object. sent from my phone On Jan 22, 2015 9:19 AM, "St?phane ?pardaud" wrote: > On 01/22/2015 03:08 PM, Palo Marton wrote: > >> What Stephane probably needs is something like this: >> >> I java.lang.reflect.Proxy.specializedToBoxed(Class interface, >> Object specialized_object); >> >> Well, yes again this can be solved by reflection less elegantly. But > note that if we have such a method, then we can provide autoboxing in > public fields whose type includes `any T` which either solves _that_ > problem of my proposal, or at least means that both approaches will need to > solve this issue. > From maurizio.cimadamore at oracle.com Thu Jan 22 14:33:15 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 14:33:15 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C1064E.4020908@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C10193.40900@oracle.com> <54C1021B.3090605@epardaud.fr> <54C10495.9020309@oracle.com> <54C1064E.4020908@epardaud.fr> Message-ID: <54C10A2B.4030204@oracle.com> On 22/01/15 14:16, St?phane ?pardaud wrote: > And that's where you may be aware of technical issues that would make > this hard that I'm not aware of yet. If it's just about being able to > find back erased(A) from and instance of specialized(A) then it > doesn't sound too hard, I suppose that information will be available > in the runtime class for specialized(A), so `checkcast` and > `instanceof` and friends can use that for checks. Pretty much the same > way we'd allow these types to be interchangeable at compile-time from > javac. I believe you are looking at something that's vaguely resemblant of this [1] (look at the 'interface injection' section) and [2]. Which is a big VM feature on its own. Maurizio [1] - http://openjdk.java.net/projects/mlvm/subprojects.html [2] - https://blogs.oracle.com/jrose/entry/interface_injection_in_the_vm From stef at epardaud.fr Thu Jan 22 14:34:01 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 15:34:01 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> Message-ID: <54C10A59.2000000@epardaud.fr> On 01/22/2015 03:30 PM, Vitaly Davidovich wrote: > > I don't quite get the elegance comment - there's already nothing > elegant about heavy use of reflection. Yes, it gets the job done in > some cases, but it's a minority use case (I hope!). > Well, IMO there's a lot more elegance in List being an instance of List (variance issues aside). > > Also, keep in mind that calling a Collection taking method with > value type collections may not be safe as the code in that method may > be relying on Object semantics (e.g. locks on objects, compares > reference identity, etc); in some ways it's safer to call any T > methods because presumably person writing those is conscious of value > types being a possibility and also compiler may warn if any > assumptions are made about it being an Object. > Sure, except it needs to be possible to do ;) From stef at epardaud.fr Thu Jan 22 14:36:56 2015 From: stef at epardaud.fr (Stef Epardaud) Date: Thu, 22 Jan 2015 15:36:56 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C105B5.6050708@oracle.com> References: <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> <54C105B5.6050708@oracle.com> Message-ID: <20150122143655.GA15186@inforealm.org> On Thu, Jan 22, 2015 at 02:14:13PM +0000, Maurizio Cimadamore wrote: > I think both Stephane (and you) and Vitaly have a point; such > dynamic frameworks have two alternatives: > > * keep working the way they do now - which implies List is > related to List/List > * let them be generified, and then use reflection (on steroids) to > let the right thing happen at runtime Yes, it is possible that you could specialise code at runtime via the reflection API (perhaps it will already be required by the current proto), so you could do something like: Object fieldValue = field.get(instance); if(Collection.class.isSpecialisedInstance(fieldValue)){ Method m = getClass().getMethod("traversalMethod").specialize(fieldValue.getSpecialisedTypeParameters().get(0)); m.invoke(fieldValue); } With: void traversalMethod(Collection collection){...} That requires runtime specialisation but I suppose that the current proto already supports that in other locations (linking). TBH, while I find this much less elegant, if the only issue there is with the current proto is WRT frameworks (the points we've raised), and since there are far less framework code than user code, and this approach is simpler to implement (if only because there already is a proto for that), then perhaps that's enough? I guess the reflection layer will have to do autoboxing for value types the way it already has to do with primitive autoboxing, so that's about the only thing missing to be able to deal with compile-time unknown value type argument instantiations. -- St?phane Epardaud From stef at epardaud.fr Thu Jan 22 14:40:02 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 15:40:02 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C10A2B.4030204@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C10193.40900@oracle.com> <54C1021B.3090605@epardaud.fr> <54C10495.9020309@oracle.com> <54C1064E.4020908@epardaud.fr> <54C10A2B.4030204@oracle.com> Message-ID: <54C10BC2.6010403@epardaud.fr> On 01/22/2015 03:33 PM, Maurizio Cimadamore wrote: > > On 22/01/15 14:16, St?phane ?pardaud wrote: >> And that's where you may be aware of technical issues that would make >> this hard that I'm not aware of yet. If it's just about being able to >> find back erased(A) from and instance of specialized(A) then it >> doesn't sound too hard, I suppose that information will be available >> in the runtime class for specialized(A), so `checkcast` and >> `instanceof` and friends can use that for checks. Pretty much the >> same way we'd allow these types to be interchangeable at compile-time >> from javac. > I believe you are looking at something that's vaguely resemblant of > this [1] (look at the 'interface injection' section) and [2]. Which is > a big VM feature on its own. Well, not really because all the methods are available already in the class, they're not injected (at least, not more so than the specialised methods already implemented). It's "just" a change in the type equivalence predicates. From vitalyd at gmail.com Thu Jan 22 14:42:15 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 09:42:15 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C10A59.2000000@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> Message-ID: Ok can we first agree that List is NOT List ? :) That's unsound since you can't add any Object in there if things are reified. I think you guys mean List . sent from my phone On Jan 22, 2015 9:34 AM, "St?phane ?pardaud" wrote: > On 01/22/2015 03:30 PM, Vitaly Davidovich wrote: > >> >> I don't quite get the elegance comment - there's already nothing elegant >> about heavy use of reflection. Yes, it gets the job done in some cases, >> but it's a minority use case (I hope!). >> >> Well, IMO there's a lot more elegance in List being an > instance of List (variance issues aside). > >> >> Also, keep in mind that calling a Collection taking method with value >> type collections may not be safe as the code in that method may be relying >> on Object semantics (e.g. locks on objects, compares reference identity, >> etc); in some ways it's safer to call any T methods because presumably >> person writing those is conscious of value types being a possibility and >> also compiler may warn if any assumptions are made about it being an Object. >> >> Sure, except it needs to be possible to do ;) > From stef at epardaud.fr Thu Jan 22 14:46:57 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 15:46:57 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> Message-ID: <54C10D61.5020606@epardaud.fr> On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: > > Ok can we first agree that List is NOT List ? :) That's > unsound since you can't add any Object in there if things are > reified. I think you guys mean List . > > Well, that what I meant by "variance issues aside". It is unsound but detected at runtime. This is already a limitation of List in Java at compile-time, I'm not proposing to fix it. From vitalyd at gmail.com Thu Jan 22 14:48:48 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 09:48:48 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C10A59.2000000@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> Message-ID: About the "possible to do"; frankly, even if java supported this, I'd be worried about calling such a method for the reasons I mentioned. Inevitably, some code like that will break in very subtle ways. I'd want the framework I'm using to have been properly upgraded to support any T. I don't think backcompat should mean nobody has to do anything at all. That's at odds with evolving the language in some cases. sent from my phone On Jan 22, 2015 9:34 AM, "St?phane ?pardaud" wrote: > On 01/22/2015 03:30 PM, Vitaly Davidovich wrote: > >> >> I don't quite get the elegance comment - there's already nothing elegant >> about heavy use of reflection. Yes, it gets the job done in some cases, >> but it's a minority use case (I hope!). >> >> Well, IMO there's a lot more elegance in List being an > instance of List (variance issues aside). > >> >> Also, keep in mind that calling a Collection taking method with value >> type collections may not be safe as the code in that method may be relying >> on Object semantics (e.g. locks on objects, compares reference identity, >> etc); in some ways it's safer to call any T methods because presumably >> person writing those is conscious of value types being a possibility and >> also compiler may warn if any assumptions are made about it being an Object. >> >> Sure, except it needs to be possible to do ;) > From scolebourne at joda.org Thu Jan 22 14:48:37 2015 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 22 Jan 2015 14:48:37 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C105B5.6050708@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> <54C105B5.6050708@oracle.com> Message-ID: On 22 January 2015 at 14:14, Maurizio Cimadamore wrote: > I think both Stephane (and you) and Vitaly have a point; such dynamic > frameworks have two alternatives: > > * keep working the way they do now - which implies List is related to > List/List > * let them be generified, and then use reflection (on steroids) to let the > right thing happen at runtime Most frameworks are OSS and will be coded to work today on JDK 8 (or 9, assuming value types are in 10). Saying that those frameworks should use reflection on steroids is a non-option. You cannot write a library today (or in 9) that will work in 10. This would force each OSS library to release a dedicated version for 10, effectively asking each project to have a pre-10 version and a post-10 version. This is a terrible incompatibility in the eco-system, with huge knock-on effects for years (such as jar-hell). So, yes I *really* think if those are the only two options, then it must be option 1, relating List to List. Stephen From vitalyd at gmail.com Thu Jan 22 15:00:09 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 10:00:09 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FA89.2000906@epardaud.fr> <54C105B5.6050708@oracle.com> Message-ID: But value types (and reified generics) are a fundamental change to the language - even if compiler allowed compiling this stuff, things will break if you call code that never expected value types - they have fundamentally different behavior. Do you want things to *just* compile or to also have some sense that they'll work? Haven't you ever made breaking changes to your libs for the betterment of it in the long term? Also, if you expose ? methods to the users, can you not add new ones taking any T and deprecate the old ones? If this reflection stuff is internal to the framework/lib, then user just has to upgrade the JDK. Can you give a concrete example where any of this would be too burdensome? sent from my phone On Jan 22, 2015 9:49 AM, "Stephen Colebourne" wrote: > On 22 January 2015 at 14:14, Maurizio Cimadamore > wrote: > > I think both Stephane (and you) and Vitaly have a point; such dynamic > > frameworks have two alternatives: > > > > * keep working the way they do now - which implies List is > related to > > List/List > > * let them be generified, and then use reflection (on steroids) to let > the > > right thing happen at runtime > > Most frameworks are OSS and will be coded to work today on JDK 8 (or > 9, assuming value types are in 10). Saying that those frameworks > should use reflection on steroids is a non-option. You cannot write a > library today (or in 9) that will work in 10. This would force each > OSS library to release a dedicated version for 10, effectively asking > each project to have a pre-10 version and a post-10 version. This is a > terrible incompatibility in the eco-system, with huge knock-on effects > for years (such as jar-hell). > > So, yes I *really* think if those are the only two options, then it > must be option 1, relating List to List. > > Stephen > From stef at epardaud.fr Thu Jan 22 15:03:17 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 16:03:17 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C10D61.5020606@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> Message-ID: <54C11135.7000503@epardaud.fr> On 01/22/2015 03:46 PM, St?phane ?pardaud wrote: > On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: >> >> Ok can we first agree that List is NOT List ? :) That's >> unsound since you can't add any Object in there if things are >> reified. I think you guys mean List . >> >> > Well, that what I meant by "variance issues aside". It is unsound but > detected at runtime. This is already a limitation of List in Java at > compile-time, I'm not proposing to fix it. And yeah, goes to show how I got used to List being a subtype of List in Ceylon where List is immutable and covariant in T ;) From vitalyd at gmail.com Thu Jan 22 15:11:55 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 10:11:55 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C10D61.5020606@epardaud.fr> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> Message-ID: Ok, got you. It's detected at runtime just like dereferencing a null or indexing outside the bounds of an array -- nothing sound about these, but java's a safe language so has the safeguards. By the way, the issue of List not being covariant with List was also a drag sometimes in .NET where you knew you weren't mutating the list and only wanted to read the values (which is safe). So, C# has the in/out keywords when you define the generic type (only works on interfaces and delegates) -- compiler then checks your signatures: https://msdn.microsoft.com/en-us/library/dd469487.aspx and https://msdn.microsoft.com/en-us/library/dd469484.aspx. I hate to keep tooting .NET's horn on this list, but I am impressed with how they've addressed these various issues. On Thu, Jan 22, 2015 at 9:46 AM, St?phane ?pardaud wrote: > On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: > >> >> Ok can we first agree that List is NOT List ? :) That's >> unsound since you can't add any Object in there if things are reified. I >> think you guys mean List . >> >> >> Well, that what I meant by "variance issues aside". It is unsound but > detected at runtime. This is already a limitation of List in Java at > compile-time, I'm not proposing to fix it. > From brian.goetz at oracle.com Thu Jan 22 16:32:31 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 22 Jan 2015 11:32:31 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> Message-ID: <54C1261F.5030708@oracle.com> Thanks Vitaly, explicating how things are done on .NET is highly constructive, even if we can't just "do what they did." On the other hand, we've seen some impassioned defense of "I want to keep being able to use List (or raw List)" on this list -- and .NET doesn't even have an analogue of these types, you have to use generic methods to abstract over types. So while one might find the .NET type system "better", we must acknowledge it doesn't provide Java developers with the tools they're used to. (You could argue this is a feature rather than a bug; in a world with no existing code or developer experience, this might be true, but that's not the world we're in right now.) And this is the essential tension; a solution that retains compatibility (not necessarily just with existing code, but people's existing expectations of how it works) is necessarily going to result in a "worse" language going forward. There's no magic solution here (though there are bad solutions), the art is putting the pain where it is felt least. Anyone who thinks they have "the" answer to this problem obviously doesn't understand the problem. So for those trying to contribute: it would be more helpful if, rather than coming at it from an "I have the answer" perspective, try something more like "I might have a small piece of an answer." Being in such a mental place is far more likely to result in real contribution! On 1/22/2015 10:11 AM, Vitaly Davidovich wrote: > Ok, got you. It's detected at runtime just like dereferencing a null or > indexing outside the bounds of an array -- nothing sound about these, but > java's a safe language so has the safeguards. > > By the way, the issue of List not being covariant with List > was also a drag sometimes in .NET where you knew you weren't mutating the > list and only wanted to read the values (which is safe). So, C# has the > in/out keywords when you define the generic type (only works on interfaces > and delegates) -- compiler then checks your signatures: > https://msdn.microsoft.com/en-us/library/dd469487.aspx and > https://msdn.microsoft.com/en-us/library/dd469484.aspx. > > I hate to keep tooting .NET's horn on this list, but I am impressed with > how they've addressed these various issues. > > On Thu, Jan 22, 2015 at 9:46 AM, St?phane ?pardaud wrote: > >> On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: >> >>> >>> Ok can we first agree that List is NOT List ? :) That's >>> unsound since you can't add any Object in there if things are reified. I >>> think you guys mean List . >>> >>> >>> Well, that what I meant by "variance issues aside". It is unsound but >> detected at runtime. This is already a limitation of List in Java at >> compile-time, I'm not proposing to fix it. >> From vitalyd at gmail.com Thu Jan 22 17:19:31 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 12:19:31 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C1261F.5030708@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> <54C1261F.5030708@oracle.com> Message-ID: I hear you Brian, and by no means think I have THE answer. On the other hand, we've seen some impassioned defense of "I want to keep > being able to use List (or raw List)" on this list -- and .NET doesn't > even have an analogue of these types, you have to use generic methods to > abstract over types. So while one might find the .NET type system > "better", we must acknowledge it doesn't provide Java developers with the > tools they're used to. (You could argue this is a feature rather than a > bug; in a world with no existing code or developer experience, this might > be true, but that's not the world we're in right now.) I'm not sure "tools they're used to" is all that meaningful, IMHO. Erased generics were added, no precedent in java prior to that. Lambdas were added, no precedent to that beforehand. JMM was revised, no precedent. Fork join added. Streams added. Value types will be added. By definition, things will not always be "what people are used to" unless the language doesn't adapt and progress. Again, just my opinion of course. And this is the essential tension; a solution that retains compatibility > (not necessarily just with existing code, but people's existing > expectations of how it works) is necessarily going to result in a "worse" > language going forward. There's no magic solution here (though there are > bad solutions), the art is putting the pain where it is felt least. What does compatibility mean in this particular context? I argue that "code compiles" is a low barrier here -- value types are a completely new concept to java, and calling "old" code that simply took Collection is not necessarily backcompat, for the reasons I mentioned. Also, as I mentioned above, there's nothing wrong with re-learning things to adjust expectations when new things are added. See, I don't want a "worse" language just to drag along historical artifacts :) The language is going to outlive (if it remains relevant and "refreshed") the vast majority of existing libs/frameworks/etc that are being alluded to in this thread. I'm not saying java should be evolved recklessly nor carelessly nor ignorant of existing code, but I'd say the bar for "we're going to make the language/type system/compiler/JVM/etc a bit 'worse' for the sake of framework/lib/code Y" has to be very high. On Thu, Jan 22, 2015 at 11:32 AM, Brian Goetz wrote: > Thanks Vitaly, explicating how things are done on .NET is highly > constructive, even if we can't just "do what they did." > > On the other hand, we've seen some impassioned defense of "I want to keep > being able to use List (or raw List)" on this list -- and .NET doesn't > even have an analogue of these types, you have to use generic methods to > abstract over types. So while one might find the .NET type system > "better", we must acknowledge it doesn't provide Java developers with the > tools they're used to. (You could argue this is a feature rather than a > bug; in a world with no existing code or developer experience, this might > be true, but that's not the world we're in right now.) > > And this is the essential tension; a solution that retains compatibility > (not necessarily just with existing code, but people's existing > expectations of how it works) is necessarily going to result in a "worse" > language going forward. There's no magic solution here (though there are > bad solutions), the art is putting the pain where it is felt least. > > Anyone who thinks they have "the" answer to this problem obviously doesn't > understand the problem. So for those trying to contribute: it would be > more helpful if, rather than coming at it from an "I have the answer" > perspective, try something more like "I might have a small piece of an > answer." Being in such a mental place is far more likely to result in real > contribution! > > > On 1/22/2015 10:11 AM, Vitaly Davidovich wrote: > >> Ok, got you. It's detected at runtime just like dereferencing a null or >> indexing outside the bounds of an array -- nothing sound about these, but >> java's a safe language so has the safeguards. >> >> By the way, the issue of List not being covariant with List >> was also a drag sometimes in .NET where you knew you weren't mutating the >> list and only wanted to read the values (which is safe). So, C# has the >> in/out keywords when you define the generic type (only works on interfaces >> and delegates) -- compiler then checks your signatures: >> https://msdn.microsoft.com/en-us/library/dd469487.aspx and >> https://msdn.microsoft.com/en-us/library/dd469484.aspx. >> >> I hate to keep tooting .NET's horn on this list, but I am impressed with >> how they've addressed these various issues. >> >> On Thu, Jan 22, 2015 at 9:46 AM, St?phane ?pardaud >> wrote: >> >> On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: >>> >>> >>>> Ok can we first agree that List is NOT List ? :) That's >>>> unsound since you can't add any Object in there if things are reified. >>>> I >>>> think you guys mean List . >>>> >>>> >>>> Well, that what I meant by "variance issues aside". It is unsound but >>>> >>> detected at runtime. This is already a limitation of List in Java at >>> compile-time, I'm not proposing to fix it. >>> >>> From stef at epardaud.fr Thu Jan 22 17:45:25 2015 From: stef at epardaud.fr (=?UTF-8?B?U3TDqXBoYW5lIMOJcGFyZGF1ZA==?=) Date: Thu, 22 Jan 2015 18:45:25 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C1261F.5030708@oracle.com> References: <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> <54C1261F.5030708@oracle.com> Message-ID: <54C13735.10509@epardaud.fr> On 01/22/2015 05:32 PM, Brian Goetz wrote: > And this is the essential tension; a solution that retains > compatibility (not necessarily just with existing code, but people's > existing expectations of how it works) is necessarily going to result > in a "worse" language going forward. There's no magic solution here > (though there are bad solutions), the art is putting the pain where it > is felt least. Which is exactly what we're discussing. > Anyone who thinks they have "the" answer to this problem obviously > doesn't understand the problem. So for those trying to contribute: it > would be more helpful if, rather than coming at it from an "I have the > answer" perspective, try something more like "I might have a small > piece of an answer." Being in such a mental place is far more likely > to result in real contribution! I could pretend this is not meant to be relevant to the discussion we had today, but then that'd be a really unfortunate coincidence. So on the (reasonable) assumption that you are talking about my proposal I think you have real issues with how you read people's proposals. I certainly haven't approached this by belittling what you guys came up with, and haven't claimed that my solution is better and easier. What I have done is expose some issues with your decisions and prototype and proposed some tweaks and then asked for feedback on my proposal, which people have done here and we've advanced the discussion quite a lot WRT the good/bad of it. Frankly if your contribution to people just trying to help is this kind of attitude it's no wonder we got further in the discussion while you were not involved. I just don't get it, it's almost as if you only want feedback on your proposal as long as it validates it and does not question it or offer alternatives or raise issues with it. I hope I'm just wrong and you were talking about some other person or something that happened to you today IRL and it's just bad luck that it happened to be directed at people trying to help and you looking like you're patronising us. Except I've already seen such behaviour from you on this list in the little time I've spent here, which does not lead me to assume I'm wrong. I don't understand your frequent attacks, especially in an open-source context. Hell, if I was telling contributors in the projects I work on how unhelpful they were in such passive-aggressive terms I don't think we'd have anyone helping us any more. I think the issue with "mental state" is not where you think it is. Sincerely. From vitalyd at gmail.com Thu Jan 22 17:46:53 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 12:46:53 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> <54C1261F.5030708@oracle.com> Message-ID: By the way, is it fair to say that one of the main reasons generics were implemented with erasure back in Java 5 was also for backcompat reasons? If so, here we are many years down the road, with lots more code, patterns, familiarity, etc to preserve, where that decision is making it harder to evolve the language (and where reification, or at least some form of it, is going to be done anyway, but at higher cost now potentially). Just saying :) On Thu, Jan 22, 2015 at 12:19 PM, Vitaly Davidovich wrote: > I hear you Brian, and by no means think I have THE answer. > > On the other hand, we've seen some impassioned defense of "I want to keep >> being able to use List (or raw List)" on this list -- and .NET doesn't >> even have an analogue of these types, you have to use generic methods to >> abstract over types. So while one might find the .NET type system >> "better", we must acknowledge it doesn't provide Java developers with the >> tools they're used to. (You could argue this is a feature rather than a >> bug; in a world with no existing code or developer experience, this might >> be true, but that's not the world we're in right now.) > > > I'm not sure "tools they're used to" is all that meaningful, IMHO. Erased > generics were added, no precedent in java prior to that. Lambdas were > added, no precedent to that beforehand. JMM was revised, no precedent. > Fork join added. Streams added. Value types will be added. By > definition, things will not always be "what people are used to" unless the > language doesn't adapt and progress. Again, just my opinion of course. > > And this is the essential tension; a solution that retains compatibility >> (not necessarily just with existing code, but people's existing >> expectations of how it works) is necessarily going to result in a "worse" >> language going forward. There's no magic solution here (though there are >> bad solutions), the art is putting the pain where it is felt least. > > > What does compatibility mean in this particular context? I argue that > "code compiles" is a low barrier here -- value types are a completely new > concept to java, and calling "old" code that simply took Collection is > not necessarily backcompat, for the reasons I mentioned. Also, as I > mentioned above, there's nothing wrong with re-learning things to adjust > expectations when new things are added. > > See, I don't want a "worse" language just to drag along historical > artifacts :) The language is going to outlive (if it remains relevant and > "refreshed") the vast majority of existing libs/frameworks/etc that are > being alluded to in this thread. I'm not saying java should be evolved > recklessly nor carelessly nor ignorant of existing code, but I'd say the > bar for "we're going to make the language/type system/compiler/JVM/etc a > bit 'worse' for the sake of framework/lib/code Y" has to be very high. > > > On Thu, Jan 22, 2015 at 11:32 AM, Brian Goetz > wrote: > >> Thanks Vitaly, explicating how things are done on .NET is highly >> constructive, even if we can't just "do what they did." >> >> On the other hand, we've seen some impassioned defense of "I want to keep >> being able to use List (or raw List)" on this list -- and .NET doesn't >> even have an analogue of these types, you have to use generic methods to >> abstract over types. So while one might find the .NET type system >> "better", we must acknowledge it doesn't provide Java developers with the >> tools they're used to. (You could argue this is a feature rather than a >> bug; in a world with no existing code or developer experience, this might >> be true, but that's not the world we're in right now.) >> >> And this is the essential tension; a solution that retains compatibility >> (not necessarily just with existing code, but people's existing >> expectations of how it works) is necessarily going to result in a "worse" >> language going forward. There's no magic solution here (though there are >> bad solutions), the art is putting the pain where it is felt least. >> >> Anyone who thinks they have "the" answer to this problem obviously >> doesn't understand the problem. So for those trying to contribute: it >> would be more helpful if, rather than coming at it from an "I have the >> answer" perspective, try something more like "I might have a small piece of >> an answer." Being in such a mental place is far more likely to result in >> real contribution! >> >> >> On 1/22/2015 10:11 AM, Vitaly Davidovich wrote: >> >>> Ok, got you. It's detected at runtime just like dereferencing a null or >>> indexing outside the bounds of an array -- nothing sound about these, but >>> java's a safe language so has the safeguards. >>> >>> By the way, the issue of List not being covariant with List>> T> >>> was also a drag sometimes in .NET where you knew you weren't mutating the >>> list and only wanted to read the values (which is safe). So, C# has the >>> in/out keywords when you define the generic type (only works on >>> interfaces >>> and delegates) -- compiler then checks your signatures: >>> https://msdn.microsoft.com/en-us/library/dd469487.aspx and >>> https://msdn.microsoft.com/en-us/library/dd469484.aspx. >>> >>> I hate to keep tooting .NET's horn on this list, but I am impressed with >>> how they've addressed these various issues. >>> >>> On Thu, Jan 22, 2015 at 9:46 AM, St?phane ?pardaud >>> wrote: >>> >>> On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: >>>> >>>> >>>>> Ok can we first agree that List is NOT List ? :) That's >>>>> unsound since you can't add any Object in there if things are >>>>> reified. I >>>>> think you guys mean List . >>>>> >>>>> >>>>> Well, that what I meant by "variance issues aside". It is unsound but >>>>> >>>> detected at runtime. This is already a limitation of List in Java at >>>> compile-time, I'm not proposing to fix it. >>>> >>>> > From maurizio.cimadamore at oracle.com Thu Jan 22 17:57:11 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jan 2015 17:57:11 +0000 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> <54C1261F.5030708@oracle.com> Message-ID: <54C139F7.5000102@oracle.com> This is an old classic: http://gafter.blogspot.co.uk/2004/09/puzzling-through-erasure-answer.html Maurizio On 22/01/15 17:46, Vitaly Davidovich wrote: > By the way, is it fair to say that one of the main reasons generics were > implemented with erasure back in Java 5 was also for backcompat reasons? If > so, here we are many years down the road, with lots more code, patterns, > familiarity, etc to preserve, where that decision is making it harder to > evolve the language (and where reification, or at least some form of it, is > going to be done anyway, but at higher cost now potentially). Just saying > :) > > On Thu, Jan 22, 2015 at 12:19 PM, Vitaly Davidovich > wrote: > >> I hear you Brian, and by no means think I have THE answer. >> >> On the other hand, we've seen some impassioned defense of "I want to keep >>> being able to use List (or raw List)" on this list -- and .NET doesn't >>> even have an analogue of these types, you have to use generic methods to >>> abstract over types. So while one might find the .NET type system >>> "better", we must acknowledge it doesn't provide Java developers with the >>> tools they're used to. (You could argue this is a feature rather than a >>> bug; in a world with no existing code or developer experience, this might >>> be true, but that's not the world we're in right now.) >> >> I'm not sure "tools they're used to" is all that meaningful, IMHO. Erased >> generics were added, no precedent in java prior to that. Lambdas were >> added, no precedent to that beforehand. JMM was revised, no precedent. >> Fork join added. Streams added. Value types will be added. By >> definition, things will not always be "what people are used to" unless the >> language doesn't adapt and progress. Again, just my opinion of course. >> >> And this is the essential tension; a solution that retains compatibility >>> (not necessarily just with existing code, but people's existing >>> expectations of how it works) is necessarily going to result in a "worse" >>> language going forward. There's no magic solution here (though there are >>> bad solutions), the art is putting the pain where it is felt least. >> >> What does compatibility mean in this particular context? I argue that >> "code compiles" is a low barrier here -- value types are a completely new >> concept to java, and calling "old" code that simply took Collection is >> not necessarily backcompat, for the reasons I mentioned. Also, as I >> mentioned above, there's nothing wrong with re-learning things to adjust >> expectations when new things are added. >> >> See, I don't want a "worse" language just to drag along historical >> artifacts :) The language is going to outlive (if it remains relevant and >> "refreshed") the vast majority of existing libs/frameworks/etc that are >> being alluded to in this thread. I'm not saying java should be evolved >> recklessly nor carelessly nor ignorant of existing code, but I'd say the >> bar for "we're going to make the language/type system/compiler/JVM/etc a >> bit 'worse' for the sake of framework/lib/code Y" has to be very high. >> >> >> On Thu, Jan 22, 2015 at 11:32 AM, Brian Goetz >> wrote: >> >>> Thanks Vitaly, explicating how things are done on .NET is highly >>> constructive, even if we can't just "do what they did." >>> >>> On the other hand, we've seen some impassioned defense of "I want to keep >>> being able to use List (or raw List)" on this list -- and .NET doesn't >>> even have an analogue of these types, you have to use generic methods to >>> abstract over types. So while one might find the .NET type system >>> "better", we must acknowledge it doesn't provide Java developers with the >>> tools they're used to. (You could argue this is a feature rather than a >>> bug; in a world with no existing code or developer experience, this might >>> be true, but that's not the world we're in right now.) >>> >>> And this is the essential tension; a solution that retains compatibility >>> (not necessarily just with existing code, but people's existing >>> expectations of how it works) is necessarily going to result in a "worse" >>> language going forward. There's no magic solution here (though there are >>> bad solutions), the art is putting the pain where it is felt least. >>> >>> Anyone who thinks they have "the" answer to this problem obviously >>> doesn't understand the problem. So for those trying to contribute: it >>> would be more helpful if, rather than coming at it from an "I have the >>> answer" perspective, try something more like "I might have a small piece of >>> an answer." Being in such a mental place is far more likely to result in >>> real contribution! >>> >>> >>> On 1/22/2015 10:11 AM, Vitaly Davidovich wrote: >>> >>>> Ok, got you. It's detected at runtime just like dereferencing a null or >>>> indexing outside the bounds of an array -- nothing sound about these, but >>>> java's a safe language so has the safeguards. >>>> >>>> By the way, the issue of List not being covariant with List>>> T> >>>> was also a drag sometimes in .NET where you knew you weren't mutating the >>>> list and only wanted to read the values (which is safe). So, C# has the >>>> in/out keywords when you define the generic type (only works on >>>> interfaces >>>> and delegates) -- compiler then checks your signatures: >>>> https://msdn.microsoft.com/en-us/library/dd469487.aspx and >>>> https://msdn.microsoft.com/en-us/library/dd469484.aspx. >>>> >>>> I hate to keep tooting .NET's horn on this list, but I am impressed with >>>> how they've addressed these various issues. >>>> >>>> On Thu, Jan 22, 2015 at 9:46 AM, St?phane ?pardaud >>>> wrote: >>>> >>>> On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: >>>>> >>>>>> Ok can we first agree that List is NOT List ? :) That's >>>>>> unsound since you can't add any Object in there if things are >>>>>> reified. I >>>>>> think you guys mean List . >>>>>> >>>>>> >>>>>> Well, that what I meant by "variance issues aside". It is unsound but >>>>>> >>>>> detected at runtime. This is already a limitation of List in Java at >>>>> compile-time, I'm not proposing to fix it. >>>>> >>>>> From vitalyd at gmail.com Thu Jan 22 18:17:52 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 13:17:52 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C139F7.5000102@oracle.com> References: <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> <54C1261F.5030708@oracle.com> <54C139F7.5000102@oracle.com> Message-ID: So yeah, be mindful of compatibility with existing code but please don't let that dominate the evolution of the language. :) But to this day, I find it crazy that I write source code with type information available, but then JVM sees Object (or erased upper bound) and has to have machinery to try and regain that knowledge for optimization purposes. Sorry, that was a digression :). sent from my phone On Jan 22, 2015 12:57 PM, "Maurizio Cimadamore" < maurizio.cimadamore at oracle.com> wrote: > This is an old classic: > > http://gafter.blogspot.co.uk/2004/09/puzzling-through-erasure-answer.html > > Maurizio > > On 22/01/15 17:46, Vitaly Davidovich wrote: > >> By the way, is it fair to say that one of the main reasons generics were >> implemented with erasure back in Java 5 was also for backcompat reasons? >> If >> so, here we are many years down the road, with lots more code, patterns, >> familiarity, etc to preserve, where that decision is making it harder to >> evolve the language (and where reification, or at least some form of it, >> is >> going to be done anyway, but at higher cost now potentially). Just saying >> :) >> >> On Thu, Jan 22, 2015 at 12:19 PM, Vitaly Davidovich >> wrote: >> >> I hear you Brian, and by no means think I have THE answer. >>> >>> On the other hand, we've seen some impassioned defense of "I want to keep >>> >>>> being able to use List (or raw List)" on this list -- and .NET >>>> doesn't >>>> even have an analogue of these types, you have to use generic methods to >>>> abstract over types. So while one might find the .NET type system >>>> "better", we must acknowledge it doesn't provide Java developers with >>>> the >>>> tools they're used to. (You could argue this is a feature rather than a >>>> bug; in a world with no existing code or developer experience, this >>>> might >>>> be true, but that's not the world we're in right now.) >>>> >>> >>> I'm not sure "tools they're used to" is all that meaningful, IMHO. >>> Erased >>> generics were added, no precedent in java prior to that. Lambdas were >>> added, no precedent to that beforehand. JMM was revised, no precedent. >>> Fork join added. Streams added. Value types will be added. By >>> definition, things will not always be "what people are used to" unless >>> the >>> language doesn't adapt and progress. Again, just my opinion of course. >>> >>> And this is the essential tension; a solution that retains compatibility >>> >>>> (not necessarily just with existing code, but people's existing >>>> expectations of how it works) is necessarily going to result in a >>>> "worse" >>>> language going forward. There's no magic solution here (though there >>>> are >>>> bad solutions), the art is putting the pain where it is felt least. >>>> >>> >>> What does compatibility mean in this particular context? I argue that >>> "code compiles" is a low barrier here -- value types are a completely new >>> concept to java, and calling "old" code that simply took Collection is >>> not necessarily backcompat, for the reasons I mentioned. Also, as I >>> mentioned above, there's nothing wrong with re-learning things to adjust >>> expectations when new things are added. >>> >>> See, I don't want a "worse" language just to drag along historical >>> artifacts :) The language is going to outlive (if it remains relevant and >>> "refreshed") the vast majority of existing libs/frameworks/etc that are >>> being alluded to in this thread. I'm not saying java should be evolved >>> recklessly nor carelessly nor ignorant of existing code, but I'd say the >>> bar for "we're going to make the language/type system/compiler/JVM/etc a >>> bit 'worse' for the sake of framework/lib/code Y" has to be very high. >>> >>> >>> On Thu, Jan 22, 2015 at 11:32 AM, Brian Goetz >>> wrote: >>> >>> Thanks Vitaly, explicating how things are done on .NET is highly >>>> constructive, even if we can't just "do what they did." >>>> >>>> On the other hand, we've seen some impassioned defense of "I want to >>>> keep >>>> being able to use List (or raw List)" on this list -- and .NET >>>> doesn't >>>> even have an analogue of these types, you have to use generic methods to >>>> abstract over types. So while one might find the .NET type system >>>> "better", we must acknowledge it doesn't provide Java developers with >>>> the >>>> tools they're used to. (You could argue this is a feature rather than a >>>> bug; in a world with no existing code or developer experience, this >>>> might >>>> be true, but that's not the world we're in right now.) >>>> >>>> And this is the essential tension; a solution that retains compatibility >>>> (not necessarily just with existing code, but people's existing >>>> expectations of how it works) is necessarily going to result in a >>>> "worse" >>>> language going forward. There's no magic solution here (though there >>>> are >>>> bad solutions), the art is putting the pain where it is felt least. >>>> >>>> Anyone who thinks they have "the" answer to this problem obviously >>>> doesn't understand the problem. So for those trying to contribute: it >>>> would be more helpful if, rather than coming at it from an "I have the >>>> answer" perspective, try something more like "I might have a small >>>> piece of >>>> an answer." Being in such a mental place is far more likely to result >>>> in >>>> real contribution! >>>> >>>> >>>> On 1/22/2015 10:11 AM, Vitaly Davidovich wrote: >>>> >>>> Ok, got you. It's detected at runtime just like dereferencing a null >>>>> or >>>>> indexing outside the bounds of an array -- nothing sound about these, >>>>> but >>>>> java's a safe language so has the safeguards. >>>>> >>>>> By the way, the issue of List not being covariant with List>>>> T> >>>>> was also a drag sometimes in .NET where you knew you weren't mutating >>>>> the >>>>> list and only wanted to read the values (which is safe). So, C# has >>>>> the >>>>> in/out keywords when you define the generic type (only works on >>>>> interfaces >>>>> and delegates) -- compiler then checks your signatures: >>>>> https://msdn.microsoft.com/en-us/library/dd469487.aspx and >>>>> https://msdn.microsoft.com/en-us/library/dd469484.aspx. >>>>> >>>>> I hate to keep tooting .NET's horn on this list, but I am impressed >>>>> with >>>>> how they've addressed these various issues. >>>>> >>>>> On Thu, Jan 22, 2015 at 9:46 AM, St?phane ?pardaud >>>>> wrote: >>>>> >>>>> On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: >>>>> >>>>>> >>>>>> Ok can we first agree that List is NOT List ? :) That's >>>>>>> unsound since you can't add any Object in there if things are >>>>>>> reified. I >>>>>>> think you guys mean List . >>>>>>> >>>>>>> >>>>>>> Well, that what I meant by "variance issues aside". It is unsound >>>>>>> but >>>>>>> >>>>>>> detected at runtime. This is already a limitation of List in Java at >>>>>> compile-time, I'm not proposing to fix it. >>>>>> >>>>>> >>>>>> > From brian.goetz at oracle.com Thu Jan 22 18:33:46 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 22 Jan 2015 13:33:46 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> <54C1261F.5030708@oracle.com> Message-ID: <54C1428A.4000501@oracle.com> Short answer: yes. (Some of the reasoning for this is reconstructed in the latest State of the Specialization.) The key requirement was, and continues to be: gradual migration compatibility. This has multiple facets: - It should be practical to generify an existing class (within reason), rather than throwing it out and starting over - It should be practical to generify libraries gradually, for example generifying a superclass separately from a subclass - Generification of libraries and their clients must not require a "flag day". It should be possible to generify the library first and generify the client later (or not). We have the same requirement now; existing generic classes should be extendable to any-fied generics, it should not be necessary to any-fy libraries all at once (though they do have to be any-fied top-down), and -- most importantly -- existing client code must continue to work against any-fied libraries. What is meant by "continue to work" can be subtle, though. For example, some have balked at the idea that List would continue to mean List -- as it always has meant -- is not "continuing to work", because it's not "what the user thought they meant when they wrote it ten years ago." But, yes, the requirement in 5 were centered around gradual migration compatibility from non-generic to generic; we have the same requirements when migrating from generic to "any-generic". On 1/22/2015 12:46 PM, Vitaly Davidovich wrote: > By the way, is it fair to say that one of the main reasons generics were > implemented with erasure back in Java 5 was also for backcompat reasons? > If so, here we are many years down the road, with lots more code, > patterns, familiarity, etc to preserve, where that decision is making it > harder to evolve the language (and where reification, or at least some > form of it, is going to be done anyway, but at higher cost now > potentially). Just saying :) > > On Thu, Jan 22, 2015 at 12:19 PM, Vitaly Davidovich > wrote: > > I hear you Brian, and by no means think I have THE answer. > > On the other hand, we've seen some impassioned defense of "I > want to keep being able to use List (or raw List)" on this > list -- and .NET doesn't even have an analogue of these types, > you have to use generic methods to abstract over types. So > while one might find the .NET type system "better", we must > acknowledge it doesn't provide Java developers with the tools > they're used to. (You could argue this is a feature rather than > a bug; in a world with no existing code or developer experience, > this might be true, but that's not the world we're in right now.) > > > I'm not sure "tools they're used to" is all that meaningful, IMHO. > Erased generics were added, no precedent in java prior to that. > Lambdas were added, no precedent to that beforehand. JMM was > revised, no precedent. Fork join added. Streams added. Value > types will be added. By definition, things will not always be "what > people are used to" unless the language doesn't adapt and progress. > Again, just my opinion of course. > > And this is the essential tension; a solution that retains > compatibility (not necessarily just with existing code, but > people's existing expectations of how it works) is necessarily > going to result in a "worse" language going forward. There's no > magic solution here (though there are bad solutions), the art is > putting the pain where it is felt least. > > > What does compatibility mean in this particular context? I argue > that "code compiles" is a low barrier here -- value types are a > completely new concept to java, and calling "old" code that simply > took Collection is not necessarily backcompat, for the reasons I > mentioned. Also, as I mentioned above, there's nothing wrong with > re-learning things to adjust expectations when new things are added. > > See, I don't want a "worse" language just to drag along historical > artifacts :) The language is going to outlive (if it remains > relevant and "refreshed") the vast majority of existing > libs/frameworks/etc that are being alluded to in this thread. I'm > not saying java should be evolved recklessly nor carelessly nor > ignorant of existing code, but I'd say the bar for "we're going to > make the language/type system/compiler/JVM/etc a bit 'worse' for the > sake of framework/lib/code Y" has to be very high. > > > On Thu, Jan 22, 2015 at 11:32 AM, Brian Goetz > > wrote: > > Thanks Vitaly, explicating how things are done on .NET is highly > constructive, even if we can't just "do what they did." > > On the other hand, we've seen some impassioned defense of "I > want to keep being able to use List (or raw List)" on this > list -- and .NET doesn't even have an analogue of these types, > you have to use generic methods to abstract over types. So > while one might find the .NET type system "better", we must > acknowledge it doesn't provide Java developers with the tools > they're used to. (You could argue this is a feature rather than > a bug; in a world with no existing code or developer experience, > this might be true, but that's not the world we're in right now.) > > And this is the essential tension; a solution that retains > compatibility (not necessarily just with existing code, but > people's existing expectations of how it works) is necessarily > going to result in a "worse" language going forward. There's no > magic solution here (though there are bad solutions), the art is > putting the pain where it is felt least. > > Anyone who thinks they have "the" answer to this problem > obviously doesn't understand the problem. So for those trying > to contribute: it would be more helpful if, rather than coming > at it from an "I have the answer" perspective, try something > more like "I might have a small piece of an answer." Being in > such a mental place is far more likely to result in real > contribution! > > > On 1/22/2015 10:11 AM, Vitaly Davidovich wrote: > > Ok, got you. It's detected at runtime just like > dereferencing a null or > indexing outside the bounds of an array -- nothing sound > about these, but > java's a safe language so has the safeguards. > > By the way, the issue of List not being covariant with > List > was also a drag sometimes in .NET where you knew you weren't > mutating the > list and only wanted to read the values (which is safe). > So, C# has the > in/out keywords when you define the generic type (only works > on interfaces > and delegates) -- compiler then checks your signatures: > https://msdn.microsoft.com/en-__us/library/dd469487.aspx > and > https://msdn.microsoft.com/en-__us/library/dd469484.aspx > . > > I hate to keep tooting .NET's horn on this list, but I am > impressed with > how they've addressed these various issues. > > On Thu, Jan 22, 2015 at 9:46 AM, St?phane ?pardaud > > wrote: > > On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: > > > Ok can we first agree that List is NOT List > ? :) That's > unsound since you can't add any Object in there if > things are reified. I > think you guys mean List . > > > Well, that what I meant by "variance issues > aside". It is unsound but > > detected at runtime. This is already a limitation of > List in Java at > compile-time, I'm not proposing to fix it. > > > From forax at univ-mlv.fr Thu Jan 22 19:13:57 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 22 Jan 2015 20:13:57 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C1428A.4000501@oracle.com> References: <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> <54C1261F.5030708@oracle.com> <54C1428A.4000501@oracle.com> Message-ID: <54C14BF5.8030401@univ-mlv.fr> On 01/22/2015 07:33 PM, Brian Goetz wrote: > Short answer: yes. (Some of the reasoning for this is reconstructed > in the latest State of the Specialization.) > > The key requirement was, and continues to be: gradual migration > compatibility. This has multiple facets: > - It should be practical to generify an existing class (within > reason), rather than throwing it out and starting over > - It should be practical to generify libraries gradually, for example > generifying a superclass separately from a subclass > - Generification of libraries and their clients must not require a > "flag day". It should be possible to generify the library first and > generify the client later (or not). > > We have the same requirement now; existing generic classes should be > extendable to any-fied generics, it should not be necessary to any-fy > libraries all at once (though they do have to be any-fied top-down), > and -- most importantly -- existing client code must continue to work > against any-fied libraries. > > What is meant by "continue to work" can be subtle, though. For > example, some have balked at the idea that List would continue to > mean List -- as it always has meant -- is not > "continuing to work", because it's not "what the user thought they > meant when they wrote it ten years ago." I disagree on that point, in Java, subtyping is defined on object types not on primitive types, this is why by example the following code doesn't compile even if it's technically possible to change the way bridges are generated in order to compile it. class A { double foo() { ... } } class B extends A { int foo() { ... } } The Liskov principle is defined on objects not on primitive, we have the same limitation in Java. While I agree that Java is the last language with a strong separation between primitive types and object types. It's how Java was defined, so I don't think it will surprise anyone if there is not subtyping relationship between List and List. > > But, yes, the requirement in 5 were centered around gradual migration > compatibility from non-generic to generic; we have the same > requirements when migrating from generic to "any-generic". strong backward compatibility is in the Java ADN, it's a pain, it's a curse, but it's also a blessing because it makes its evolution possible. R?mi > > > On 1/22/2015 12:46 PM, Vitaly Davidovich wrote: >> By the way, is it fair to say that one of the main reasons generics were >> implemented with erasure back in Java 5 was also for backcompat reasons? >> If so, here we are many years down the road, with lots more code, >> patterns, familiarity, etc to preserve, where that decision is making it >> harder to evolve the language (and where reification, or at least some >> form of it, is going to be done anyway, but at higher cost now >> potentially). Just saying :) >> >> On Thu, Jan 22, 2015 at 12:19 PM, Vitaly Davidovich > > wrote: >> >> I hear you Brian, and by no means think I have THE answer. >> >> On the other hand, we've seen some impassioned defense of "I >> want to keep being able to use List (or raw List)" on this >> list -- and .NET doesn't even have an analogue of these types, >> you have to use generic methods to abstract over types. So >> while one might find the .NET type system "better", we must >> acknowledge it doesn't provide Java developers with the tools >> they're used to. (You could argue this is a feature rather than >> a bug; in a world with no existing code or developer experience, >> this might be true, but that's not the world we're in right >> now.) >> >> >> I'm not sure "tools they're used to" is all that meaningful, IMHO. >> Erased generics were added, no precedent in java prior to that. >> Lambdas were added, no precedent to that beforehand. JMM was >> revised, no precedent. Fork join added. Streams added. Value >> types will be added. By definition, things will not always be "what >> people are used to" unless the language doesn't adapt and progress. >> Again, just my opinion of course. >> >> And this is the essential tension; a solution that retains >> compatibility (not necessarily just with existing code, but >> people's existing expectations of how it works) is necessarily >> going to result in a "worse" language going forward. There's no >> magic solution here (though there are bad solutions), the art is >> putting the pain where it is felt least. >> >> >> What does compatibility mean in this particular context? I argue >> that "code compiles" is a low barrier here -- value types are a >> completely new concept to java, and calling "old" code that simply >> took Collection is not necessarily backcompat, for the reasons I >> mentioned. Also, as I mentioned above, there's nothing wrong with >> re-learning things to adjust expectations when new things are added. >> >> See, I don't want a "worse" language just to drag along historical >> artifacts :) The language is going to outlive (if it remains >> relevant and "refreshed") the vast majority of existing >> libs/frameworks/etc that are being alluded to in this thread. I'm >> not saying java should be evolved recklessly nor carelessly nor >> ignorant of existing code, but I'd say the bar for "we're going to >> make the language/type system/compiler/JVM/etc a bit 'worse' for the >> sake of framework/lib/code Y" has to be very high. >> >> >> On Thu, Jan 22, 2015 at 11:32 AM, Brian Goetz >> > wrote: >> >> Thanks Vitaly, explicating how things are done on .NET is highly >> constructive, even if we can't just "do what they did." >> >> On the other hand, we've seen some impassioned defense of "I >> want to keep being able to use List (or raw List)" on this >> list -- and .NET doesn't even have an analogue of these types, >> you have to use generic methods to abstract over types. So >> while one might find the .NET type system "better", we must >> acknowledge it doesn't provide Java developers with the tools >> they're used to. (You could argue this is a feature rather than >> a bug; in a world with no existing code or developer experience, >> this might be true, but that's not the world we're in right >> now.) >> >> And this is the essential tension; a solution that retains >> compatibility (not necessarily just with existing code, but >> people's existing expectations of how it works) is necessarily >> going to result in a "worse" language going forward. There's no >> magic solution here (though there are bad solutions), the art is >> putting the pain where it is felt least. >> >> Anyone who thinks they have "the" answer to this problem >> obviously doesn't understand the problem. So for those trying >> to contribute: it would be more helpful if, rather than coming >> at it from an "I have the answer" perspective, try something >> more like "I might have a small piece of an answer." Being in >> such a mental place is far more likely to result in real >> contribution! >> >> >> On 1/22/2015 10:11 AM, Vitaly Davidovich wrote: >> >> Ok, got you. It's detected at runtime just like >> dereferencing a null or >> indexing outside the bounds of an array -- nothing sound >> about these, but >> java's a safe language so has the safeguards. >> >> By the way, the issue of List not being covariant with >> List >> was also a drag sometimes in .NET where you knew you weren't >> mutating the >> list and only wanted to read the values (which is safe). >> So, C# has the >> in/out keywords when you define the generic type (only works >> on interfaces >> and delegates) -- compiler then checks your signatures: >> https://msdn.microsoft.com/en-__us/library/dd469487.aspx >> and >> https://msdn.microsoft.com/en-__us/library/dd469484.aspx >> . >> >> I hate to keep tooting .NET's horn on this list, but I am >> impressed with >> how they've addressed these various issues. >> >> On Thu, Jan 22, 2015 at 9:46 AM, St?phane ?pardaud >> > wrote: >> >> On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: >> >> >> Ok can we first agree that List is NOT List >> ? :) That's >> unsound since you can't add any Object in there if >> things are reified. I >> think you guys mean List . >> >> >> Well, that what I meant by "variance issues >> aside". It is unsound but >> >> detected at runtime. This is already a limitation of >> List in Java at >> compile-time, I'm not proposing to fix it. >> >> >> From vitalyd at gmail.com Thu Jan 22 19:59:21 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 14:59:21 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C14BF5.8030401@univ-mlv.fr> References: <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0F94F.3020404@oracle.com> <54C0FCCA.6000100@epardaud.fr> <54C0FE38.7090905@oracle.com> <54C0FF6A.3080407@epardaud.fr> <54C0FFDB.6010205@oracle.com> <54C100D2.9000506@epardaud.fr> <54C106C8.3030801@epardaud.fr> <54C10A59.2000000@epardaud.fr> <54C10D61.5020606@epardaud.fr> <54C1261F.5030708@oracle.com> <54C1428A.4000501@oracle.com> <54C14BF5.8030401@univ-mlv.fr> Message-ID: > > I disagree on that point, in Java, subtyping is defined on object types > not on primitive types, > this is why by example the following code doesn't compile even if it's > technically possible to change > the way bridges are generated in order to compile it. > > class A { > double foo() { ... } > } > class B extends A { > int foo() { ... } > } > > The Liskov principle is defined on objects not on primitive, we have the > same limitation in Java. > > While I agree that Java is the last language with a strong separation > between primitive types and object types. It's how Java was defined, so I > don't think it will surprise anyone if there is not subtyping relationship > between List and List. > The issue at hand (in this thread) is about iterating over opaque lists (or any iterable, for that matter). The crux of the issue is that what fits most naturally with any-fied generics, void iterate(List ...), is today being written using List. And one of the (predominant?) reasons that's used today is because of late-binding/reflection where the T isn't known at compile time. I don't think anyone is claiming there's any subtype relationship between List and List from an OO perspective. On Thu, Jan 22, 2015 at 2:13 PM, Remi Forax wrote: > > On 01/22/2015 07:33 PM, Brian Goetz wrote: > >> Short answer: yes. (Some of the reasoning for this is reconstructed in >> the latest State of the Specialization.) >> >> The key requirement was, and continues to be: gradual migration >> compatibility. This has multiple facets: >> - It should be practical to generify an existing class (within reason), >> rather than throwing it out and starting over >> - It should be practical to generify libraries gradually, for example >> generifying a superclass separately from a subclass >> - Generification of libraries and their clients must not require a "flag >> day". It should be possible to generify the library first and generify the >> client later (or not). >> >> We have the same requirement now; existing generic classes should be >> extendable to any-fied generics, it should not be necessary to any-fy >> libraries all at once (though they do have to be any-fied top-down), and -- >> most importantly -- existing client code must continue to work against >> any-fied libraries. >> >> What is meant by "continue to work" can be subtle, though. For example, >> some have balked at the idea that List would continue to mean List> extends Object> -- as it always has meant -- is not "continuing to work", >> because it's not "what the user thought they meant when they wrote it ten >> years ago." >> > > I disagree on that point, in Java, subtyping is defined on object types > not on primitive types, > this is why by example the following code doesn't compile even if it's > technically possible to change > the way bridges are generated in order to compile it. > > class A { > double foo() { ... } > } > class B extends A { > int foo() { ... } > } > > The Liskov principle is defined on objects not on primitive, we have the > same limitation in Java. > > While I agree that Java is the last language with a strong separation > between primitive types and object types. It's how Java was defined, so I > don't think it will surprise anyone if there is not subtyping relationship > between List and List. > > >> But, yes, the requirement in 5 were centered around gradual migration >> compatibility from non-generic to generic; we have the same requirements >> when migrating from generic to "any-generic". >> > > strong backward compatibility is in the Java ADN, it's a pain, it's a > curse, but it's also a blessing because it makes its evolution possible. > > R?mi > > > >> >> On 1/22/2015 12:46 PM, Vitaly Davidovich wrote: >> >>> By the way, is it fair to say that one of the main reasons generics were >>> implemented with erasure back in Java 5 was also for backcompat reasons? >>> If so, here we are many years down the road, with lots more code, >>> patterns, familiarity, etc to preserve, where that decision is making it >>> harder to evolve the language (and where reification, or at least some >>> form of it, is going to be done anyway, but at higher cost now >>> potentially). Just saying :) >>> >>> On Thu, Jan 22, 2015 at 12:19 PM, Vitaly Davidovich >> > wrote: >>> >>> I hear you Brian, and by no means think I have THE answer. >>> >>> On the other hand, we've seen some impassioned defense of "I >>> want to keep being able to use List (or raw List)" on this >>> list -- and .NET doesn't even have an analogue of these types, >>> you have to use generic methods to abstract over types. So >>> while one might find the .NET type system "better", we must >>> acknowledge it doesn't provide Java developers with the tools >>> they're used to. (You could argue this is a feature rather than >>> a bug; in a world with no existing code or developer experience, >>> this might be true, but that's not the world we're in right now.) >>> >>> >>> I'm not sure "tools they're used to" is all that meaningful, IMHO. >>> Erased generics were added, no precedent in java prior to that. >>> Lambdas were added, no precedent to that beforehand. JMM was >>> revised, no precedent. Fork join added. Streams added. Value >>> types will be added. By definition, things will not always be "what >>> people are used to" unless the language doesn't adapt and progress. >>> Again, just my opinion of course. >>> >>> And this is the essential tension; a solution that retains >>> compatibility (not necessarily just with existing code, but >>> people's existing expectations of how it works) is necessarily >>> going to result in a "worse" language going forward. There's no >>> magic solution here (though there are bad solutions), the art is >>> putting the pain where it is felt least. >>> >>> >>> What does compatibility mean in this particular context? I argue >>> that "code compiles" is a low barrier here -- value types are a >>> completely new concept to java, and calling "old" code that simply >>> took Collection is not necessarily backcompat, for the reasons I >>> mentioned. Also, as I mentioned above, there's nothing wrong with >>> re-learning things to adjust expectations when new things are added. >>> >>> See, I don't want a "worse" language just to drag along historical >>> artifacts :) The language is going to outlive (if it remains >>> relevant and "refreshed") the vast majority of existing >>> libs/frameworks/etc that are being alluded to in this thread. I'm >>> not saying java should be evolved recklessly nor carelessly nor >>> ignorant of existing code, but I'd say the bar for "we're going to >>> make the language/type system/compiler/JVM/etc a bit 'worse' for the >>> sake of framework/lib/code Y" has to be very high. >>> >>> >>> On Thu, Jan 22, 2015 at 11:32 AM, Brian Goetz >>> > wrote: >>> >>> Thanks Vitaly, explicating how things are done on .NET is highly >>> constructive, even if we can't just "do what they did." >>> >>> On the other hand, we've seen some impassioned defense of "I >>> want to keep being able to use List (or raw List)" on this >>> list -- and .NET doesn't even have an analogue of these types, >>> you have to use generic methods to abstract over types. So >>> while one might find the .NET type system "better", we must >>> acknowledge it doesn't provide Java developers with the tools >>> they're used to. (You could argue this is a feature rather than >>> a bug; in a world with no existing code or developer experience, >>> this might be true, but that's not the world we're in right now.) >>> >>> And this is the essential tension; a solution that retains >>> compatibility (not necessarily just with existing code, but >>> people's existing expectations of how it works) is necessarily >>> going to result in a "worse" language going forward. There's no >>> magic solution here (though there are bad solutions), the art is >>> putting the pain where it is felt least. >>> >>> Anyone who thinks they have "the" answer to this problem >>> obviously doesn't understand the problem. So for those trying >>> to contribute: it would be more helpful if, rather than coming >>> at it from an "I have the answer" perspective, try something >>> more like "I might have a small piece of an answer." Being in >>> such a mental place is far more likely to result in real >>> contribution! >>> >>> >>> On 1/22/2015 10:11 AM, Vitaly Davidovich wrote: >>> >>> Ok, got you. It's detected at runtime just like >>> dereferencing a null or >>> indexing outside the bounds of an array -- nothing sound >>> about these, but >>> java's a safe language so has the safeguards. >>> >>> By the way, the issue of List not being covariant with >>> List >>> was also a drag sometimes in .NET where you knew you weren't >>> mutating the >>> list and only wanted to read the values (which is safe). >>> So, C# has the >>> in/out keywords when you define the generic type (only works >>> on interfaces >>> and delegates) -- compiler then checks your signatures: >>> https://msdn.microsoft.com/en-__us/library/dd469487.aspx >>> and >>> https://msdn.microsoft.com/en-__us/library/dd469484.aspx >>> . >>> >>> I hate to keep tooting .NET's horn on this list, but I am >>> impressed with >>> how they've addressed these various issues. >>> >>> On Thu, Jan 22, 2015 at 9:46 AM, St?phane ?pardaud >>> > wrote: >>> >>> On 01/22/2015 03:42 PM, Vitaly Davidovich wrote: >>> >>> >>> Ok can we first agree that List is NOT List >>> ? :) That's >>> unsound since you can't add any Object in there if >>> things are reified. I >>> think you guys mean List . >>> >>> >>> Well, that what I meant by "variance issues >>> aside". It is unsound but >>> >>> detected at runtime. This is already a limitation of >>> List in Java at >>> compile-time, I'm not proposing to fix it. >>> >>> >>> >>> > From Mohammad.Rezaei at gs.com Thu Jan 22 22:30:55 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Thu, 22 Jan 2015 17:30:55 -0500 Subject: What happened to the "mutable struct" debate? Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> I recently joined the list and I can't find any references to the justification behind restricting value types to final fields (aka immutable). As Brian says (http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-January/000734.html), "There are some people who would prefer something more like mutable structs" and yet, the value types proposal (http://cr.openjdk.java.net/~jrose/values/values-0.html) takes immutability as a given. Is the issue considered settled? Is the justification documented? Thanks Moh From brian.goetz at oracle.com Thu Jan 22 22:36:44 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 22 Jan 2015 17:36:44 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: <54C17B7C.5060300@oracle.com> Hi Moh! Good to have you here. The sad answer is: we've got a long list of "road not taken" items for which we intend to write up our thoughts on why those roads were not taken. As you can imagine, though, such writeups compete for time with many other activities, and so they often lag behind... On 1/22/2015 5:30 PM, Rezaei, Mohammad A. wrote: > I recently joined the list and I can't find any references to the justification behind restricting value types to final fields (aka immutable). > As Brian says (http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-January/000734.html), "There are some people who would prefer something more like mutable structs" and yet, the value types proposal (http://cr.openjdk.java.net/~jrose/values/values-0.html) takes immutability as a given. > > Is the issue considered settled? Is the justification documented? > > Thanks > Moh > From apurtell at apache.org Fri Jan 23 03:31:06 2015 From: apurtell at apache.org (Andrew Purtell) Date: Thu, 22 Jan 2015 19:31:06 -0800 Subject: What happened to the "mutable struct" debate? In-Reply-To: <54C17B7C.5060300@oracle.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> Message-ID: I've also recently arrived here. This would seem to be a fairly important writeup left wanting. Our mantra is ?codes like a class, works like an int.? Stating the obvious, from a newcomer's perspective, ints are mutable. In my part of the Java universe, we are under pressure because data volume, response latency requirements, and arrival rates are too high for copying the bulk of data (in GC or working with immutable data structures). This class of problem is encompassing more and more of the so-called big data space as time goes on, and the best available existing strategies for a Java application is a resort to kludgy custom representations with primitive arrays or "structs" stored off-heap (with use of Unsafe for allocation and direct access). Java cannot provide object-like views over data that is simply too bulky for copying around. The impedance mismatch suggests consideration of alternatives. Some in the space are abandoning Java for alternatives like C++, C#, and Go. Maybe I fundamentally misunderstand the intent and usefulness of value types and they cannot be an answer for this, but more information available in this regard would be very helpful, especially if there is another option. I think increasingly the assumption is Java will concede the space of big data applications to other languages. On Thu, Jan 22, 2015 at 2:36 PM, Brian Goetz wrote: > Hi Moh! Good to have you here. > > The sad answer is: we've got a long list of "road not taken" items for > which we intend to write up our thoughts on why those roads were not > taken. As you can imagine, though, such writeups compete for time with > many other activities, and so they often lag behind... > > > > On 1/22/2015 5:30 PM, Rezaei, Mohammad A. wrote: > >> I recently joined the list and I can't find any references to the >> justification behind restricting value types to final fields (aka >> immutable). >> As Brian says (http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- >> January/000734.html), "There are some people who would prefer something >> more like mutable structs" and yet, the value types proposal ( >> http://cr.openjdk.java.net/~jrose/values/values-0.html) takes >> immutability as a given. >> >> Is the issue considered settled? Is the justification documented? >> >> Thanks >> Moh >> >> -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) From vitalyd at gmail.com Fri Jan 23 03:51:43 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 22 Jan 2015 22:51:43 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> Message-ID: So let me say that I'm one of the people that would like to see mutable value types *offered*, not encouraged or promoted for general use cases. Also, ints aren't mutable :) Variables holding the int are (e.g. locals, fields), but not ints themselves. If Brian & co could somehow be swayed to add them, I suspect this would drag in the requirement that you should be able to pass them by reference. Without that, they're of marginal utility since you can't avoid copying costs. Passing things by reference would be yet another entirely new concept to java, and given some of the sentiment I've read on this list thus far, this would comatose some people. I'm with you in terms of being annoyed with having to manually unroll data structures/abstractions to get performance. I'll still take immutable value types with a welcome sign, but alas, the mutable aspect doesn't seem promising. P.S. this is the first I hear of anyone moving away from java to C# or Go in this space (Go, in particular, has its own problems with a subpar GC as far as I know). Do share. sent from my phone On Jan 22, 2015 10:32 PM, "Andrew Purtell" wrote: > I've also recently arrived here. This would seem to be a fairly important > writeup left wanting. > > Our mantra is ?codes like a class, works like an int.? > > > Stating the obvious, from a newcomer's perspective, ints are mutable. > > In my part of the Java universe, we are under pressure because data volume, > response latency requirements, and arrival rates are too high for copying > the bulk of data (in GC or working with immutable data structures). This > class of problem is encompassing more and more of the so-called big data > space as time goes on, and the best available existing strategies for a > Java application is a resort to kludgy custom representations with > primitive arrays or "structs" stored off-heap (with use of Unsafe for > allocation and direct access). Java cannot provide object-like views over > data that is simply too bulky for copying around. The impedance mismatch > suggests consideration of alternatives. Some in the space are abandoning > Java for alternatives like C++, C#, and Go. Maybe I fundamentally > misunderstand the intent and usefulness of value types and they cannot be > an answer for this, but more information available in this regard would be > very helpful, especially if there is another option. I think increasingly > the assumption is Java will concede the space of big data applications to > other languages. > > > On Thu, Jan 22, 2015 at 2:36 PM, Brian Goetz > wrote: > > > Hi Moh! Good to have you here. > > > > The sad answer is: we've got a long list of "road not taken" items for > > which we intend to write up our thoughts on why those roads were not > > taken. As you can imagine, though, such writeups compete for time with > > many other activities, and so they often lag behind... > > > > > > > > On 1/22/2015 5:30 PM, Rezaei, Mohammad A. wrote: > > > >> I recently joined the list and I can't find any references to the > >> justification behind restricting value types to final fields (aka > >> immutable). > >> As Brian says ( > http://mail.openjdk.java.net/pipermail/valhalla-dev/2015- > >> January/000734.html), "There are some people who would prefer something > >> more like mutable structs" and yet, the value types proposal ( > >> http://cr.openjdk.java.net/~jrose/values/values-0.html) takes > >> immutability as a given. > >> > >> Is the issue considered settled? Is the justification documented? > >> > >> Thanks > >> Moh > >> > >> > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > From mark at talios.com Fri Jan 23 04:16:06 2015 From: mark at talios.com (Mark Derricutt) Date: Fri, 23 Jan 2015 17:16:06 +1300 Subject: What happened to the "mutable struct" debate? In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> Message-ID: <86F31685-FEEF-4741-BADD-520F3B272AC0@talios.com> On 23 Jan 2015, at 16:31, Andrew Purtell wrote: > Our mantra is ?codes like a class, works like an int.? > Stating the obvious, from a newcomer's perspective, ints are mutable. Surely it's the *reference* to the int that's mutable, not the int itself. 5 = 0; 10 / 5 -> DivByZero? I think ( and hope ) not. int v = 5 v = 0 10 / v -> DivByZero. -- Mark Derricutt http://www.theoryinpractice.net http://plus.google.com/+MarkDerricutt http://twitter.com/talios http://facebook.com/mderricutt From brian.goetz at oracle.com Fri Jan 23 04:38:33 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 22 Jan 2015 23:38:33 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> Message-ID: <54C1D049.7000108@oracle.com> > If Brian & co could somehow be swayed to add them, I suspect this would > drag in the requirement that you should be able to pass them by reference. > Without that, they're of marginal utility since you can't avoid copying > costs. Passing things by reference would be yet another entirely new > concept to java, and given some of the sentiment I've read on this list > thus far, this would comatose some people. Very much so. This is indeed Big Strike #1 against them; this is a new big complexity that Java (platform and developers) have not had to deal with. Which leads directly to Big Strike #2: passing mutable structs by reference leads to interior pointers (pointers into the middle of objects), which is a huge additional burden for the GC. (I'm told this is one of the major reasons why .NET GC is so much slower than the JVM, because they have to deal with this additional requirement.) From brian.goetz at oracle.com Fri Jan 23 04:42:47 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 22 Jan 2015 23:42:47 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> Message-ID: <54C1D147.20008@oracle.com> > Stating the obvious, from a newcomer's perspective, ints are mutable. Except they're not :( ints are immutable; you can't change the value 3. Two threes are the same int, regardless of where they are stored. You can have a *mutable* cell that holds an int; similarly, you can have a mutable cell that holds a value. From vitalyd at gmail.com Fri Jan 23 05:00:49 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 23 Jan 2015 00:00:49 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: <54C1D049.7000108@oracle.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> Message-ID: Brian, Where did you hear this? :) It's true that interior pointers slow down marking as you need to find the beginning of the object, but it's pay-as-you-go in that you pay this cost for interior pointers only. To reduce # of such pointers, CLR allows such pointers on-stack only (i.e. you cannot have interior pointers in objects on the heap), so their number should be pretty small in typical code. I don't think this impacts much else, and I'd be very surprised if this is one of the *major* reasons for the difference. But, perhaps I'm missing something, in which case, please do tell. sent from my phone sent from my phone On Jan 22, 2015 11:40 PM, "Brian Goetz" wrote: > If Brian & co could somehow be swayed to add them, I suspect this would >> drag in the requirement that you should be able to pass them by reference. >> Without that, they're of marginal utility since you can't avoid copying >> costs. Passing things by reference would be yet another entirely new >> concept to java, and given some of the sentiment I've read on this list >> thus far, this would comatose some people. >> > > Very much so. This is indeed Big Strike #1 against them; this is a new > big complexity that Java (platform and developers) have not had to deal > with. > > Which leads directly to Big Strike #2: passing mutable structs by > reference leads to interior pointers (pointers into the middle of objects), > which is a huge additional burden for the GC. (I'm told this is one of the > major reasons why .NET GC is so much slower than the JVM, because they have > to deal with this additional requirement.) > > From Mohammad.Rezaei at gs.com Fri Jan 23 15:11:36 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Fri, 23 Jan 2015 10:11:36 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: <54C1D147.20008@oracle.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D147.20008@oracle.com> Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A73@GSCMAMP06EX.firmwide.corp.gs.com> I disagree with this linguistic characterization. There is no such thing as a "cell" in the JLS. There are only two things defined there: 1) Primitive Value (per JLS 4.2) 2) Variable of Primitive Type (per JLS 4.12.1) In any reasonable English interpretation, and in common usage, both are just called "primitive". Said differently, the answer to the question "given the statement 'int x;', is x a primitive?" would be "yes". So the question "are primitives mutable?" must address both (1) and (2). The only logically consistent interpretation here is : 1) Primitive Values are immutable. 2) Variables of Primitive type are mutable (except when declared final). 3) Primitives (meaning either (1) or (2)) is therefore mutable. Brian's language uses "int" instead of "primitive". I would suggest asking the question "given the statement 'int x;' is x an int?" If the answer is "no", something has gone horribly wrong. A very good answer would be "yes, x is variable of int type", which fixes the linguistic ambiguity of "int". Thanks Moh >-----Original Message----- >From: valhalla-dev [mailto:valhalla-dev-bounces at openjdk.java.net] On Behalf Of >Brian Goetz >Sent: Thursday, January 22, 2015 11:43 PM >To: Andrew Purtell; valhalla-dev at openjdk.java.net >Subject: Re: What happened to the "mutable struct" debate? > >> Stating the obvious, from a newcomer's perspective, ints are mutable. > >Except they're not :( > >ints are immutable; you can't change the value 3. Two threes are the >same int, regardless of where they are stored. You can have a *mutable* >cell that holds an int; similarly, you can have a mutable cell that >holds a value. From Mohammad.Rezaei at gs.com Fri Jan 23 15:54:09 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Fri, 23 Jan 2015 10:54:09 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: <54C1D049.7000108@oracle.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> >Very much so. This is indeed Big Strike #1 against them; this is a new >big complexity that Java (platform and developers) have not had to deal >with. > I don't really see why mutable structs would necessitate pass by reference. Java developers are used to pass by value semantics and there is a well-known, but admittedly ugly pattern for pass by reference (use an array or mutable wrapper). The reference question comes up even in all-final value types. But I'll pose that question in a separate thread. Thanks Moh From vitalyd at gmail.com Fri Jan 23 16:00:03 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 23 Jan 2015 11:00:03 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: If mutable structs are not allowed to be passed by ref, then one is more likely to lose writes as the struct is passed through a method chain. There's no *necessity* per say, but the danger zone is expanded. The by-ref would be useful for immutable structs as well when you want to avoid copying costs for a largish struct. On Fri, Jan 23, 2015 at 10:54 AM, Rezaei, Mohammad A. < Mohammad.Rezaei at gs.com> wrote: > >Very much so. This is indeed Big Strike #1 against them; this is a new > >big complexity that Java (platform and developers) have not had to deal > >with. > > > > I don't really see why mutable structs would necessitate pass by > reference. Java developers are used to pass by value semantics and there is > a well-known, but admittedly ugly pattern for pass by reference (use an > array or mutable wrapper). > > The reference question comes up even in all-final value types. But I'll > pose that question in a separate thread. > > Thanks > Moh > From vitalyd at gmail.com Fri Jan 23 16:06:18 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 23 Jan 2015 11:06:18 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: Also, here's another thing to consider for mutable structs: MyStruct[] arr = ... arr[i].intField = 10; Vs MyStruct[] arr = ... MyStruct s = arr[i]; s.intField = 10; These wouldn't have the same semantic yet not super obvious to some from cursory glance (or innocent refactoring). sent from my phone On Jan 23, 2015 11:00 AM, "Vitaly Davidovich" wrote: > If mutable structs are not allowed to be passed by ref, then one is more > likely to lose writes as the struct is passed through a method chain. > There's no *necessity* per say, but the danger zone is expanded. > > The by-ref would be useful for immutable structs as well when you want to > avoid copying costs for a largish struct. > > On Fri, Jan 23, 2015 at 10:54 AM, Rezaei, Mohammad A. < > Mohammad.Rezaei at gs.com> wrote: > >> >Very much so. This is indeed Big Strike #1 against them; this is a new >> >big complexity that Java (platform and developers) have not had to deal >> >with. >> > >> >> I don't really see why mutable structs would necessitate pass by >> reference. Java developers are used to pass by value semantics and there is >> a well-known, but admittedly ugly pattern for pass by reference (use an >> array or mutable wrapper). >> >> The reference question comes up even in all-final value types. But I'll >> pose that question in a separate thread. >> >> Thanks >> Moh >> > > From timo.kinnunen at gmail.com Fri Jan 23 16:08:10 2015 From: timo.kinnunen at gmail.com (=?utf-8?Q?Timo_Kinnunen?=) Date: Fri, 23 Jan 2015 16:08:10 +0000 Subject: =?utf-8?Q?Re:_What_happened_to_the_"mutable_struct"_debate=3F?= In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com>, Message-ID: <54c27298.e40cc30a.66bd.ffffe9b5@mx.google.com> Pass-by-ref is easily emulated by boxing a mutable struct and then operating on the boxed struct. -- Have a nice day, Timo. Sent from Windows Mail From: Vitaly Davidovich Sent: ?Friday?, ?January? ?23?, ?2015 ?17?:?00 To: Rezaei, Mohammad A. Cc: valhalla-dev at openjdk.java.net If mutable structs are not allowed to be passed by ref, then one is more likely to lose writes as the struct is passed through a method chain. There's no *necessity* per say, but the danger zone is expanded. The by-ref would be useful for immutable structs as well when you want to avoid copying costs for a largish struct. On Fri, Jan 23, 2015 at 10:54 AM, Rezaei, Mohammad A. < Mohammad.Rezaei at gs.com> wrote: > >Very much so. This is indeed Big Strike #1 against them; this is a new > >big complexity that Java (platform and developers) have not had to deal > >with. > > > > I don't really see why mutable structs would necessitate pass by > reference. Java developers are used to pass by value semantics and there is > a well-known, but admittedly ugly pattern for pass by reference (use an > array or mutable wrapper). > > The reference question comes up even in all-final value types. But I'll > pose that question in a separate thread. > > Thanks > Moh > From Mohammad.Rezaei at gs.com Fri Jan 23 16:12:32 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Fri, 23 Jan 2015 11:12:32 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A88@GSCMAMP06EX.firmwide.corp.gs.com> I don?t believe this is confusing. It?s not confusing with Java objects. MyObject foo = new MyObject(); foo.inField = 10; vs. MyObject foo = new MyObject(); int x = foo.intField; x = 10; The rules are simple and the exactly the same as regular primitives: - Assignment is copy. - Pass into method is copy. - return from a method is copy. - dot (.) is not a copy. Incidentally, I?m not a C# expert, but I believe the 4th rule doesn?t apply there and it causes much confusion. Thanks Moh From: Vitaly Davidovich [mailto:vitalyd at gmail.com] Sent: Friday, January 23, 2015 11:06 AM To: Rezaei, Mohammad A. [Tech] Cc: Brian Goetz; valhalla-dev at openjdk.java.net Subject: Re: What happened to the "mutable struct" debate? Also, here's another thing to consider for mutable structs: MyStruct[] arr = ... arr[i].intField = 10; Vs MyStruct[] arr = ... MyStruct s = arr[i]; s.intField = 10; These wouldn't have the same semantic yet not super obvious to some from cursory glance (or innocent refactoring). sent from my phone On Jan 23, 2015 11:00 AM, "Vitaly Davidovich" > wrote: If mutable structs are not allowed to be passed by ref, then one is more likely to lose writes as the struct is passed through a method chain. There's no *necessity* per say, but the danger zone is expanded. The by-ref would be useful for immutable structs as well when you want to avoid copying costs for a largish struct. On Fri, Jan 23, 2015 at 10:54 AM, Rezaei, Mohammad A. > wrote: >Very much so. This is indeed Big Strike #1 against them; this is a new >big complexity that Java (platform and developers) have not had to deal >with. > I don't really see why mutable structs would necessitate pass by reference. Java developers are used to pass by value semantics and there is a well-known, but admittedly ugly pattern for pass by reference (use an array or mutable wrapper). The reference question comes up even in all-final value types. But I'll pose that question in a separate thread. Thanks Moh From vitalyd at gmail.com Fri Jan 23 16:12:57 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 23 Jan 2015 11:12:57 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: <54c27298.e40cc30a.66bd.ffffe9b5@mx.google.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> <54c27298.e40cc30a.66bd.ffffe9b5@mx.google.com> Message-ID: The point is to avoid allocation! :) sent from my phone On Jan 23, 2015 11:11 AM, "Timo Kinnunen" wrote: > Pass-by-ref is easily emulated by boxing a mutable struct and then > operating on the boxed struct. > > > > > -- > Have a nice day, > Timo. > > Sent from Windows Mail > > *From:* Vitaly Davidovich > *Sent:* ?Friday?, ?January? ?23?, ?2015 ?17?:?00 > *To:* Rezaei, Mohammad A. > *Cc:* valhalla-dev at openjdk.java.net > > If mutable structs are not allowed to be passed by ref, then one is more > likely to lose writes as the struct is passed through a method chain. > There's no *necessity* per say, but the danger zone is expanded. > > The by-ref would be useful for immutable structs as well when you want to > avoid copying costs for a largish struct. > > On Fri, Jan 23, 2015 at 10:54 AM, Rezaei, Mohammad A. < > Mohammad.Rezaei at gs.com> wrote: > > > >Very much so. This is indeed Big Strike #1 against them; this is a new > > >big complexity that Java (platform and developers) have not had to deal > > >with. > > > > > > > I don't really see why mutable structs would necessitate pass by > > reference. Java developers are used to pass by value semantics and there > is > > a well-known, but admittedly ugly pattern for pass by reference (use an > > array or mutable wrapper). > > > > The reference question comes up even in all-final value types. But I'll > > pose that question in a separate thread. > > > > Thanks > > Moh > > > From vitalyd at gmail.com Fri Jan 23 16:15:02 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 23 Jan 2015 11:15:02 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A88@GSCMAMP06EX.firmwide.corp.gs.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A88@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: I don't think this is a valid example; rewrite what I wrote with a class and you get different behavior and everything looks the same. sent from my phone On Jan 23, 2015 11:12 AM, "Rezaei, Mohammad A." wrote: > I don?t believe this is confusing. It?s not confusing with Java objects. > > > > MyObject foo = new MyObject(); > > foo.inField = 10; > > > > vs. > > > > MyObject foo = new MyObject(); > > int x = foo.intField; > > x = 10; > > > > The rules are simple and the exactly the same as regular primitives: > > - Assignment is copy. > > - Pass into method is copy. > > - return from a method is copy. > > - dot (.) is not a copy. > > > > Incidentally, I?m not a C# expert, but I believe the 4th rule doesn?t > apply there and it causes much confusion. > > > > Thanks > > Moh > > > > *From:* Vitaly Davidovich [mailto:vitalyd at gmail.com] > *Sent:* Friday, January 23, 2015 11:06 AM > *To:* Rezaei, Mohammad A. [Tech] > *Cc:* Brian Goetz; valhalla-dev at openjdk.java.net > *Subject:* Re: What happened to the "mutable struct" debate? > > > > Also, here's another thing to consider for mutable structs: > > MyStruct[] arr = ... > arr[i].intField = 10; > > Vs > MyStruct[] arr = ... > MyStruct s = arr[i]; > s.intField = 10; > > These wouldn't have the same semantic yet not super obvious to some from > cursory glance (or innocent refactoring). > > sent from my phone > > On Jan 23, 2015 11:00 AM, "Vitaly Davidovich" wrote: > > If mutable structs are not allowed to be passed by ref, then one is more > likely to lose writes as the struct is passed through a method chain. > There's no *necessity* per say, but the danger zone is expanded. > > > > The by-ref would be useful for immutable structs as well when you want to > avoid copying costs for a largish struct. > > > > On Fri, Jan 23, 2015 at 10:54 AM, Rezaei, Mohammad A. < > Mohammad.Rezaei at gs.com> wrote: > > >Very much so. This is indeed Big Strike #1 against them; this is a new > >big complexity that Java (platform and developers) have not had to deal > >with. > > > > I don't really see why mutable structs would necessitate pass by > reference. Java developers are used to pass by value semantics and there is > a well-known, but admittedly ugly pattern for pass by reference (use an > array or mutable wrapper). > > The reference question comes up even in all-final value types. But I'll > pose that question in a separate thread. > > Thanks > Moh > > > From timo.kinnunen at gmail.com Fri Jan 23 16:14:33 2015 From: timo.kinnunen at gmail.com (=?utf-8?Q?Timo_Kinnunen?=) Date: Fri, 23 Jan 2015 16:14:33 +0000 Subject: =?utf-8?Q?Re:_What_happened_to_the_"mutable_struct"_debate=3F?= In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> <54c27298.e40cc30a.66bd.ffffe9b5@mx.google.com>, Message-ID: <54c273ca.6169b40a.35ff.ffffdbd1@mx.google.com> I think that?s too early to say because we don?t know how boxing of values will be implemented. -- Have a nice day, Timo. Sent from Windows Mail From: Vitaly Davidovich Sent: ?Friday?, ?January? ?23?, ?2015 ?17?:?12 To: Timo Kinnunen Cc: valhalla-dev at openjdk.java.net, Rezaei, Mohammad A. The point is to avoid allocation! :) sent from my phone On Jan 23, 2015 11:11 AM, "Timo Kinnunen" wrote: Pass-by-ref is easily emulated by boxing a mutable struct and then operating on the boxed struct. -- Have a nice day, Timo. Sent from Windows Mail From: Vitaly Davidovich Sent: ?Friday?, ?January? ?23?, ?2015 ?17?:?00 To: Rezaei, Mohammad A. Cc: valhalla-dev at openjdk.java.net If mutable structs are not allowed to be passed by ref, then one is more likely to lose writes as the struct is passed through a method chain. There's no *necessity* per say, but the danger zone is expanded. The by-ref would be useful for immutable structs as well when you want to avoid copying costs for a largish struct. On Fri, Jan 23, 2015 at 10:54 AM, Rezaei, Mohammad A. < Mohammad.Rezaei at gs.com> wrote: > >Very much so. This is indeed Big Strike #1 against them; this is a new > >big complexity that Java (platform and developers) have not had to deal > >with. > > > > I don't really see why mutable structs would necessitate pass by > reference. Java developers are used to pass by value semantics and there is > a well-known, but admittedly ugly pattern for pass by reference (use an > array or mutable wrapper). > > The reference question comes up even in all-final value types. But I'll > pose that question in a separate thread. > > Thanks > Moh > From simon at ochsenreither.de Fri Jan 23 16:17:03 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Fri, 23 Jan 2015 17:17:03 +0100 (CET) Subject: What happened to the "mutable struct" debate? In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: <883637020.154877.1422029823871.JavaMail.open-xchange@srv005.service.ps-server.net> > If mutable structs are not allowed to be passed by ref, then one is more > likely to lose writes as the struct is passed through a method chain. > There's no *necessity* per say, but the danger zone is expanded. +1. Not sure why people are eager to repeat a mistake which is deeply regretted by its inventors. Together with not having statics with class-level generics, it's one of the few things which are better than .NET. From vitalyd at gmail.com Fri Jan 23 16:17:29 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 23 Jan 2015 11:17:29 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A88@GSCMAMP06EX.firmwide.corp.gs.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A88@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: Also, dot is not a copy in C# unless I'm missing your point. sent from my phone On Jan 23, 2015 11:12 AM, "Rezaei, Mohammad A." wrote: > I don?t believe this is confusing. It?s not confusing with Java objects. > > > > MyObject foo = new MyObject(); > > foo.inField = 10; > > > > vs. > > > > MyObject foo = new MyObject(); > > int x = foo.intField; > > x = 10; > > > > The rules are simple and the exactly the same as regular primitives: > > - Assignment is copy. > > - Pass into method is copy. > > - return from a method is copy. > > - dot (.) is not a copy. > > > > Incidentally, I?m not a C# expert, but I believe the 4th rule doesn?t > apply there and it causes much confusion. > > > > Thanks > > Moh > > > > *From:* Vitaly Davidovich [mailto:vitalyd at gmail.com] > *Sent:* Friday, January 23, 2015 11:06 AM > *To:* Rezaei, Mohammad A. [Tech] > *Cc:* Brian Goetz; valhalla-dev at openjdk.java.net > *Subject:* Re: What happened to the "mutable struct" debate? > > > > Also, here's another thing to consider for mutable structs: > > MyStruct[] arr = ... > arr[i].intField = 10; > > Vs > MyStruct[] arr = ... > MyStruct s = arr[i]; > s.intField = 10; > > These wouldn't have the same semantic yet not super obvious to some from > cursory glance (or innocent refactoring). > > sent from my phone > > On Jan 23, 2015 11:00 AM, "Vitaly Davidovich" wrote: > > If mutable structs are not allowed to be passed by ref, then one is more > likely to lose writes as the struct is passed through a method chain. > There's no *necessity* per say, but the danger zone is expanded. > > > > The by-ref would be useful for immutable structs as well when you want to > avoid copying costs for a largish struct. > > > > On Fri, Jan 23, 2015 at 10:54 AM, Rezaei, Mohammad A. < > Mohammad.Rezaei at gs.com> wrote: > > >Very much so. This is indeed Big Strike #1 against them; this is a new > >big complexity that Java (platform and developers) have not had to deal > >with. > > > > I don't really see why mutable structs would necessitate pass by > reference. Java developers are used to pass by value semantics and there is > a well-known, but admittedly ugly pattern for pass by reference (use an > array or mutable wrapper). > > The reference question comes up even in all-final value types. But I'll > pose that question in a separate thread. > > Thanks > Moh > > > From vitalyd at gmail.com Fri Jan 23 16:20:50 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 23 Jan 2015 11:20:50 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: <883637020.154877.1422029823871.JavaMail.open-xchange@srv005.service.ps-server.net> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> <883637020.154877.1422029823871.JavaMail.open-xchange@srv005.service.ps-server.net> Message-ID: For the record (again), I'd support and like mutable structs in java. They wouldn't be used often (and aren't in .NET) but they're useful for some scenarios. Yes, they're subtle at times but so is, e.g., multithreading. sent from my phone On Jan 23, 2015 11:17 AM, "Simon Ochsenreither" wrote: > > If mutable structs are not allowed to be passed by ref, then one is more > > likely to lose writes as the struct is passed through a method chain. > > There's no *necessity* per say, but the danger zone is expanded. > > +1. > > Not sure why people are eager to repeat a mistake which is deeply > regretted by > its inventors. Together with not having statics with class-level generics, > it's > one of the few things which are better than .NET. > From Mohammad.Rezaei at gs.com Fri Jan 23 16:25:46 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Fri, 23 Jan 2015 11:25:46 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A88@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A8A@GSCMAMP06EX.firmwide.corp.gs.com> Sorry, I?m not a C# practitioner, but I?ve been reading about it because they supposedly had issues with mutable structs. According to http://blogs.msdn.com/b/ericlippert/archive/2008/05/14/mutating-readonly-structs.aspx , ?accessing a value type gives you a COPY of the value. When you say t.m, you get a copy of whatever is presently stored in m? That sure sounds like Dot is a copy. Thanks Moh From: Vitaly Davidovich [mailto:vitalyd at gmail.com] Sent: Friday, January 23, 2015 11:17 AM To: Rezaei, Mohammad A. [Tech] Cc: valhalla-dev at openjdk.java.net Subject: RE: What happened to the "mutable struct" debate? Also, dot is not a copy in C# unless I'm missing your point. sent from my phone On Jan 23, 2015 11:12 AM, "Rezaei, Mohammad A." > wrote: I don?t believe this is confusing. It?s not confusing with Java objects. MyObject foo = new MyObject(); foo.inField = 10; vs. MyObject foo = new MyObject(); int x = foo.intField; x = 10; The rules are simple and the exactly the same as regular primitives: - Assignment is copy. - Pass into method is copy. - return from a method is copy. - dot (.) is not a copy. Incidentally, I?m not a C# expert, but I believe the 4th rule doesn?t apply there and it causes much confusion. Thanks Moh From: Vitaly Davidovich [mailto:vitalyd at gmail.com] Sent: Friday, January 23, 2015 11:06 AM To: Rezaei, Mohammad A. [Tech] Cc: Brian Goetz; valhalla-dev at openjdk.java.net Subject: Re: What happened to the "mutable struct" debate? Also, here's another thing to consider for mutable structs: MyStruct[] arr = ... arr[i].intField = 10; Vs MyStruct[] arr = ... MyStruct s = arr[i]; s.intField = 10; These wouldn't have the same semantic yet not super obvious to some from cursory glance (or innocent refactoring). sent from my phone On Jan 23, 2015 11:00 AM, "Vitaly Davidovich" > wrote: If mutable structs are not allowed to be passed by ref, then one is more likely to lose writes as the struct is passed through a method chain. There's no *necessity* per say, but the danger zone is expanded. The by-ref would be useful for immutable structs as well when you want to avoid copying costs for a largish struct. On Fri, Jan 23, 2015 at 10:54 AM, Rezaei, Mohammad A. > wrote: >Very much so. This is indeed Big Strike #1 against them; this is a new >big complexity that Java (platform and developers) have not had to deal >with. > I don't really see why mutable structs would necessitate pass by reference. Java developers are used to pass by value semantics and there is a well-known, but admittedly ugly pattern for pass by reference (use an array or mutable wrapper). The reference question comes up even in all-final value types. But I'll pose that question in a separate thread. Thanks Moh From vitalyd at gmail.com Fri Jan 23 16:38:31 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 23 Jan 2015 11:38:31 -0500 Subject: What happened to the "mutable struct" debate? In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A8A@GSCMAMP06EX.firmwide.corp.gs.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A3F@GSCMAMP06EX.firmwide.corp.gs.com> <54C17B7C.5060300@oracle.com> <54C1D049.7000108@oracle.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A7E@GSCMAMP06EX.firmwide.corp.gs.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A88@GSCMAMP06EX.firmwide.corp.gs.com> <6882C9A35DFB9B4FA2779F7BF5B9757D206C009A8A@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: Ah, you're talking about *readonly* interaction (I thought you meant dot notation in general). Well, this is specific to readonly struct fields, and that's why myStructArray[i].X = 10 actually mutates the X member of the struct and not of a copy (the values in the array are not readonly, obviously). It's just that with *readonly* struct members in a class C, C.struct *always* returns a copy. Yes, there are subtleties involved, and that's why mutable structs are *not* preferred, but forbidding them outright is a bit draconian (IMHO). They (MSFT) could've decided to not give this treatment to readonly structs though, and I'm not sure why they did it this way other than to perhaps fit their general CLI spec: ..if the field is readonly and the reference occurs outside an instance > constructor of the class in which the field is declared, then the result is > a value, namely the value of the field I in the object referenced by E. On Fri, Jan 23, 2015 at 11:25 AM, Rezaei, Mohammad A. < Mohammad.Rezaei at gs.com> wrote: > Sorry, I?m not a C# practitioner, but I?ve been reading about it because > they supposedly had issues with mutable structs. > > > > According to > http://blogs.msdn.com/b/ericlippert/archive/2008/05/14/mutating-readonly-structs.aspx > , ?accessing a value type gives you a COPY of the value. When you say > t.m, you get a copy of whatever is presently stored in m? > > > > That sure sounds like Dot is a copy. > > > > Thanks > > Moh > > > > *From:* Vitaly Davidovich [mailto:vitalyd at gmail.com] > *Sent:* Friday, January 23, 2015 11:17 AM > *To:* Rezaei, Mohammad A. [Tech] > *Cc:* valhalla-dev at openjdk.java.net > *Subject:* RE: What happened to the "mutable struct" debate? > > > > Also, dot is not a copy in C# unless I'm missing your point. > > sent from my phone > > On Jan 23, 2015 11:12 AM, "Rezaei, Mohammad A." > wrote: > > I don?t believe this is confusing. It?s not confusing with Java objects. > > > > MyObject foo = new MyObject(); > > foo.inField = 10; > > > > vs. > > > > MyObject foo = new MyObject(); > > int x = foo.intField; > > x = 10; > > > > The rules are simple and the exactly the same as regular primitives: > > - Assignment is copy. > > - Pass into method is copy. > > - return from a method is copy. > > - dot (.) is not a copy. > > > > Incidentally, I?m not a C# expert, but I believe the 4th rule doesn?t > apply there and it causes much confusion. > > > > Thanks > > Moh > > > > *From:* Vitaly Davidovich [mailto:vitalyd at gmail.com] > *Sent:* Friday, January 23, 2015 11:06 AM > *To:* Rezaei, Mohammad A. [Tech] > *Cc:* Brian Goetz; valhalla-dev at openjdk.java.net > *Subject:* Re: What happened to the "mutable struct" debate? > > > > Also, here's another thing to consider for mutable structs: > > MyStruct[] arr = ... > arr[i].intField = 10; > > Vs > MyStruct[] arr = ... > MyStruct s = arr[i]; > s.intField = 10; > > These wouldn't have the same semantic yet not super obvious to some from > cursory glance (or innocent refactoring). > > sent from my phone > > On Jan 23, 2015 11:00 AM, "Vitaly Davidovich" wrote: > > If mutable structs are not allowed to be passed by ref, then one is more > likely to lose writes as the struct is passed through a method chain. > There's no *necessity* per say, but the danger zone is expanded. > > > > The by-ref would be useful for immutable structs as well when you want to > avoid copying costs for a largish struct. > > > > On Fri, Jan 23, 2015 at 10:54 AM, Rezaei, Mohammad A. < > Mohammad.Rezaei at gs.com> wrote: > > >Very much so. This is indeed Big Strike #1 against them; this is a new > >big complexity that Java (platform and developers) have not had to deal > >with. > > > > I don't really see why mutable structs would necessitate pass by > reference. Java developers are used to pass by value semantics and there is > a well-known, but admittedly ugly pattern for pass by reference (use an > array or mutable wrapper). > > The reference question comes up even in all-final value types. But I'll > pose that question in a separate thread. > > Thanks > Moh > > > From twhitmore.nz at gmail.com Mon Jan 26 03:00:42 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Mon, 26 Jan 2015 16:00:42 +1300 Subject: generic List traversal and VT/VO compatibility Message-ID: Hi Stephane, Stephen, Vitaly, The issue at hand (in this thread) is about iterating over opaque lists (or > any iterable, for that matter). The crux of the issue is that what fits > most naturally with any-fied generics, void iterate(List ...), > is today being written using List. And one of the (predominant?) > reasons that's used today is because of late-binding/reflection where the T > isn't known at compile time. I don't think anyone is claiming there's any > subtype relationship between List and List from an OO perspective. > I am not fully clear why we can't fit "reasonable" generic traversal & processing of List/ Listinto the existing language scheme. We can fairly consider an interface "List" to mean _a List of items coercable to Object._ As an interface, this merely specifies that the Items can be accessed via an "Object" reftype API. This is perfectly possible for List to implement -- primitives & value-types can (and for generality, should) offer a boxed API. This gives general iteration/ compatibility "for free", out of the box. Internally bridge methods would be used. When we write specific type-code against List or List, these would still be nominally compatible with List but the more efficient type-specific methods would be bound. So efficiency would be highest for type-specific code, but generality would be be preserved via List including all List, List. Efficiency of generic iteration of primitive/ valtype would be similar to today's boxed lists. An important point on types & type relations: - There should presumably be some relationship in the type system between List and List. - This is not a proper "supertype/subtype" relationship. List is a narrowing of List, rather than a real subtype. - We could consider it as a "specializes" or "reifies/generalizes" relation. - We should not try to treat/ or consider this relation as something it is not, as our conclusions/ results may likely be incorrect. (in regard of design) I can still imagine updating the language, such that List now effectively meant "List". Boxing would be automatic and implicit to fill in the gaps. Some improvement to method-selection & despatch rules might be required. So: "" would now mean "". List would declare: Object get (int index) void add (Object item) List would declare: int get (int index) void add (int item) and expose as bridges: Object get (int index) void add (Object item) Existing framework code should almost totally "just work" -- with similar performance to today -- via the boxed APIs. Hopefully, having clear & meaningful "specializes/generalizes" type relation would help framework code in future to do even better. Question: can Java method despatch be tweaked/ enhanced to achieve the above? ---------- Last point: Interface "compatibility" may conceivably be easier/ more possible, than the same thing for classes. I don't know if this a solid conclusion or just instinct, prompted by two things: - the avoidance of concrete class implementation inheritance (T[] elements), - greater flexibility of interface method-dispatch. For this reason I am proposing the relation for interface types, as a starting point. Regards Thomas Whitmore From vitalyd at gmail.com Mon Jan 26 03:15:00 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Sun, 25 Jan 2015 22:15:00 -0500 Subject: generic List traversal and VT/VO compatibility In-Reply-To: References: Message-ID: You could possibly make List work as List for readonly cases (iteration), with the performance cost of boxing. But having add (Object) on it is just not cool - that implies you can add any type, including null (provided the List implementation allows for it), but clearly you'll get a runtime exception. When it comes to reified generics, I don't expect such runtime exceptions if my code compiles. This may also break some existing code that makes these assumptions. The fundamental problem is that List implicitly assumes erased generics and that the erasure is a reference type (i.e. null is valid). sent from my phone On Jan 25, 2015 10:00 PM, "Thomas W" wrote: > Hi Stephane, Stephen, Vitaly, > > The issue at hand (in this thread) is about iterating over opaque lists (or >> any iterable, for that matter). The crux of the issue is that what fits >> most naturally with any-fied generics, void iterate(List ...), >> is today being written using List. And one of the (predominant?) >> reasons that's used today is because of late-binding/reflection where the >> T >> isn't known at compile time. I don't think anyone is claiming there's any >> subtype relationship between List and List from an OO perspective. >> > > I am not fully clear why we can't fit "reasonable" generic traversal & > processing of List/ Listinto the existing language scheme. > > We can fairly consider an interface "List" to mean _a > List of items coercable to Object._ As an interface, this merely specifies > that the Items can be accessed via an "Object" reftype API. > > This is perfectly possible for List to implement -- primitives & > value-types can (and for generality, should) offer a boxed API. This gives > general iteration/ compatibility "for free", out of the box. Internally > bridge methods would be used. > > When we write specific type-code against List or List, > these would still be nominally compatible with List but the more > efficient type-specific methods would be bound. > > So efficiency would be highest for type-specific code, but generality > would be be preserved via List including all List, > List. Efficiency of generic iteration of primitive/ valtype would > be similar to today's boxed lists. > > An important point on types & type relations: > - There should presumably be some relationship in the type system between > List and List. > - This is not a proper "supertype/subtype" relationship. List is a > narrowing of List, rather than a real subtype. > - We could consider it as a "specializes" or "reifies/generalizes" > relation. > - We should not try to treat/ or consider this relation as something it is > not, as our conclusions/ results may likely be incorrect. (in regard of > design) > > I can still imagine updating the language, such that List now > effectively meant "List". Boxing would be automatic and implicit to > fill in the gaps. Some improvement to method-selection & despatch rules > might be required. > > So: "" would now mean "". > > List would declare: > Object get (int index) > void add (Object item) > > List would declare: > int get (int index) > void add (int item) > and expose as bridges: > Object get (int index) > void add (Object item) > > Existing framework code should almost totally "just work" -- with similar > performance to today -- via the boxed APIs. Hopefully, having clear & > meaningful "specializes/generalizes" type relation would help framework > code in future to do even better. > > Question: can Java method despatch be tweaked/ enhanced to achieve the > above? > ---------- > > Last point: Interface "compatibility" may conceivably be easier/ more > possible, than the same thing for classes. I don't know if this a solid > conclusion or just instinct, prompted by two things: > - the avoidance of concrete class implementation inheritance (T[] > elements), > - greater flexibility of interface method-dispatch. > > For this reason I am proposing the relation for interface types, as a > starting point. > > > Regards > Thomas Whitmore > > From vitalyd at gmail.com Mon Jan 26 03:19:45 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Sun, 25 Jan 2015 22:19:45 -0500 Subject: generic List traversal and VT/VO compatibility In-Reply-To: References: Message-ID: By the way, you can't add (Object) on wildcard collections today, so I was only responding to your proposal verbatim. It does, however, allow null so that would still be an issue. sent from my phone On Jan 25, 2015 10:15 PM, "Vitaly Davidovich" wrote: > You could possibly make List work as List for readonly cases > (iteration), with the performance cost of boxing. But having add (Object) > on it is just not cool - that implies you can add any type, including null > (provided the List implementation allows for it), but clearly you'll get a > runtime exception. When it comes to reified generics, I don't expect such > runtime exceptions if my code compiles. This may also break some existing > code that makes these assumptions. The fundamental problem is that List > implicitly assumes erased generics and that the erasure is a reference type > (i.e. null is valid). > > sent from my phone > On Jan 25, 2015 10:00 PM, "Thomas W" wrote: > >> Hi Stephane, Stephen, Vitaly, >> >> The issue at hand (in this thread) is about iterating over opaque lists >>> (or >>> any iterable, for that matter). The crux of the issue is that what fits >>> most naturally with any-fied generics, void iterate(List ...), >>> is today being written using List. And one of the (predominant?) >>> reasons that's used today is because of late-binding/reflection where >>> the T >>> isn't known at compile time. I don't think anyone is claiming there's >>> any >>> subtype relationship between List and List from an OO >>> perspective. >>> >> >> I am not fully clear why we can't fit "reasonable" generic traversal & >> processing of List/ Listinto the existing language scheme. >> >> We can fairly consider an interface "List" to mean _a >> List of items coercable to Object._ As an interface, this merely specifies >> that the Items can be accessed via an "Object" reftype API. >> >> This is perfectly possible for List to implement -- primitives & >> value-types can (and for generality, should) offer a boxed API. This gives >> general iteration/ compatibility "for free", out of the box. Internally >> bridge methods would be used. >> >> When we write specific type-code against List or List, >> these would still be nominally compatible with List but the more >> efficient type-specific methods would be bound. >> >> So efficiency would be highest for type-specific code, but generality >> would be be preserved via List including all List, >> List. Efficiency of generic iteration of primitive/ valtype would >> be similar to today's boxed lists. >> >> An important point on types & type relations: >> - There should presumably be some relationship in the type system between >> List and List. >> - This is not a proper "supertype/subtype" relationship. List is a >> narrowing of List, rather than a real subtype. >> - We could consider it as a "specializes" or "reifies/generalizes" >> relation. >> - We should not try to treat/ or consider this relation as something it >> is not, as our conclusions/ results may likely be incorrect. (in regard of >> design) >> >> I can still imagine updating the language, such that List now >> effectively meant "List". Boxing would be automatic and implicit to >> fill in the gaps. Some improvement to method-selection & despatch rules >> might be required. >> >> So: "" would now mean "". >> >> List would declare: >> Object get (int index) >> void add (Object item) >> >> List would declare: >> int get (int index) >> void add (int item) >> and expose as bridges: >> Object get (int index) >> void add (Object item) >> >> Existing framework code should almost totally "just work" -- with similar >> performance to today -- via the boxed APIs. Hopefully, having clear & >> meaningful "specializes/generalizes" type relation would help framework >> code in future to do even better. >> >> Question: can Java method despatch be tweaked/ enhanced to achieve the >> above? >> ---------- >> >> Last point: Interface "compatibility" may conceivably be easier/ more >> possible, than the same thing for classes. I don't know if this a solid >> conclusion or just instinct, prompted by two things: >> - the avoidance of concrete class implementation inheritance (T[] >> elements), >> - greater flexibility of interface method-dispatch. >> >> For this reason I am proposing the relation for interface types, as a >> starting point. >> >> >> Regards >> Thomas Whitmore >> >> From twhitmore.nz at gmail.com Tue Jan 27 00:33:19 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Tue, 27 Jan 2015 13:33:19 +1300 Subject: generic List traversal and VT/VO compatibility In-Reply-To: References: Message-ID: Thanks Vitaly, I was trying to make the point about "List API compatibility" for List. By the way, you can't add (Object) on wildcard collections today, so I was > only responding to your proposal verbatim. It does, however, allow null so > that would still be an issue. You are correct, there should be no add(Object). remove(Object) would be an alternative method to illustrate the bridging, but that method brings up the separate (solveable) issue of name-collisions. My point is really about compatibility & assignability of List to List; not by it being a subtype, but by a "specializes/generalizes" relation. I think it's important to keep that distinction clear. So, my questions are: - can existing method selection/ despatch allow this, or be tweaked/ enhanced to efficiently do so? - can cast/ checkcast efficiently check a "specializes" relation also? -------- To fix up my previous example: List would declare: Object get (int index) void add (Object item) boolean removeItem (Object item) // renamed for illustrative purposes, to avoid name collision List would declare: int get (int index) void add (int item) boolean removeItem (int item) and expose as bridges: Integer get (int index) // compatible with List void add (Integer item) boolean removeItem (Object item) Interestingly, writing this brings up the point that bridge methods are already present in reftyped generics; bridging from a subtypes (with different bounds) to superclass inherited methods. Our circumstance here is a little different -- we want our List to include reftype API compatibility. Regards, Thomas From volker.simonis at gmail.com Wed Jan 28 20:40:49 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 28 Jan 2015 21:40:49 +0100 Subject: What's the status of / relation between "JEP 169: Value Objects" / "Value Types for Java" / "Object Layout" Message-ID: Hi everybody, I've recently did some research on Java "value objects" / "value types" / "object layout" (I'll be actually giving a short talk on the topic at FOSDEM[0] this weekend). I just want to quickly summarize my current findings here and gently ask for feedback in case you think I've totally misunderstood something. Of course any comments and additional information is highly welcome as well. 1. JEP 169: Value Objects [1] - Created by John Rose in 2012 (last update in Sep. 2014) - Still in "Draft" state - Proposes a new "lockPermanently()" operator which marks objects as immutable - Seems to be only a little "helper functionality" to simplify automatic boxing/unboxing and escape analysis - Referenced the mlwm mailing list and repository but the mlwm repo seems dead since about 15 month now Question: is JEP 169 still under active development or has it been merged into the more general "Value types for Java" proposal below? 2. "Value types for Java" / "State of the Values" [2] - By J. Rose, B. Goetz and Guy Stele - Based on earlier ideas from "Value Types in the VM" [3] - Newest and most elaborate proposal - Proposes general (i.e. function arguments, return values, variables, arrays), "immutable" value types - Requires fundamental changes to the VM as well as to the Java language - Related to the "State of the Specialization" proposal [4] about support for generics over primitive and value types by B Goetz. - Discussed and developed in the OpenJDK "Valhalla" [5] project - Still very early stage (i.e. no "code" available yet) 3. PackedObjects as provided by the IBM J9 [6,7] - Flattens the memory layout of "@Packed" object fields and array - Removes object headers of and references to "@Packed" objects - Object headers can be generated on the fly (kind of "auto-boxing") - Currently the most complete and mature solution - Not Java-compatible (e.g. can not write to a nested "@Packed" fields). Must be enabled as an experimental extension. 4. ObjectLayout [8] - A pure Java, layout-optimized data structure package - Designed similar to "@ValueSafe"/"ValueType" in [3] and "Value-base classes" in Java 8 [9] - Designed such that it can be tranparently optimized within the VM - VM can transparently layout "@Intrinsic" objects within other objects - All objects are still complete Java object with valid header - The Java part of the library is mature, first native VM-optimizations on the way [10] The "Value types for Java" approach clearly seems to be the most general but also the most complex proposal. It's out of scope for Java 9 and still questionable for Java 10 and above. The "PackedObject" and "ObjectLayout" approaches are clearly simpler and more limited in scope as they only concentrate on better object layout. However the "ObjectLayout" proposal demonstrates that this is still possible within the current Java specification while the "PackedObjects" proposal demonstrated that an optimizing implementation is feasible. I've recently built a prototype which intrinsifies/optimizes some parts of the "ObjectLayout" proposal in the HotSpot [10]. Question: is there a chance to get a some sort of Java-only but transparently optimizable structure package like "ObjectLayout" into Java early (i.e. Java 9)? In my eyes this wouldn't contradict with a more general solution like the one proposed in the "Value types for Java" approach while still offering quite significant performance improvements for quite a big range of problems. And if carefully designed, it could be easily retrofitted to use the new, general "Value Types" once they are available. Question: what would be the right place to propose something like the "ObjectLayout" library for Java 9/10? Would that fit within the umbrella of the Valhalla project or would it be done within its own project / under it's own JEP? Thanks for your patience, Volker [0] https://fosdem.org/2015/schedule/event/packed_objects/ [1] http://openjdk.java.net/jeps/169 [2] http://cr.openjdk.java.net/~jrose/values/values-0.html [3] https://blogs.oracle.com/jrose/entry/value_types_in_the_vm [4] http://cr.openjdk.java.net/~briangoetz/valhalla/specialization.html [5] http://openjdk.java.net/projects/valhalla [6] http://www.slideshare.net/rsciampacone/javaone-2013-introduction-to-packedobjects?related=1 [7] http://medianetwork.oracle.com/video/player/2623645005001 [8] http://objectlayout.org [9] http://docs.oracle.com/javase/8/docs/api/java/lang/doc-files/ValueBased.html [10] https://github.com/simonis/ObjectLayout/tree/hotspot_intrinsification/hotspot From peter.levart at gmail.com Wed Jan 28 21:56:09 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 28 Jan 2015 22:56:09 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C0FAA3.900@oracle.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FAA3.900@oracle.com> Message-ID: <54C95AF9.3090004@gmail.com> On 01/22/2015 02:26 PM, Maurizio Cimadamore wrote: > > On 22/01/15 13:19, Vitaly Davidovich wrote: >> Can you expand a bit on the part where you say frameworks can't >> iterate a >> Collection without knowing the instantiation? Do you mean >> existing >> methods that take Collection won't work without change or >> something else? > In the current prototype, such methods will work, but they will work > with all reference-parameterizations - example: > > void m(Collection c) { ... } > > List ls = ... > List lo = ... > List li = ... > List lmpv = ... > > m(ls); //ok > m(lo); //ok > m(li); //fail, List is not subtype of Collection > m(lmpv); //fail, List is not subtype of Collection > > In other words, Collection means Collection. > Therefore, primitives and values are outside the domain supported by > the unbounded wildcard. > > To fix that, you need to change 'm' as follows: > > void m2(Collection c) { ... } > > m(ls); //ok > m(lo); //ok > m(li); //ok > m(lmpv); //ok > > Is this what you were asking? > > Maurizio There could also be an abbreviation: void m3(Collection c) { ... } ...which would be equivalent to m2 above, but used only if you don't need T in the method (you only invoke methods on 'c' that are not mentioning T from Collection in their signature, such as size(), clear(), ...). Maybe you could also call methods, that mention T in return type, but you could not assign the result to anything (or maybe to Object in certain situations), for example: class Foo { T getOne() { ... } Optional getMaybe() { ... } } void m4(Foo anyFoo) { anyFoo.getOne(); // can call, but not use the result Object r = anyFoo.getMaybe(); // can assign to Object since Optional, Optional, etc. are all Objects } (I'm still considering Optional as reference type here) Peter > >> >> sent from my phone >> On Jan 22, 2015 8:12 AM, "St?phane ?pardaud" wrote: >> >>> On 01/22/2015 12:18 PM, Maurizio Cimadamore wrote: >>> >>>> I don't think there's an 'hole' as such in your proposal - but >>>> there are >>>> 'unknowns'. >>>> >>> Well, that's already good news. I prefer unknowns to known holes ;) >>> >>> Performance-wise, I think the 'risk' is that if we keep the API >>> the way >>>> they are today (i.e. removeAll(Collection) and friends), the >>>> boxing path >>>> will pretty much be the norm. Now, in a perfect world, as value >>>> types are >>>> non-polymorphic (or their polymorphism is very restricted) and >>>> immutable, >>>> that would suggest that VM should have enough of an hint to perform >>>> boxing >>>> elimination and such, so that the cost you end up paying is >>>> negligible. How >>>> much of this is reality? As you, I'm not a VM guru - but I think >>>> it's a >>>> question worth asking. It seems a likely scenario that the JVM will do >>>> great in most cases, and have some bad performance cliffs in others >>>> - is >>>> that something we are willing to sign up for? >>>> >>> OK, that's fair. By allowing people to use value types boxed we >>> allow them >>> to shoot themselves in the foot perf-wise (assuming the VM can't help), >>> because we make value types much more accessible. This is mostly due to >>> `any T` being opt-in rather than the new default for `T` where `T >>> extends >>> Object` is not specified. If we also changed `T` to mean `any T` >>> (source >>> only, so for newly compiled code) then all generics code would be >>> specializable by default and that means that users of generic code will >>> always be able to use the specialised code as long as they can >>> instantiate >>> the generic type argument at compile-time (as is already the >>> limitation). >>> If they can't, well they already have to use `Object` and boxing and >>> can't >>> traverse collections because they're not `List` (ATM). >>> >>> Making it the default has down sides, probably in larger class >>> files, but >>> as long as it's not the default, there will always be >>> incompatibilities in >>> libraries that will forget to opt-in, which will mean that either (ATM) >>> they can't be used with containers of value types, or that the >>> containers >>> have to be wrapped to box the value types (unless we fix that as I >>> suggest). IMO that's already going to cause compatibility issues and >>> there >>> may arise a "coding guideline" (remember them from C++ sore points?) >>> that >>> people will be strongly encourage to use `val T` everywhere just in >>> case >>> someone wants to use an unboxed value type. >>> >>> The rift between primitives and Object is already a famous sore >>> point in >>> Java (not criticising, this was a choice made a long time ago, for >>> valid >>> reasons, we just have to deal with it) which has been "fixed" in most >>> non-Java JVM-languages due to popular demand. I don't think the new >>> default >>> with value types should be that generics don't accept value types. >>> >>> On the language-side unknowns, how much code out there is relying on >>>> being able to access fields on Foo or raw Foo types? This is >>>> perhaps not >>>> common, but I think we need to gather data points on this i.e. by >>>> looking >>>> at existing open source projects (help welcome here!). Other >>>> possible weak >>>> points are that this doesn't necessarily address all the issue w.r.t. >>>> language uniformity - i.e. how is an ArrayList supposed to >>>> answer to a >>>> question of the kind 'is instance of List' ? >>>> >>> Fields that don't involve the `any T` are fine. Fields that do may >>> have to >>> revert to autoboxing _if_ we feel we _must_ accomodate that to >>> autobox not >>> just value types but their containers. >>> >>> `ArrayList` is special, mostly due to the fact that I'm not >>> sure we >>> can retrofit `Integer` to be a value type, so existing primitives >>> may not >>> get the `List === List` that I suggest for value types >>> (though I'd be very interested in making this work). >>> >>> If we take an easier example assuming a value type named `Date`, >>> then yes >>> I expect that: >>> >>> - `List instanceof List` == `true` >>> - `List<__Boxed Date> instanceof List` == `true` >>> >>> I'm not denying there's something there worth exploring (as I have in >>>> fact already said), but it seems to me that, while you can go a >>>> long way >>>> with bridges, there are still questions that bridges alone simply >>>> do not >>>> have an answer for. >>>> >>> Thanks, and yes I agree there are still questions remaining, and more >>> importantly an implementation lacking, but hopefully if the proposal >>> sounds >>> decent I can help with the proto. >>> >>> Thanks for your answers BTW :) >>> > From vitalyd at gmail.com Thu Jan 29 03:58:35 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 28 Jan 2015 22:58:35 -0500 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: <54C95AF9.3090004@gmail.com> References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FAA3.900@oracle.com> <54C95AF9.3090004@gmail.com> Message-ID: Sorry Peter, I'm having a hard time understanding what this buys us - could you explain what problem this solves exactly? It's certainly not for backcompat from the description, so unclear. sent from my phone On Jan 28, 2015 4:56 PM, "Peter Levart" wrote: > > On 01/22/2015 02:26 PM, Maurizio Cimadamore wrote: > > > On 22/01/15 13:19, Vitaly Davidovich wrote: > > Can you expand a bit on the part where you say frameworks can't iterate a > Collection without knowing the instantiation? Do you mean existing > methods that take Collection won't work without change or something > else? > > In the current prototype, such methods will work, but they will work with > all reference-parameterizations - example: > > void m(Collection c) { ... } > > List ls = ... > List lo = ... > List li = ... > List lmpv = ... > > m(ls); //ok > m(lo); //ok > m(li); //fail, List is not subtype of Collection > m(lmpv); //fail, List is not subtype of Collection > > In other words, Collection means Collection. > Therefore, primitives and values are outside the domain supported by the > unbounded wildcard. > > To fix that, you need to change 'm' as follows: > > void m2(Collection c) { ... } > > m(ls); //ok > m(lo); //ok > m(li); //ok > m(lmpv); //ok > > Is this what you were asking? > > Maurizio > > > There could also be an abbreviation: > > void m3(Collection c) { ... } > > ...which would be equivalent to m2 above, but used only if you don't need > T in the method (you only invoke methods on 'c' that are not mentioning T > from Collection in their signature, such as size(), clear(), ...). > Maybe you could also call methods, that mention T in return type, but you > could not assign the result to anything (or maybe to Object in certain > situations), for example: > > class Foo { > > T getOne() { ... } > > Optional getMaybe() { ... } > > } > > > void m4(Foo anyFoo) { > > anyFoo.getOne(); // can call, but not use the result > > Object r = anyFoo.getMaybe(); // can assign to Object since > Optional, Optional, etc. are all Objects > > } > > (I'm still considering Optional as reference type here) > > Peter > > > > > sent from my phone > On Jan 22, 2015 8:12 AM, "St?phane ?pardaud" > wrote: > > On 01/22/2015 12:18 PM, Maurizio Cimadamore wrote: > > I don't think there's an 'hole' as such in your proposal - but there are > 'unknowns'. > > Well, that's already good news. I prefer unknowns to known holes ;) > > Performance-wise, I think the 'risk' is that if we keep the API the way > > they are today (i.e. removeAll(Collection) and friends), the boxing > path > will pretty much be the norm. Now, in a perfect world, as value types are > non-polymorphic (or their polymorphism is very restricted) and immutable, > that would suggest that VM should have enough of an hint to perform boxing > elimination and such, so that the cost you end up paying is negligible. > How > much of this is reality? As you, I'm not a VM guru - but I think it's a > question worth asking. It seems a likely scenario that the JVM will do > great in most cases, and have some bad performance cliffs in others - is > that something we are willing to sign up for? > > OK, that's fair. By allowing people to use value types boxed we allow > them > to shoot themselves in the foot perf-wise (assuming the VM can't help), > because we make value types much more accessible. This is mostly due to > `any T` being opt-in rather than the new default for `T` where `T extends > Object` is not specified. If we also changed `T` to mean `any T` (source > only, so for newly compiled code) then all generics code would be > specializable by default and that means that users of generic code will > always be able to use the specialised code as long as they can instantiate > the generic type argument at compile-time (as is already the limitation). > If they can't, well they already have to use `Object` and boxing and can't > traverse collections because they're not `List` (ATM). > > Making it the default has down sides, probably in larger class files, but > as long as it's not the default, there will always be incompatibilities in > libraries that will forget to opt-in, which will mean that either (ATM) > they can't be used with containers of value types, or that the containers > have to be wrapped to box the value types (unless we fix that as I > suggest). IMO that's already going to cause compatibility issues and there > may arise a "coding guideline" (remember them from C++ sore points?) that > people will be strongly encourage to use `val T` everywhere just in case > someone wants to use an unboxed value type. > > The rift between primitives and Object is already a famous sore point in > Java (not criticising, this was a choice made a long time ago, for valid > reasons, we just have to deal with it) which has been "fixed" in most > non-Java JVM-languages due to popular demand. I don't think the new > default > with value types should be that generics don't accept value types. > > On the language-side unknowns, how much code out there is relying on > > being able to access fields on Foo or raw Foo types? This is perhaps > not > common, but I think we need to gather data points on this i.e. by looking > at existing open source projects (help welcome here!). Other possible weak > points are that this doesn't necessarily address all the issue w.r.t. > language uniformity - i.e. how is an ArrayList supposed to answer to > a > question of the kind 'is instance of List' ? > > Fields that don't involve the `any T` are fine. Fields that do may have > to > revert to autoboxing _if_ we feel we _must_ accomodate that to autobox not > just value types but their containers. > > `ArrayList` is special, mostly due to the fact that I'm not sure we > can retrofit `Integer` to be a value type, so existing primitives may not > get the `List === List` that I suggest for value types > (though I'd be very interested in making this work). > > If we take an easier example assuming a value type named `Date`, then yes > I expect that: > > - `List instanceof List` == `true` > - `List<__Boxed Date> instanceof List` == `true` > > I'm not denying there's something there worth exploring (as I have in > > fact already said), but it seems to me that, while you can go a long way > with bridges, there are still questions that bridges alone simply do not > have an answer for. > > Thanks, and yes I agree there are still questions remaining, and more > importantly an implementation lacking, but hopefully if the proposal > sounds > decent I can help with the proto. > > Thanks for your answers BTW :) > > > > From peter.levart at gmail.com Thu Jan 29 06:55:16 2015 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 29 Jan 2015 07:55:16 +0100 Subject: Idea how to implement VT/VO compatibility in JVM In-Reply-To: References: <54B014F2.9080805@oracle.com> <54B0280A.3020707@oracle.com> <20150112155709.GA15204@inforealm.org> <54B3F0B2.3000208@oracle.com> <54B3F6A3.8050500@epardaud.fr> <54BFF278.9030500@oracle.com> <54C0B3D8.6080902@epardaud.fr> <54C0DC98.3020204@oracle.com> <54C0F71A.2010406@epardaud.fr> <54C0FAA3.900@oracle.com> <54C95AF9.3090004@gmail.com> Message-ID: <54C9D954.9080209@gmail.com> On 01/29/2015 04:58 AM, Vitaly Davidovich wrote: > Sorry Peter, I'm having a hard time understanding what this buys us - could > you explain what problem this solves exactly? It's certainly not for > backcompat from the description, so unclear. I'm just contemplating about the usefulness of a construct. Equivalent to relation of: void m(Collection c) { ... } vs. void m(Collection c) { ... } There could also be: void m(Collection c) { ... } vs. void m(Collection c) { ... } Although the later (any ?) is less useful than the former (?), there is still some use to be found in the anyfied one. And the language features should be orthogonal if at all possible. Regards, Peter > > sent from my phone > On Jan 28, 2015 4:56 PM, "Peter Levart" wrote: > >> On 01/22/2015 02:26 PM, Maurizio Cimadamore wrote: >> >> >> On 22/01/15 13:19, Vitaly Davidovich wrote: >> >> Can you expand a bit on the part where you say frameworks can't iterate a >> Collection without knowing the instantiation? Do you mean existing >> methods that take Collection won't work without change or something >> else? >> >> In the current prototype, such methods will work, but they will work with >> all reference-parameterizations - example: >> >> void m(Collection c) { ... } >> >> List ls = ... >> List lo = ... >> List li = ... >> List lmpv = ... >> >> m(ls); //ok >> m(lo); //ok >> m(li); //fail, List is not subtype of Collection >> m(lmpv); //fail, List is not subtype of Collection >> >> In other words, Collection means Collection. >> Therefore, primitives and values are outside the domain supported by the >> unbounded wildcard. >> >> To fix that, you need to change 'm' as follows: >> >> void m2(Collection c) { ... } >> >> m(ls); //ok >> m(lo); //ok >> m(li); //ok >> m(lmpv); //ok >> >> Is this what you were asking? >> >> Maurizio >> >> >> There could also be an abbreviation: >> >> void m3(Collection c) { ... } >> >> ...which would be equivalent to m2 above, but used only if you don't need >> T in the method (you only invoke methods on 'c' that are not mentioning T >> from Collection in their signature, such as size(), clear(), ...). >> Maybe you could also call methods, that mention T in return type, but you >> could not assign the result to anything (or maybe to Object in certain >> situations), for example: >> >> class Foo { >> >> T getOne() { ... } >> >> Optional getMaybe() { ... } >> >> } >> >> >> void m4(Foo anyFoo) { >> >> anyFoo.getOne(); // can call, but not use the result >> >> Object r = anyFoo.getMaybe(); // can assign to Object since >> Optional, Optional, etc. are all Objects >> >> } >> >> (I'm still considering Optional as reference type here) >> >> Peter >> >> >> >> >> sent from my phone >> On Jan 22, 2015 8:12 AM, "St?phane ?pardaud" >> wrote: >> >> On 01/22/2015 12:18 PM, Maurizio Cimadamore wrote: >> >> I don't think there's an 'hole' as such in your proposal - but there are >> 'unknowns'. >> >> Well, that's already good news. I prefer unknowns to known holes ;) >> >> Performance-wise, I think the 'risk' is that if we keep the API the way >> >> they are today (i.e. removeAll(Collection) and friends), the boxing >> path >> will pretty much be the norm. Now, in a perfect world, as value types are >> non-polymorphic (or their polymorphism is very restricted) and immutable, >> that would suggest that VM should have enough of an hint to perform boxing >> elimination and such, so that the cost you end up paying is negligible. >> How >> much of this is reality? As you, I'm not a VM guru - but I think it's a >> question worth asking. It seems a likely scenario that the JVM will do >> great in most cases, and have some bad performance cliffs in others - is >> that something we are willing to sign up for? >> >> OK, that's fair. By allowing people to use value types boxed we allow >> them >> to shoot themselves in the foot perf-wise (assuming the VM can't help), >> because we make value types much more accessible. This is mostly due to >> `any T` being opt-in rather than the new default for `T` where `T extends >> Object` is not specified. If we also changed `T` to mean `any T` (source >> only, so for newly compiled code) then all generics code would be >> specializable by default and that means that users of generic code will >> always be able to use the specialised code as long as they can instantiate >> the generic type argument at compile-time (as is already the limitation). >> If they can't, well they already have to use `Object` and boxing and can't >> traverse collections because they're not `List` (ATM). >> >> Making it the default has down sides, probably in larger class files, but >> as long as it's not the default, there will always be incompatibilities in >> libraries that will forget to opt-in, which will mean that either (ATM) >> they can't be used with containers of value types, or that the containers >> have to be wrapped to box the value types (unless we fix that as I >> suggest). IMO that's already going to cause compatibility issues and there >> may arise a "coding guideline" (remember them from C++ sore points?) that >> people will be strongly encourage to use `val T` everywhere just in case >> someone wants to use an unboxed value type. >> >> The rift between primitives and Object is already a famous sore point in >> Java (not criticising, this was a choice made a long time ago, for valid >> reasons, we just have to deal with it) which has been "fixed" in most >> non-Java JVM-languages due to popular demand. I don't think the new >> default >> with value types should be that generics don't accept value types. >> >> On the language-side unknowns, how much code out there is relying on >> >> being able to access fields on Foo or raw Foo types? This is perhaps >> not >> common, but I think we need to gather data points on this i.e. by looking >> at existing open source projects (help welcome here!). Other possible weak >> points are that this doesn't necessarily address all the issue w.r.t. >> language uniformity - i.e. how is an ArrayList supposed to answer to >> a >> question of the kind 'is instance of List' ? >> >> Fields that don't involve the `any T` are fine. Fields that do may have >> to >> revert to autoboxing _if_ we feel we _must_ accomodate that to autobox not >> just value types but their containers. >> >> `ArrayList` is special, mostly due to the fact that I'm not sure we >> can retrofit `Integer` to be a value type, so existing primitives may not >> get the `List === List` that I suggest for value types >> (though I'd be very interested in making this work). >> >> If we take an easier example assuming a value type named `Date`, then yes >> I expect that: >> >> - `List instanceof List` == `true` >> - `List<__Boxed Date> instanceof List` == `true` >> >> I'm not denying there's something there worth exploring (as I have in >> >> fact already said), but it seems to me that, while you can go a long way >> with bridges, there are still questions that bridges alone simply do not >> have an answer for. >> >> Thanks, and yes I agree there are still questions remaining, and more >> importantly an implementation lacking, but hopefully if the proposal >> sounds >> decent I can help with the proto. >> >> Thanks for your answers BTW :) >> >> >> >> From daniel.latremoliere at gmail.com Thu Jan 29 11:02:32 2015 From: daniel.latremoliere at gmail.com (=?UTF-8?B?RGFuaWVsIExhdHLDqW1vbGnDqHJl?=) Date: Thu, 29 Jan 2015 12:02:32 +0100 Subject: What's the status of / relation between "JEP 169: Value Objects" / "Value Types for Java" / "Object Layout" In-Reply-To: References: Message-ID: <54CA1348.5050903@gmail.com> > I just want to quickly summarize my > current findings here and gently ask for feedback in case you think > I've totally misunderstood something. Of course any comments and > additional information is highly welcome as well. I don't know if that can be useful, but here is my point of view of developer oriented towards the question: "What feature for solving my problem?". This contains probably some or many errors, but it is another point of view (only mine), if useful. I will not use strictly projects/proposal list as the structure of my mail because content of proposal is changing and it is not my target. I am oriented towards the final user, i.e. the developer consuming these projects, not the implementer working in each of these projects. I will preferably split in three scopes following my perceived split of job between developer and runtime. The problem is data, then what can do JVM/GC with an object? I find two possibilities regarding this domain: move it, clone it. If JVM can clone the object, JVM can also move the object because the clone will not have the same address, then we have the following three features: --- 1) JVM can clone and move objects (Project Valhalla): Constraint: no complex constructor/no complex finalizer, because lifecycle of object is managed by JVM (JVM can clone, then JVM can create and destroy the object like JVM want). Only field affectation constructor, possibly with simple conversion of data format. Constraint: immutable, because we don't know which clone is good when one is modified and because modifying all clones simultaneously is slow/complex/parallel-unfriendly. Constraint: non-null because cloning a non-existing object is a non-existing problem. Use-case "Performance": objects to clone for being closer to execution silicon and better parallelism (registers or cache of CPU/GPU) - Runtime: expose features of CPU/GPU like SIMD (mostly like a modern version of javax.vecmath). - Developer: create custom low-level structures for CPU/GPU parallel computing. - Java language: small tuples, like complex numbers (immutable by performance choice, like SIMD, for being close to silicon; cloned at each pass by value). Use-case "Language": objects to clone for being closer to registers (in stack, then less allocations in heap; simpler than escape analysis) - Java language: multiple return values from a method (immutable because it's a result; cloned, by example, at the return of each delegate or not even created when stack-only). Use-case "Efficiency": others immutable non-null objects possibly concerned for reducing indirection/improving cache, given by specialization of collection classes - Database: primary key for Map (like HashMap)/B-Tree (like MapDB)/SQL (like JPA). A primary key is immutable and non-null by choice of developer, then possible gains. --- 2) JVM can move but not clone objects It's current state of Java objects: Constraint: developer need to define lifecycle in object, for being triggered by GC (constructor/finalizer) like current Java class. Constraint: small object, because when GC move a big object, there is possibly a noticeable latency. Constraint: usable directly only in Java code (because native code will need an indirection level for finding the real address of the object, changing after each move) Improvement by adding custom layout for objects (Project Panama on heap / ObjectLayout): Specific constraint: objects which are near identity-less, i.e. only one other object (the owner) know their identity/have pointer on it. Non-constraint: applicable to all objects types, contrary to Project Valhalla. Applicable to complex constructor, because complex constructor can be inlined in owner code where called. Applicable to mutable objects , because no cloning then no incoherency. Applicable to nullable objects only by adding a boolean field in the custom layout for storing potential existence or non-existence of the inlined object, and updating code testing nullability for using this boolean. Use-case "General efficiency": Custom layout (Inline sub-object in the object owning it): - Reduce memory use with less objects then less headers and less pointers. - Improve cache performance with better locality (objects inlined are in same cache line, then no reference to follow). - Applicable to many fields containing reference, requiring only the referenced object to be invisible from all objects except one (the owner). By example, a private field containing an internal ArrayList (without getter/setter) can probably be replaced by the integer containing the used size and the reference to backing array, with inlining of the few methods of ArrayList really used. It need probably to be driven by developer after real profiling for finding best ratio between efficiency/code expansion. It will probably have much more use-cases when AOT will be available and developer-manageable precisely (Jigsaw???), because most slow work of object-code inlining and following optimizations can be done at AOT time, while gains will be at running time. Probably useful for the hottest code (JIT after this pre-optimization at AOT time) and clearly bad for the coldest code (interpreter then avoid code expansion), but very useful for the big quantity of code between, which will gain from AOT if complex optimizations are available. This will very probably require developer help/instructions/annotations using profiler data obtained on functional tests of application. --- 3) JVM can not move or clone objects (Project Panama off heap / PackedObjects) Constraint: developer need to manage externally the full lifecycle of object and need to choose when creating or destroying it. Object is off-heap and an handle is on-heap for managing off-heap part. Constraint: potential fragmentation of free memory when frequently creating and removing objects not having the same size (taking attention to object size vs. page size is probably important). Use-case "GC Latency": big data structure inducing GC latency when moved if stored in heap - All big chunks of data, like Big Data or textures in games, etc. - Few number of objects for being manageable more explicitly by developer (without too much work). Use-case "Native": communicate with native library - Modern version of JNI Only my 2 cents, Daniel. From blackdrag at gmx.org Thu Jan 29 11:55:39 2015 From: blackdrag at gmx.org (Jochen Theodorou) Date: Thu, 29 Jan 2015 12:55:39 +0100 Subject: What's the status of / relation between "JEP 169: Value Objects" / "Value Types for Java" / "Object Layout" In-Reply-To: <54CA1348.5050903@gmail.com> References: <54CA1348.5050903@gmail.com> Message-ID: <54CA1FBB.7020601@gmx.org> Am 29.01.2015 12:02, schrieb Daniel Latr?moli?re: > >> I just want to quickly summarize my >> current findings here and gently ask for feedback in case you think >> I've totally misunderstood something. Of course any comments and >> additional information is highly welcome as well. > I don't know if that can be useful, but here is my point of view of > developer oriented towards the question: "What feature for solving my > problem?". This contains probably some or many errors, but it is another > point of view (only mine), if useful. [...] > 3) JVM can not move or clone objects (Project Panama off heap / > PackedObjects) > Constraint: developer need to manage externally the full lifecycle of > object and need to choose when creating or destroying it. Object is > off-heap and an handle is on-heap for managing off-heap part. > Constraint: potential fragmentation of free memory when frequently > creating and removing objects not having the same size (taking attention > to object size vs. page size is probably important). > > Use-case "GC Latency": big data structure inducing GC latency when moved > if stored in heap > - All big chunks of data, like Big Data or textures in games, etc. > - Few number of objects for being manageable more explicitly by > developer (without too much work). > > Use-case "Native": communicate with native library > - Modern version of JNI From that view it makes me wonder if that is really in the scope of JEP 169. bye Jochen -- Jochen "blackdrag" Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org From brian.goetz at oracle.com Thu Jan 29 17:05:23 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 29 Jan 2015 12:05:23 -0500 Subject: What's the status of / relation between "JEP 169: Value Objects" / "Value Types for Java" / "Object Layout" In-Reply-To: References: Message-ID: <54CA6853.2060601@oracle.com> > Question: is JEP 169 still under active development or has it been > merged into the more general "Value types for Java" proposal below? It has been merged into the more general Value Types for Java proposal. > The "Value types for Java" approach clearly seems to be the most > general but also the most complex proposal. For some meanings of "complex". It is certainly the most intrusive and large; new bytecodes, new type signatures. But from a user-model perspective, value types are actually fairly simple. > It's out of scope for Java > 9 and still questionable for Java 10 and above. The "PackedObject" and > "ObjectLayout" approaches are clearly simpler and more limited in > scope as they only concentrate on better object layout. To your list, I'd add: Project Panama, the sister project to Valhalla. Panama focuses on interop with native code and data, including layout specification. A key goal of Packed was to be able to access off-heap native data in its native format, rather than marshalling it across the JNI boundary. Panama is focused on this problem as well, but aims to treat it as a separate problem from Java object layout, resulting in what we believe to be a cleaner decomposition of the two concerns. Packed is an interesting mix of memory density (object embedding and packed arrays) and native interop. But mixing the two goals also has costs; our approach is to separate them into orthogonal concerns, and we think that Valhalla and Panama do just that. So in many ways, while a larger project, the combination of Valhalla+Panama addresses the problem that Packed did, in a cleaner way. > Question: is there a chance to get a some sort of Java-only but > transparently optimizable structure package like "ObjectLayout" into > Java early (i.e. Java 9)? It would depend on a lot of things -- including the level of readiness of the design and implementation, and the overlap with anticipated future features. We've reviewed some of the early design of ObjectLayout and provided feedback to the projects architects; currently, I think it's in the "promising exploration" stage, but I think multiple rounds of simplification are needed before it is ready to be considered for "everybody's Java." But if the choice is to push something that's not ready into 9, or to wait longer -- there's not actually a choice to be made there. I appreciate the desire to "get something you can use now", but we have to be prepared to support whatever we push into Java for the next 20 years, and deal with the additional constraints it generates -- which can be an enormous cost. (Even thought the direct cost is mostly borne by Oracle, the indirect cost is borne by everyone, in the form of slower progress on everything else.) So I am very wary of the motivation of "well, something better is coming, but this works now, so can we push it in?" I'd prefer to focus on answering whether this is right thing for Java for the next 20 years. > In my eyes this wouldn't contradict with a more general solution like > the one proposed in the "Value types for Java" approach while still > offering quite significant performance improvements for quite a big > range of problems. The goals of the ObjectLayout effort has overlap with, but also differs from, the goals of Valhalla. And herein is the problem; neither generalizes the other, and I don't think we do the user base a great favor by pursuing two separate neither-coincident-nor-orthogonal approaches. I suspect, though, that after a few rounds of simplification, ObjectLayout could morph into something that fit either coincidently or orthogonally with the Valhalla work -- which would be great. But, as you know, our resources are limited, so we (Oracle) can't really afford to invest in both. And such simplification takes time -- getting to that "aha" moment when you realize you can simplify something is generally an incompressible process. > Question: what would be the right place to propose something like the > "ObjectLayout" library for Java 9/10? Would that fit within the > umbrella of the Valhalla project or would it be done within its own > project / under it's own JEP? Suggesting a version number at this point would be putting the cart before the horse (you'll note that we've not even proposed a version number for Valhalla; the closest we've gotten to that is "after 9".) OpenJDK Projects are a tool for building a community around a body of work; JEPs are a project-management tool for defining, scoping, and tracking the progress of a feature. Given where OL is, it would be reasonable to start a Project, which would become the nexus of collaboration that could eventually produce a JEP. Hope this helps, -Brian From volker.simonis at gmail.com Thu Jan 29 17:31:09 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 29 Jan 2015 18:31:09 +0100 Subject: What's the status of / relation between "JEP 169: Value Objects" / "Value Types for Java" / "Object Layout" In-Reply-To: <54CA1348.5050903@gmail.com> References: <54CA1348.5050903@gmail.com> Message-ID: Hi Daniel, thanks a lot for sharing your point of view. I haven't been aware of the fact that Project Panama is also working on similar topics (I always thought it is only about the Foreign Function Interface and the next generation JNI). In [1,2] John Rose nicely explains that new data layouts in the JVM heap are very well on the agenda of Project Panama and he also mentions IBM's PackedObjects and Gil Ten's Object Layout proposals. Regards, Volker [1] http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html [2] https://blogs.oracle.com/jrose/entry/the_isthmus_in_the_vm On Thu, Jan 29, 2015 at 12:02 PM, Daniel Latr?moli?re wrote: > >> I just want to quickly summarize my >> current findings here and gently ask for feedback in case you think >> I've totally misunderstood something. Of course any comments and >> additional information is highly welcome as well. > > I don't know if that can be useful, but here is my point of view of > developer oriented towards the question: "What feature for solving my > problem?". This contains probably some or many errors, but it is another > point of view (only mine), if useful. > > I will not use strictly projects/proposal list as the structure of my mail > because content of proposal is changing and it is not my target. I am > oriented towards the final user, i.e. the developer consuming these > projects, not the implementer working in each of these projects. > > I will preferably split in three scopes following my perceived split of job > between developer and runtime. The problem is data, then what can do JVM/GC > with an object? I find two possibilities regarding this domain: move it, > clone it. > > If JVM can clone the object, JVM can also move the object because the clone > will not have the same address, then we have the following three features: > --- > 1) JVM can clone and move objects (Project Valhalla): > Constraint: no complex constructor/no complex finalizer, because lifecycle > of object is managed by JVM (JVM can clone, then JVM can create and destroy > the object like JVM want). Only field affectation constructor, possibly with > simple conversion of data format. > Constraint: immutable, because we don't know which clone is good when one is > modified and because modifying all clones simultaneously is > slow/complex/parallel-unfriendly. > Constraint: non-null because cloning a non-existing object is a non-existing > problem. > > Use-case "Performance": objects to clone for being closer to execution > silicon and better parallelism (registers or cache of CPU/GPU) > - Runtime: expose features of CPU/GPU like SIMD (mostly like a modern > version of javax.vecmath). > - Developer: create custom low-level structures for CPU/GPU parallel > computing. > - Java language: small tuples, like complex numbers (immutable by > performance choice, like SIMD, for being close to silicon; cloned at each > pass by value). > > Use-case "Language": objects to clone for being closer to registers (in > stack, then less allocations in heap; simpler than escape analysis) > - Java language: multiple return values from a method (immutable because > it's a result; cloned, by example, at the return of each delegate or not > even created when stack-only). > > Use-case "Efficiency": others immutable non-null objects possibly concerned > for reducing indirection/improving cache, given by specialization of > collection classes > - Database: primary key for Map (like HashMap)/B-Tree (like MapDB)/SQL (like > JPA). A primary key is immutable and non-null by choice of developer, then > possible gains. > --- > 2) JVM can move but not clone objects > > It's current state of Java objects: > Constraint: developer need to define lifecycle in object, for being > triggered by GC (constructor/finalizer) like current Java class. > Constraint: small object, because when GC move a big object, there is > possibly a noticeable latency. > Constraint: usable directly only in Java code (because native code will need > an indirection level for finding the real address of the object, changing > after each move) > > Improvement by adding custom layout for objects (Project Panama on heap / > ObjectLayout): > Specific constraint: objects which are near identity-less, i.e. only one > other object (the owner) know their identity/have pointer on it. > Non-constraint: applicable to all objects types, contrary to Project > Valhalla. Applicable to complex constructor, because complex constructor can > be inlined in owner code where called. Applicable to mutable objects , > because no cloning then no incoherency. Applicable to nullable objects only > by adding a boolean field in the custom layout for storing potential > existence or non-existence of the inlined object, and updating code testing > nullability for using this boolean. > > Use-case "General efficiency": Custom layout (Inline sub-object in the > object owning it): > - Reduce memory use with less objects then less headers and less pointers. > - Improve cache performance with better locality (objects inlined are in > same cache line, then no reference to follow). > - Applicable to many fields containing reference, requiring only the > referenced object to be invisible from all objects except one (the owner). > > By example, a private field containing an internal ArrayList (without > getter/setter) can probably be replaced by the integer containing the used > size and the reference to backing array, with inlining of the few methods of > ArrayList really used. > It need probably to be driven by developer after real profiling for finding > best ratio between efficiency/code expansion. It will probably have much > more use-cases when AOT will be available and developer-manageable precisely > (Jigsaw???), because most slow work of object-code inlining and following > optimizations can be done at AOT time, while gains will be at running time. > Probably useful for the hottest code (JIT after this pre-optimization at AOT > time) and clearly bad for the coldest code (interpreter then avoid code > expansion), but very useful for the big quantity of code between, which will > gain from AOT if complex optimizations are available. This will very > probably require developer help/instructions/annotations using profiler data > obtained on functional tests of application. > --- > 3) JVM can not move or clone objects (Project Panama off heap / > PackedObjects) > Constraint: developer need to manage externally the full lifecycle of object > and need to choose when creating or destroying it. Object is off-heap and an > handle is on-heap for managing off-heap part. > Constraint: potential fragmentation of free memory when frequently creating > and removing objects not having the same size (taking attention to object > size vs. page size is probably important). > > Use-case "GC Latency": big data structure inducing GC latency when moved if > stored in heap > - All big chunks of data, like Big Data or textures in games, etc. > - Few number of objects for being manageable more explicitly by developer > (without too much work). > > Use-case "Native": communicate with native library > - Modern version of JNI > > Only my 2 cents, > Daniel. From martijnverburg at gmail.com Fri Jan 30 15:02:00 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Fri, 30 Jan 2015 15:02:00 +0000 Subject: Project page updated Message-ID: Hi all, The project page has been updated with lots more useful resources, a contribution guide + how to build valhalla :-). Thanks to many folks on this list for various ideas and Gavin Bierman for helping me get the new page up! Cheers, Martijn -- Cheers, Martijn From brian.goetz at oracle.com Fri Jan 30 15:19:42 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 30 Jan 2015 10:19:42 -0500 Subject: Project page updated In-Reply-To: References: Message-ID: <54CBA10E.4060508@oracle.com> Thanks Martijn! On 1/30/2015 10:02 AM, Martijn Verburg wrote: > Hi all, > > The project page has been updated with lots more useful resources, a > contribution > guide + how to build valhalla :-). > > Thanks to many folks on this list for various ideas and Gavin Bierman for > helping me get the new page up! > > Cheers, > Martijn > > From richard.warburton at gmail.com Fri Jan 30 20:04:26 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Fri, 30 Jan 2015 20:04:26 +0000 Subject: Project page updated In-Reply-To: References: Message-ID: Hi, The project page has been updated with lots more useful resources, a > contribution > guide + how to build valhalla :-). > > Thanks to many folks on this list for various ideas and Gavin Bierman for > helping me get the new page up! > Great stuff Martijn. I've got a couple of suggestions: * "Java/JVM language developers" I suspect you mean day-to-day Java developers. Might be worth clarifying as the title here is slightly ambiguous - I initially interpreted that as as developers working on Java or JVM languages! * It might be worth adding a note at the end to explain to people that they only need to re-run the last 3 commands after the first checkout. And IMO a direct link to the how to build the JDK wiki page for when something goes horrifically wrong. Currently there's only a link to the adoption group itself. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From martijnverburg at gmail.com Fri Jan 30 23:33:03 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Fri, 30 Jan 2015 23:33:03 +0000 Subject: Project page updated In-Reply-To: References: Message-ID: Hi Richard, Great suggestions, will fire out a new version this weekend, hopefully up on Monday. Cheers, Martijn On Friday, 30 January 2015, Richard Warburton wrote: > Hi, > > The project page has been updated with lots more useful resources, a >> contribution >> guide + how to build valhalla :-). >> >> Thanks to many folks on this list for various ideas and Gavin Bierman for >> helping me get the new page up! >> > > Great stuff Martijn. I've got a couple of suggestions: > > * "Java/JVM language developers" I suspect you mean day-to-day Java > developers. Might be worth clarifying as the title here is slightly > ambiguous - I initially interpreted that as as developers working on Java > or JVM languages! > * It might be worth adding a note at the end to explain to people that > they only need to re-run the last 3 commands after the first checkout. And > IMO a direct link to the how to build the JDK wiki page for when something > goes horrifically wrong. Currently there's only a link to the adoption > group itself. > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > -- Cheers, Martijn