From john.r.rose at oracle.com Mon Jun 1 21:31:09 2015 From: john.r.rose at oracle.com (John Rose) Date: Mon, 1 Jun 2015 14:31:09 -0700 Subject: experiments with fused strings Message-ID: <118FC07C-7A33-4AFA-8FE4-9E95A92F4B2E@oracle.com> This dialog is cut-n-pasted from the Valhalla chat room. I'd like to continue it via Email. [5/22/15, 4:54:22 PM] John Rose: Here?s an experiment I?d like to try some day; maybe soon. It has three forks: 1. create a hybrid instance/array object with a variable tail of raw primitive data, 2. adapt var-handles to access data in the tail (the array part), 3. make the var-handles (temporary access capabilities) optimize away using a combination of EA and Heisenbox logic. Then, 4. shake until optimized. The end result will be a credible upgrade for our current string representation (fusing the header and data array). [5/22/15, 4:55:50 PM] Stuart W. Marks: Kind of like the old char[1]-at-the-end-of-the-struct trick. John: Yes, exactly so. Note that this is orthogonal to the char/byte union. We want both tricks. I?m thinking that Paul S. has most of the parts for 2. and Roland can make progress on 3/4. Main sticking point is a prototype for 1. My inclination is to start with Stefan K.?s cleaned up mapping logic. It would be a small-ish demo use case showing the interplay of flattened objects, array-like access methods, and value-like types. (The var-handles have to at least compile to identity-free values, or we cannot replace strings with the fancy logic.) [5/26/15, 3:30:29 AM] Paul Sandoz: John, so for say a String instance the character array length and elements will be aligned (perhaps with suitable padding) after that of the hash code field? Paul: Oh, and about hashCode, it sucks that the algorithm is explicitly specified, i don't think it's possible to vectorize by viewing as long[] Re, VarHandles, i suspect it comes down to what object would be used via Unsafe to access the array contents. If it's the hybrid instance, at a particular offset from that (based on the hybrid class layout), then i believe it should be easy to create a VarHandle from the hybrid type (e.g. Unsafe.getEmbeddedArrayOffset(Class ) and stash it as a final static field. John: Yes: struct Instance_of_String { headerBits header; jint hash_field; long value_bits[]; } Vectorizing the hash could be done, awkwardly, as a packed 4x16-bit multiply times 31 exp (iota 4). In places (like JVM internals) where we have freedom to discard that thing, we should use FarmHash. Yes, that's about how the VarHandle play would work. For String per se we could use Unsafe directly, but VHs are nicer. Also, the creation of the hybrids can be done via Unsafe and wrapped in a method handle, as if by Lookup.findConstructor. Eventually our bytecodes should learn to do the dance instead of the dance, and then we can push the MH trick back down to the bytecode level. But we don't have to depend on it for prototyping. Stuart: Where would the length be stored? I'd prefer to smash it into the value_bits, in a variable-length form that favors short strings. string_value := short_bytes | long_bytes | long_chars; short_bytes := length(1..255) byte[length]; long_bytes := length(int32) | byte[length] | long_chars; long_chars := negative_length(int32) | char[length] (Some extra fiddling required to disambiguate the short guy from the long ones; it's easiest if the long cases can be expressed in 24 bits so the first byte can be zero.) Stuart: I think we need a zero-length string. [5/26/15, 1:35:07 PM] Brian Goetz: How about a negative length string, which, when concatenated with another string, lops characters off the end? Stuart: Is that like an anti-string? John: The empty string does not need to be covered by the short case; it can be the shortest long case and still hide in one cache line. (Size distributions of such things always tend to be singular at the zero point.) Stuart: Oh! So zero-length is allowed, it's just that it's the "long" form. OK. John: Yes. And you only need a small short form, since once you get past a few cache lines, you can afford to drop in a whole 32-bit size field. (Any log-sized length thingy is only a real overhead when its value is very close to zero.) Open question whether to put in a short form for UTF16 also. I would make the decision by examining code paths for decoding the cases. Stuart: Have you been talking to Sherman and Brent? John: Not recently. I know about their work. (Actually, Sherman's part of it.) It's moving usefully in this direction, to the point of having two formats for the string body. I've mainly talked to Vladimir about his part in it. Stuart: I know they're working on multiple (well, two) representations, but not much more than that. I'd be uncomfortable tying the object structure/layout to the actual string encoding though. The current model for storing strings in bytes is ASCII or maybe ISO 8859-1, but I keep hoping for UTF-8. On another note, I guess you've talked to the GC guys about the additional complexity determining the object's size, based on information not in the object header. Or is there additional stuff in the header? John: Good point; we want the object structure to be just a bag of bits (probably vectorized in longs). The interpretation of that structure would not be wired into the library but rather defined by Java library code. The GC will not know about this code. There will have to be a lower-level length indicator for the GC, probably just an array length field like today. (It might contribute to the library level sizing logic, or not.) I have been talking to the GC folks about this sort of thing for a long time. Stefan K's refactoring of GC object mapping code may help clear the ground for it. Stuart: (Sorry, I keep coming up with issues.) There is also the object identity issue that might arise when switching encodings. If a string is re-encoded its size might change; that implies a different object, right? Unfortunately == will need to stay the same. I happen to know that the permissions libraries depend on == on interned Strings, for instance.... OTOH maybe encodings don't change on the fly, and a String has whatever encoding it has at creation time. I'm not sure of the current state of the design. Paul: John, it sounds like we need a few fundamental things to get this working: - Methods on unsafe to 1) instantiate an object with a certain sized tail of bytes (length packing rules could apply) 2) obtain the offset to the tail; and 3) obtain the tail length, which should be treated as an unsigned value regarding bounds checks (perhaps via the Array check range methods Roland is working on to intrinsify) - GC to recognise this structure Once that is in place we can build on top, like you say with MHs and VHs, repeat and rinse until the generated code is good. (I dunno if there are issues for class redefinition and debugging, probably?) Paul: Re: hash code, yes for chars i think it might be possible since they are unsigned IIRC h = +[31^3, 31^2, 31^1, 31^0] * [a[0], a[1], a[2], a[3]] + 31^4 * h. I was struggling in Java code with byte[], the signed nature really seems to mess things up. Perhaps it might be easier in machine code but i could not find a way in Java code. Also, perhaps we should include new hash code methods in Arrays? John, what happens if one invokes a MemberName with fewer arguments than it's arity? For VarHandles it's very convenient to link to a single method that accepts no arguments and throws an exception for cases where an operation is unsupported (e.g. update operations for final fields or numeric operations for refs). It works (with fast debug and -XX:+VerifyMethodHandles, plus i cannot find any assertions/checks on the arity) but i don't trust it, as i suspect it could leave the stack in an undefined state. Here is a simple Scala example: class A { object AO { val x: Int = Test.getY() } def getAOX(): Int = { AO.x } } object Test { var y: Int = 0; def getY(): Int = { y = y + 1 y } def main(args: Array[String]) { print(new A().getAOX()) print(new A().getAOX()) print(new A().getAOX()) } } [I restarted Skype and it has lost all messages from this hallway from the 1st of May up to today :-( ] John: I sent [via email] a super-quick sketch about static bundles. Paul, I'll convert our previous exchange to Email. From maurizio.cimadamore at oracle.com Tue Jun 2 13:43:33 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 02 Jun 2015 13:43:33 +0000 Subject: hg: valhalla/valhalla/jdk: Fix: forgot to hg add/push SpecializerSignature class source. Message-ID: <201506021343.t52DhXFW003225@aojmv0008.oracle.com> Changeset: 8aac91963224 Author: mcimadamore Date: 2015-06-02 14:43 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/8aac91963224 Fix: forgot to hg add/push SpecializerSignature class source. + src/java.base/share/classes/valhalla/specializer/SpecializerSignature.java From maurizio.cimadamore at oracle.com Wed Jun 3 12:52:53 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 03 Jun 2015 12:52:53 +0000 Subject: hg: valhalla/valhalla/jdk: Misc specializer fixes: Message-ID: <201506031252.t53CqrYB016634@aojmv0008.oracle.com> Changeset: 895d65f8023b Author: mcimadamore Date: 2015-06-03 13:52 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/895d65f8023b Misc specializer fixes: * specialization of indy static args doesn't erase types * generic method specialization should use SpecializerSignature where available * extra attributes (SpecializerSignature, Bridge) should be dropped from specialized class * added smoke test for specialized lambdas ! src/java.base/share/classes/java/lang/invoke/GenericMethodSpecializer.java ! src/java.base/share/classes/valhalla/specializer/Specializer.java + test/valhalla/test/valhalla/specializer/LambdaTest.java From maurizio.cimadamore at oracle.com Wed Jun 3 13:10:32 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 03 Jun 2015 13:10:32 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: Missing type-witness on generic method call to map() causes build failures with certain boot JDK Message-ID: <201506031310.t53DAWF8019518@aojmv0008.oracle.com> Changeset: 52d2779ddd19 Author: mcimadamore Date: 2015-06-03 14:10 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/52d2779ddd19 Fix: Missing type-witness on generic method call to map() causes build failures with certain boot JDK ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java From maurizio.cimadamore at oracle.com Fri Jun 5 14:08:47 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 05 Jun 2015 14:08:47 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: add support for cross-specialization boundaries accessors. Message-ID: <201506051408.t55E8l6K006980@aojmv0008.oracle.com> Changeset: 460e954a0da9 Author: mcimadamore Date: 2015-06-05 15:08 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/460e954a0da9 Enhancement: add support for cross-specialization boundaries accessors. * use specialized host class when emitting invokespecial generic call sites * fix missing specialization of receiver type in the BSM args of generic calls indy * workaround: relax access MH constant-related checks on anonymous class (coming from UNSAFE.dAC) ! src/java.base/share/classes/java/lang/invoke/GenericMethodSpecializer.java ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java ! src/java.base/share/classes/sun/invoke/util/VerifyAccess.java ! src/java.base/share/classes/valhalla/specializer/Specializer.java From brian.goetz at oracle.com Fri Jun 5 16:59:46 2015 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Fri, 05 Jun 2015 16:59:46 +0000 Subject: hg: valhalla/valhalla/jdk: First cut at anyfied Pipeline Message-ID: <201506051659.t55GxkCx003351@aojmv0008.oracle.com> Changeset: ce2a34f22170 Author: briangoetz Date: 2015-06-05 12:59 -0400 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/ce2a34f22170 First cut at anyfied Pipeline ! src/java.base/share/classes/java/anyutil/function/IntConsumer.java + src/java.base/share/classes/java/anyutil/stream/Pipeline.java ! src/java.base/share/classes/java/lang/invoke/ProxyClassesDumper.java From brian.goetz at oracle.com Fri Jun 5 17:29:41 2015 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Fri, 05 Jun 2015 17:29:41 +0000 Subject: hg: valhalla/valhalla/jdk: Use lambdas instead of unrolled inner classes in Pipeline.filter; still not yet working for map (awaiting support for lambdas in specializable generic methods.) Message-ID: <201506051729.t55HTfmi008291@aojmv0008.oracle.com> Changeset: 14cde5b4cb25 Author: briangoetz Date: 2015-06-05 13:29 -0400 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/14cde5b4cb25 Use lambdas instead of unrolled inner classes in Pipeline.filter; still not yet working for map (awaiting support for lambdas in specializable generic methods.) ! src/java.base/share/classes/java/anyutil/stream/Pipeline.java From maurizio.cimadamore at oracle.com Fri Jun 5 23:58:34 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 05 Jun 2015 23:58:34 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: add support for cross-specialization boundaries accessors. Message-ID: <201506052358.t55NwYJI011993@aojmv0008.oracle.com> Changeset: 156d2dfb901f Author: mcimadamore Date: 2015-06-05 15:03 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/156d2dfb901f Enhancement: add support for cross-specialization boundaries accessors. * retain generic info associated with accessors in Lower * simulate accessors in specialzied classes * add heuristics to detect cross-specialization boundaries * add flag to mark specializable synthetic symbols * add specialization support for invokestatic/getstatic/putstatic * add smoke tests (contributed by Jan Lahoda) ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Flags.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/IndifierTranslator.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Lower.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Code.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeMaker.java + test/tools/javac/valhalla/typespec/SpecializedAccessors.java + test/tools/javac/valhalla/typespec/SpecializedAccessors2.java + test/tools/javac/valhalla/typespec/SpecializedAccessors2Aux.java + test/tools/javac/valhalla/typespec/SpecializedAccessors3.java From maurizio.cimadamore at oracle.com Sat Jun 6 01:25:41 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Sat, 06 Jun 2015 01:25:41 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: tweak heuristics to detect cross-specialization boundaries Message-ID: <201506060125.t561Pfdl027309@aojmv0008.oracle.com> Changeset: 9a9deb54d415 Author: mcimadamore Date: 2015-06-06 02:25 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/9a9deb54d415 Fix: tweak heuristics to detect cross-specialization boundaries * generalize SpecializeTypes translator so that all expression unerased types are retained * add more tests ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Lower.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! test/tools/javac/valhalla/typespec/SpecializedAccessors.java ! test/tools/javac/valhalla/typespec/SpecializedAccessors2.java ! test/tools/javac/valhalla/typespec/SpecializedAccessors2Aux.java ! test/tools/javac/valhalla/typespec/SpecializedAccessors3.java + test/tools/javac/valhalla/typespec/SpecializedAccessors4.java From maurizio.cimadamore at oracle.com Sat Jun 6 01:26:13 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Sat, 06 Jun 2015 01:26:13 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: revert anyfied Pipeline fields to private access Message-ID: <201506060126.t561QDlG027483@aojmv0008.oracle.com> Changeset: 922a3c453a3c Author: mcimadamore Date: 2015-06-06 02:23 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/922a3c453a3c Enhancement: revert anyfied Pipeline fields to private access ! src/java.base/share/classes/java/anyutil/stream/Pipeline.java From maurizio.cimadamore at oracle.com Sat Jun 6 01:38:13 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Sat, 06 Jun 2015 01:38:13 +0000 Subject: hg: valhalla/valhalla/langtools: Fix tabs in SpecializedAccessors4.java Message-ID: <201506060138.t561cDlx029554@aojmv0008.oracle.com> Changeset: a0d7ac464e8b Author: mcimadamore Date: 2015-06-06 02:38 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/a0d7ac464e8b Fix tabs in SpecializedAccessors4.java ! test/tools/javac/valhalla/typespec/SpecializedAccessors4.java From maurizio.cimadamore at oracle.com Wed Jun 10 11:11:07 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 10 Jun 2015 11:11:07 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: changes to Types.SignatureGenerator breaks clients (i.e. REPL). Message-ID: <201506101111.t5ABB7q2022020@aojmv0008.oracle.com> Changeset: 41ab3a104536 Author: mcimadamore Date: 2015-06-10 12:10 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/41ab3a104536 Fix: changes to Types.SignatureGenerator breaks clients (i.e. REPL). ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java From simon at ochsenreither.de Fri Jun 12 07:57:04 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Fri, 12 Jun 2015 09:57:04 +0200 (CEST) Subject: =?UTF-8?Q?Project_Valhalla_=E2=80=93_Value_Types_sl?= =?UTF-8?Q?ides_from_ScalaDays_Amsterdam_2015?= Message-ID: <1173011249.28218.1434095824971.JavaMail.open-xchange@srv005.service.ps-server.net> Hey everone! If you are interested, here are the slides I used in my talk at ScalaDays Amsterdam: oxnrtr.de/scaladays2015 I hope the slides gave a correct description of the current situation... Thanks! Bye, Simon From brian.goetz at oracle.com Fri Jun 12 08:47:58 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 12 Jun 2015 09:47:58 +0100 Subject: =?windows-1252?Q?Re=3A_Project_Valhalla_=96_Value_Types_slides_f?= =?windows-1252?Q?rom_ScalaDays_Amsterdam_2015?= In-Reply-To: <1173011249.28218.1434095824971.JavaMail.open-xchange@srv005.service.ps-server.net> References: <1173011249.28218.1434095824971.JavaMail.open-xchange@srv005.service.ps-server.net> Message-ID: Very nice. A few comments: #11: object headers are typically 2 words; a klass pointer and a word for various identity support (GC mark bits, locking, etc.) Which makes the density story with small objects even worse. In general, values give us both DENSITY (memory efficiency) and FLATNESS (cache efficiency.) #36: in addition to the __ByValue obviously being a syntax placeholder, the restrictions outlined e.g. in #38 (must say final) will also go away ? we?re just not focusing at all on syntax for the time being. #39: equality on value types will work as it does on primitives (as you suggest on #88) #121: good summary On Jun 12, 2015, at 8:57 AM, Simon Ochsenreither wrote: > Hey everone! > > If you are interested, here are the slides I used in my talk at ScalaDays > Amsterdam: > > oxnrtr.de/scaladays2015 > > I hope the slides gave a correct description of the current situation... > > Thanks! > > Bye, > > Simon From vitalyd at gmail.com Fri Jun 12 11:09:13 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 12 Jun 2015 07:09:13 -0400 Subject: =?UTF-8?Q?Re=3A_Project_Valhalla_=E2=80=93_Value_Types_slides_from_Sca?= =?UTF-8?Q?laDays_Amsterdam_2015?= In-Reply-To: References: <1173011249.28218.1434095824971.JavaMail.open-xchange@srv005.service.ps-server.net> Message-ID: A few questions: 1) regarding equality being like primitives - is that for == only? Wll value types support overriding equals()? 2) value types cannot hold a ref field that's null? I think I saw that on one of the slides. If so, why? Thanks sent from my phone On Jun 12, 2015 4:49 AM, "Brian Goetz" wrote: > Very nice. > > A few comments: > > #11: object headers are typically 2 words; a klass pointer and a word for > various identity support (GC mark bits, locking, etc.) Which makes the > density story with small objects even worse. > > In general, values give us both DENSITY (memory efficiency) and FLATNESS > (cache efficiency.) > > #36: in addition to the __ByValue obviously being a syntax placeholder, > the restrictions outlined e.g. in #38 (must say final) will also go away ? > we?re just not focusing at all on syntax for the time being. > > #39: equality on value types will work as it does on primitives (as you > suggest on #88) > > #121: good summary > > > > On Jun 12, 2015, at 8:57 AM, Simon Ochsenreither > wrote: > > > Hey everone! > > > > If you are interested, here are the slides I used in my talk at ScalaDays > > Amsterdam: > > > > oxnrtr.de/scaladays2015 > > > > I hope the slides gave a correct description of the current situation... > > > > Thanks! > > > > Bye, > > > > Simon > > From simon at ochsenreither.de Fri Jun 12 21:07:19 2015 From: simon at ochsenreither.de (Simon Ochsenreither) Date: Fri, 12 Jun 2015 23:07:19 +0200 Subject: Project Valhalla =?windows-1252?Q?=96_Value_Types_slid?= =?windows-1252?Q?es_from_ScalaDays_Amsterdam_2015?= In-Reply-To: References: <1173011249.28218.1434095824971.JavaMail.open-xchange@srv005.service.ps-server.net> Message-ID: <557B4A07.20905@ochsenreither.de> Hi Brian, hi Vitaly, thanks for your comments! > #11: object headers are typically 2 words; a klass pointer and a word for various identity support (GC mark bits, locking, etc.) Which makes the density story with small objects even worse. Yes, absolutely! I changed the original from "4 byte" to ">= 4 byte" only because I was concerned that it would provoke some non-useful response saying "but obscure JVM vendor implementation X version 23.42 does X in only 4 bytes!!!". So I decided that "at least 4 bytes" would be better ... but I would be happy to revise it again if you could provide more information. (E. g. "_everyone_ implements it this way...") (Btw, doesn't the klass pointer participate in compressed oops?) > In general, values give us both DENSITY (memory efficiency) and FLATNESS (cache efficiency.) You are right, I should have mentioned that! (There are a few things I mentioned that weren't on the slides, but I forgot a lot to mention, too!) For instance, I papered over the whole topic of boxing! :-/ > #36: in addition to the __ByValue obviously being a syntax placeholder, Yes, I made that clear in the talk. Although I should have added that to the slides too. > the restrictions outlined e.g. in #38 (must say final) will also go away ? > we?re just not focusing at all on syntax for the time being. Great, thanks for the info! > #39: equality on value types will work as it does on primitives (as you suggest on #88) Thanks! > #121: good summary :-) > 2) value types cannot hold a ref field that's null? I think I saw that on one of the slides. If so, why? Maybe that was worded misleadingly. What I meant was that the value type itself is non-nullable. It can of course have ref fields that can be null. (I thought the example made it clear ...) Thanks to both of you for the review. I think the didn't go too well, because I was working on my slides until the last minute (projector/filming equipment required 16:9 which meant I had 300 pixels less height than I assumed). Then I didn't get the mirroring working on my notebook, so I only had the slides in my back, the audience in front, and couldn't move to a better place because I needed to click through the slides on my notebook. :-/ I guess my English grammar was falling apart, too, due to the lack of sleep. Bye, Simon From vitalyd at gmail.com Fri Jun 12 21:26:02 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 12 Jun 2015 17:26:02 -0400 Subject: =?UTF-8?Q?Re=3A_Project_Valhalla_=E2=80=93_Value_Types_slides_from_Sca?= =?UTF-8?Q?laDays_Amsterdam_2015?= In-Reply-To: <557B4A07.20905@ochsenreither.de> References: <1173011249.28218.1434095824971.JavaMail.open-xchange@srv005.service.ps-server.net> <557B4A07.20905@ochsenreither.de> Message-ID: Simon, That's my fault - I misread the slide. So ignore that part of my initial email :). Yes, class ref can participate in being compressed. On 64 bit hotspot, you're looking at 12 bytes overhead (8 mark word + 4 compressed class); the 4 byte alignment gap is allocatable to fields that fit (i.e. int and coops and smaller), AFAIK. sent from my phone On Jun 12, 2015 5:07 PM, "Simon Ochsenreither" wrote: > Hi Brian, hi Vitaly, > > thanks for your comments! > > > #11: object headers are typically 2 words; a klass pointer and a word > for various identity support (GC mark bits, locking, etc.) Which makes the > density story with small objects even worse. > > Yes, absolutely! I changed the original from "4 byte" to ">= 4 byte" > only because I was concerned that it would provoke some non-useful > response saying "but obscure JVM vendor implementation X version 23.42 > does X in only 4 bytes!!!". > > So I decided that "at least 4 bytes" would be better ... but I would be > happy to revise it again if you could provide more information. (E. g. > "_everyone_ implements it this way...") > > (Btw, doesn't the klass pointer participate in compressed oops?) > > > In general, values give us both DENSITY (memory efficiency) and FLATNESS > (cache efficiency.) > > You are right, I should have mentioned that! (There are a few things I > mentioned that weren't on the slides, but I forgot a lot to mention, too!) > > For instance, I papered over the whole topic of boxing! :-/ > > > #36: in addition to the __ByValue obviously being a syntax placeholder, > > Yes, I made that clear in the talk. Although I should have added that to > the slides too. > > > the restrictions outlined e.g. in #38 (must say final) will also go away > ? > > we?re just not focusing at all on syntax for the time being. > > Great, thanks for the info! > > > #39: equality on value types will work as it does on primitives (as you > suggest on #88) > > Thanks! > > > #121: good summary > > :-) > > > 2) value types cannot hold a ref field that's null? I think I saw that > on one of the slides. If so, why? > > Maybe that was worded misleadingly. What I meant was that the value type > itself is non-nullable. It can of course have ref fields that can be > null. (I thought the example made it clear ...) > > Thanks to both of you for the review. > I think the didn't go too well, because I was working on my slides until > the last minute (projector/filming equipment required 16:9 which meant I > had 300 pixels less height than I assumed). Then I didn't get the > mirroring working on my notebook, so I only had the slides in my back, > the audience in front, and couldn't move to a better place because I > needed to click through the slides on my notebook. :-/ > I guess my English grammar was falling apart, too, due to the lack of > sleep. > > Bye, > > Simon > From maurizio.cimadamore at oracle.com Mon Jun 15 13:17:40 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 15 Jun 2015 13:17:40 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: add mangler to facilitate classdynamic experiments Message-ID: <201506151317.t5FDHeqI008633@aojmv0008.oracle.com> Changeset: b8f3a5f59573 Author: mcimadamore Date: 2015-06-15 14:17 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/b8f3a5f59573 Enhancement: add mangler to facilitate classdynamic experiments + src/java.base/share/classes/valhalla/classdyn/Mangler.java ! src/java.base/share/classes/valhalla/specializer/Specializer.java + test/valhalla/test/valhalla/classdyn/ManglerTest.java From maurizio.cimadamore at oracle.com Mon Jun 15 17:58:35 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 15 Jun 2015 17:58:35 +0000 Subject: hg: valhalla/valhalla/hotspot: Enhancement: retrieve host class associated with Unsafe.defineAnonymousClass Message-ID: <201506151758.t5FHwZKi029575@aojmv0008.oracle.com> Changeset: 06a7383a22cc Author: mcimadamore Date: 2015-06-15 18:58 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/hotspot/rev/06a7383a22cc Enhancement: retrieve host class associated with Unsafe.defineAnonymousClass Contributed-by: vladimir.x.ivanov at oracle.com ! src/share/vm/prims/unsafe.cpp From maurizio.cimadamore at oracle.com Mon Jun 15 18:00:32 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 15 Jun 2015 18:00:32 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: improve MH access checks w.r.t. classes defined with Unsafe.dAC Message-ID: <201506151800.t5FI0Xlw029906@aojmv0008.oracle.com> Changeset: c28ea949f8cc Author: mcimadamore Date: 2015-06-15 19:00 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/c28ea949f8cc Enhancement: improve MH access checks w.r.t. classes defined with Unsafe.dAC * Add new Unsafe.getHostClass() method (see related VM patch) * Added recursive access check logic that walks through the hostclass chain ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java ! src/java.base/share/classes/sun/invoke/util/VerifyAccess.java ! src/java.base/share/classes/sun/misc/Unsafe.java From forax at univ-mlv.fr Tue Jun 16 08:27:50 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 16 Jun 2015 10:27:50 +0200 Subject: hg: valhalla/valhalla/jdk: Enhancement: improve MH access checks w.r.t. classes defined with Unsafe.dAC In-Reply-To: <201506151800.t5FI0Xlw029906@aojmv0008.oracle.com> References: <201506151800.t5FI0Xlw029906@aojmv0008.oracle.com> Message-ID: <557FDE06.60403@univ-mlv.fr> Hi Maurizio, using 7 classes here is in my opinion overkill what about something like this ? // you can add documentation here, more readable IMO interface VisibilityCheck { boolean test(Class c1, Class c2); } enum VisibiltyChecks implements VisibilityCheck { CLASS_EQUALS, ASSIGNABLE, SAME_PACKAGE, SAME_PACKAGE_MEMBER, RELATED_CLASS, SUBCLASS ; @Overrides public boolean test(Class c1, Class c2) { switch(this) { case CLASS_EQUALS: return c1 == c2; case ASSIGNABLE: return c1.isAssignable(c2); ... default: throw new AssertionError(); } } } private static final VisibilityCheck classAccessible(int allowedModes) { return new VisibilityCheck() { @Override public boolean test(Class c1, Class c2) { return isClassAccessible(c1, c2, allowedModes); } }; } public static boolean walkHostClass(Class c, Class lookupClass, VisibilityCheck p) { for(;lookupClass != null; lookupClass = UNSAFE.getHostClass(lookupClass)) { if (p.test(c, lookupClass)) { return true; } } return false; } regards, R?mi On 06/15/2015 08:00 PM, maurizio.cimadamore at oracle.com wrote: > Changeset: c28ea949f8cc > Author: mcimadamore > Date: 2015-06-15 19:00 +0100 > URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/c28ea949f8cc > > Enhancement: improve MH access checks w.r.t. classes defined with Unsafe.dAC > * Add new Unsafe.getHostClass() method (see related VM patch) > * Added recursive access check logic that walks through the hostclass chain > > ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java > ! src/java.base/share/classes/sun/invoke/util/VerifyAccess.java > ! src/java.base/share/classes/sun/misc/Unsafe.java > From forax at univ-mlv.fr Tue Jun 16 08:33:08 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 16 Jun 2015 10:33:08 +0200 Subject: hg: valhalla/valhalla/jdk: Enhancement: improve MH access checks w.r.t. classes defined with Unsafe.dAC In-Reply-To: <557FDE06.60403@univ-mlv.fr> References: <201506151800.t5FI0Xlw029906@aojmv0008.oracle.com> <557FDE06.60403@univ-mlv.fr> Message-ID: <557FDF44.6080106@univ-mlv.fr> I should have add that downgrading lambdas to inner classes is that now all the inner classes are stored on disk and loaded and initialized when the class is initialized (unlike lambdas that are lazy constructed). R?mi On 06/16/2015 10:27 AM, Remi Forax wrote: > Hi Maurizio, > using 7 classes here is in my opinion overkill > > what about something like this ? > > // you can add documentation here, more readable IMO > interface VisibilityCheck { > boolean test(Class c1, Class c2); > } > > enum VisibiltyChecks implements VisibilityCheck { > CLASS_EQUALS, > ASSIGNABLE, > SAME_PACKAGE, > SAME_PACKAGE_MEMBER, > RELATED_CLASS, > SUBCLASS > ; > > @Overrides > public boolean test(Class c1, Class c2) { > switch(this) { > case CLASS_EQUALS: > return c1 == c2; > case ASSIGNABLE: > return c1.isAssignable(c2); > ... > default: > throw new AssertionError(); > } > } > } > > private static final VisibilityCheck classAccessible(int allowedModes) { > return new VisibilityCheck() { > @Override > public boolean test(Class c1, Class c2) { > return isClassAccessible(c1, c2, allowedModes); > } > }; > } > > public static boolean walkHostClass(Class c, Class lookupClass, > VisibilityCheck p) { > for(;lookupClass != null; lookupClass = > UNSAFE.getHostClass(lookupClass)) { > if (p.test(c, lookupClass)) { > return true; > } > } > return false; > } > > regards, > R?mi > > On 06/15/2015 08:00 PM, maurizio.cimadamore at oracle.com wrote: >> Changeset: c28ea949f8cc >> Author: mcimadamore >> Date: 2015-06-15 19:00 +0100 >> URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/c28ea949f8cc >> >> Enhancement: improve MH access checks w.r.t. classes defined with >> Unsafe.dAC >> * Add new Unsafe.getHostClass() method (see related VM patch) >> * Added recursive access check logic that walks through the hostclass >> chain >> >> ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java >> ! src/java.base/share/classes/sun/invoke/util/VerifyAccess.java >> ! src/java.base/share/classes/sun/misc/Unsafe.java >> > From maurizio.cimadamore at oracle.com Tue Jun 16 09:56:36 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 16 Jun 2015 10:56:36 +0100 Subject: hg: valhalla/valhalla/jdk: Enhancement: improve MH access checks w.r.t. classes defined with Unsafe.dAC In-Reply-To: <557FDF44.6080106@univ-mlv.fr> References: <201506151800.t5FI0Xlw029906@aojmv0008.oracle.com> <557FDE06.60403@univ-mlv.fr> <557FDF44.6080106@univ-mlv.fr> Message-ID: <557FF2D4.3050109@oracle.com> On 16/06/15 09:33, Remi Forax wrote: > I should have add that downgrading lambdas to inner classes is that > now all the inner classes are stored on disk and loaded and > initialized when the class is initialized (unlike lambdas that are > lazy constructed). Not sure I get this point? I know inner class add static footprint - the problem that I was trying to avoid is a SSO caused by the fact that the very code that is checking MH accessibility needs itself to refer to MH (if you are using lambdas). Regarding the refactoring, of course, when the code is stable enough and we are convinced this is the path we should follow, the enum refactoring seems a sensible one. Maurizio > > R?mi > > On 06/16/2015 10:27 AM, Remi Forax wrote: >> Hi Maurizio, >> using 7 classes here is in my opinion overkill >> >> what about something like this ? >> >> // you can add documentation here, more readable IMO >> interface VisibilityCheck { >> boolean test(Class c1, Class c2); >> } >> >> enum VisibiltyChecks implements VisibilityCheck { >> CLASS_EQUALS, >> ASSIGNABLE, >> SAME_PACKAGE, >> SAME_PACKAGE_MEMBER, >> RELATED_CLASS, >> SUBCLASS >> ; >> >> @Overrides >> public boolean test(Class c1, Class c2) { >> switch(this) { >> case CLASS_EQUALS: >> return c1 == c2; >> case ASSIGNABLE: >> return c1.isAssignable(c2); >> ... >> default: >> throw new AssertionError(); >> } >> } >> } >> >> private static final VisibilityCheck classAccessible(int allowedModes) { >> return new VisibilityCheck() { >> @Override >> public boolean test(Class c1, Class c2) { >> return isClassAccessible(c1, c2, allowedModes); >> } >> }; >> } >> >> public static boolean walkHostClass(Class c, Class lookupClass, >> VisibilityCheck p) { >> for(;lookupClass != null; lookupClass = >> UNSAFE.getHostClass(lookupClass)) { >> if (p.test(c, lookupClass)) { >> return true; >> } >> } >> return false; >> } >> >> regards, >> R?mi >> >> On 06/15/2015 08:00 PM, maurizio.cimadamore at oracle.com wrote: >>> Changeset: c28ea949f8cc >>> Author: mcimadamore >>> Date: 2015-06-15 19:00 +0100 >>> URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/c28ea949f8cc >>> >>> Enhancement: improve MH access checks w.r.t. classes defined with >>> Unsafe.dAC >>> * Add new Unsafe.getHostClass() method (see related VM patch) >>> * Added recursive access check logic that walks through the >>> hostclass chain >>> >>> ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java >>> ! src/java.base/share/classes/sun/invoke/util/VerifyAccess.java >>> ! src/java.base/share/classes/sun/misc/Unsafe.java >>> >> > From maurizio.cimadamore at oracle.com Tue Jun 16 12:17:48 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 16 Jun 2015 12:17:48 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: lambda to method reference translation doesn't preserve unerased types Message-ID: <201506161217.t5GCHmGi016279@aojmv0008.oracle.com> Changeset: 53dd34e16447 Author: mcimadamore Date: 2015-06-16 13:17 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/53dd34e16447 Fix: lambda to method reference translation doesn't preserve unerased types ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java + test/tools/javac/valhalla/typespec/TestLambda01.java From maurizio.cimadamore at oracle.com Tue Jun 23 13:33:37 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 23 Jun 2015 13:33:37 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: cleanup specialized accessor code Message-ID: <201506231333.t5NDXcrB008340@aojmv0008.oracle.com> Changeset: 5f295bd9529c Author: mcimadamore Date: 2015-06-23 14:33 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/5f295bd9529c Fix: cleanup specialized accessor code * add customizable flags to static BSM args (to tweak how specialization should behave) * cleanup code in Lower.access(Symbol) * Added test for generic method calling private generic method (both static and instance) ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Lower.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java + test/tools/javac/valhalla/typespec/SpecializedAccessors5.java From maurizio.cimadamore at oracle.com Tue Jun 23 13:38:42 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 23 Jun 2015 13:38:42 +0000 Subject: hg: valhalla/valhalla/langtools: Fix bad indentation in test Message-ID: <201506231338.t5NDcgBZ009361@aojmv0008.oracle.com> Changeset: 16384b2d20ea Author: mcimadamore Date: 2015-06-23 14:38 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/16384b2d20ea Fix bad indentation in test ! test/tools/javac/valhalla/typespec/SpecializedAccessors5.java From maurizio.cimadamore at oracle.com Tue Jun 23 13:39:07 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 23 Jun 2015 13:39:07 +0000 Subject: hg: valhalla/valhalla/jdk: Fix: missing erasure in code for specializing static specialized generic call Message-ID: <201506231339.t5NDd8Kl009551@aojmv0008.oracle.com> Changeset: 3e43978c92ea Author: mcimadamore Date: 2015-06-23 14:35 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/3e43978c92ea Fix: missing erasure in code for specializing static specialized generic call ! src/java.base/share/classes/java/lang/invoke/GenericMethodSpecializer.java From maurizio.cimadamore at oracle.com Wed Jun 24 21:06:45 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 24 Jun 2015 21:06:45 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: add support for lambda expressions inside generic (specializable) methods Message-ID: <201506242106.t5OL6kg5022571@aojmv0008.oracle.com> Changeset: 22afe0be1798 Author: mcimadamore Date: 2015-06-24 22:05 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/22afe0be1798 Enhancement: add support for lambda expressions inside generic (specializable) methods * tweaked altMetafactory protocol to accept additional set of static args (corresponding to generic method callsite args) * add support for specializing strings bootstrap static args * added test ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java + test/tools/javac/valhalla/typespec/Lambda01.java ! test/tools/javac/valhalla/typespec/items/tests/TestCapture.java ! test/tools/javac/valhalla/typespec/items/tests/TestGeneric2GenericCall.java ! test/tools/javac/valhalla/typespec/items/tests/TestGenericSpecializedConstructor.java ! test/tools/javac/valhalla/typespec/items/tests/TestIndy.java ! test/tools/javac/valhalla/typespec/items/tests/TestIndyFactory.java ! test/tools/javac/valhalla/typespec/items/tests/TestLambda.java ! test/tools/javac/valhalla/typespec/items/tests/TestRespecialization.java From maurizio.cimadamore at oracle.com Wed Jun 24 21:08:58 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 24 Jun 2015 21:08:58 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: add support for lambda expressions inside generic (specializable) methods Message-ID: <201506242108.t5OL8xZh022941@aojmv0008.oracle.com> Changeset: 91cb6214f7e2 Author: mcimadamore Date: 2015-06-24 22:08 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/91cb6214f7e2 Enhancement: add support for lambda expressions inside generic (specializable) methods * tweaked altMetafactory protocol to accept additional set of static args (corresponding to generic method callsite args) * generalize specialization of invokedynamic instructions ! src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java ! src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java ! src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java ! src/java.base/share/classes/valhalla/specializer/Specializer.java From maurizio.cimadamore at oracle.com Tue Jun 30 13:56:40 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 30 Jun 2015 13:56:40 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: add support for bridge method specialization Message-ID: <201506301356.t5UDufnM003643@aojmv0008.oracle.com> Changeset: 77afd487b68e Author: mcimadamore Date: 2015-06-30 14:56 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/77afd487b68e Enhancement: add support for bridge method specialization * generate SpecializerSignature for bridges * added test ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java + test/tools/javac/valhalla/typespec/Bridge01.java