From sebastian.sickelmann at gmx.de Tue Dec 1 13:15:31 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Tue, 1 Dec 2015 14:15:31 +0100 Subject: Compatible Field Resolution In-Reply-To: <33F75FAA-1BE7-470E-9629-4C78B7AF9B77@oracle.com> References: <565AEB85.7020105@gmx.de> <33F75FAA-1BE7-470E-9629-4C78B7AF9B77@oracle.com> Message-ID: <565D9D73.3000206@gmx.de> adding valhalla-dev. Thanks to Brian pointing me in that direction. On 11/29/2015 05:08 PM, Brian Goetz wrote: > I think there may be some synergy between this idea and some of the work going on in project Valhalla. So you should (also) bring those ideas there. > > Also, you might consider jlink as a vehicle for doing such transformations. > > Sent from my iPhone > >> On Nov 29, 2015, at 7:11 AM, Sebastian Sickelmann wrote: >> >> Hi, >> >> some time ago I started a discussion on jdk8-dev [0] about a change in >> field lookup and resolution to make changes to the visibility of fields >> possible without losing binary compatibility. In 2011 unfortunately I >> lost track to take my experiments[1] much further. >> >> Now that i get my feet wet again with some openjdk development and >> learned (hopefully) enough about debugging the jdk with gdb and the jdk >> itself, i started (a few days ago) my experiment again. This time in the >> jvm itself based on the cpp-interpreter (zero) of the jdk9/hs-rt repos. >> Hope to get a of this other type of proof of concept presentable soon. >> >> In the meantime I would love to get some thought about the following >> questions/topics: >> >> Q1: Do you think that java(the jvm) would benefit of some way to make >> changes to the visibility of fields in a binary compatible way? >> Q2: Do you think that this should be handled at runtime/link-time inside >> the vm? >> Q3: Or do you think that this should be handled as early as possible >> (load-time of classes) --> ex. by exchanging all field access to are not >> in the same class(/module??) to an indy-call? >> Q4: Or do you think that a "static prior runtime solution" should be >> created to update "old" jars/modules? >> Q5: If the runtime solution is your choice what to you think, should the >> runtime behavior(lookup and linking of field) of the byte-codes >> get,put,get-static,put-static also be changed for classes that are >> compiled for an older jvm and where the jars/modules are signed by an >> digital certificate? >> Q6: If not Q5. Should it be allowed by some security-related settings? >> Q7: What is about reflection functionality. Should this be changed to? >> Field-Lookups, set / get value of fields? >> >> Hope to get some discussion started. Feel free to answer to one or more >> from the questions / topics above. >> If you have more questions that should be handled, you are also welcome >> to post those. >> Every feedback is welcome, even those you say, all this is a really bad >> idea. >> At least for this last mentioned type of feedback I would prefer to get >> some reasons why you think so. >> >> -- >> Sebastian >> >> [0] http://mail.openjdk.java.net/pipermail/jdk8-dev/2011-October/000199.html >> [1] >> https://github.com/picpromusic/incubator/tree/master/jdk/compatibleFieldAccess From maurizio.cimadamore at oracle.com Wed Dec 2 16:00:22 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Dec 2015 16:00:22 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: Folding TransValues into Attr, TypeEnter Message-ID: <201512021600.tB2G0MSn020791@aojmv0008.oracle.com> Changeset: 5de3c2615ab6 Author: mcimadamore Date: 2015-12-02 16:00 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/5de3c2615ab6 Enhancement: Folding TransValues into Attr, TypeEnter Contributed-by: srikanth.adayapalam at oracle.com ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symtab.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Resolve.java - src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransValues.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TypeEnter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/main/JavaCompiler.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties ! src/jdk.compiler/share/classes/com/sun/tools/javac/util/Names.java ! test/tools/javac/Object1.out ! test/tools/javac/Object2.out + test/tools/javac/diags/examples/NoSuperTypesForJLO.java ! test/tools/javac/valhalla/values/CheckClone.java ! test/tools/javac/valhalla/values/CheckClone.out ! test/tools/javac/valhalla/values/CheckFinalize.java ! test/tools/javac/valhalla/values/CheckFinalize.out ! test/tools/javac/valhalla/values/CheckIdentityHash.java ! test/tools/javac/valhalla/values/CheckIdentityHash.out + test/tools/javac/valhalla/values/CheckIdentityHash01.java + test/tools/javac/valhalla/values/CheckIdentityHash01.out ! test/tools/javac/valhalla/values/CheckSync.java ! test/tools/javac/valhalla/values/CheckSync.out From maurizio.cimadamore at oracle.com Thu Dec 3 12:15:23 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 03 Dec 2015 12:15:23 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: add support for multivnewarray Message-ID: <201512031215.tB3CFNbA008416@aojmv0008.oracle.com> Changeset: 358b7a32d810 Author: mcimadamore Date: 2015-12-03 12:15 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/358b7a32d810 Enhancement: add support for multivnewarray Contributed-by: michael.haupt at oracle.com, srikanth.adayapalam at oracle.com ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ByteCodes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Code.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/Opcode.java From maurizio.cimadamore at oracle.com Thu Dec 3 12:16:11 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 03 Dec 2015 12:16:11 +0000 Subject: hg: valhalla/valhalla/jdk: Sync Opcode.java after latest langtools changes Message-ID: <201512031216.tB3CGCJm009294@aojmv0008.oracle.com> Changeset: 50c089cc3c73 Author: mcimadamore Date: 2015-12-03 12:15 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/50c089cc3c73 Sync Opcode.java after latest langtools changes ! src/java.base/share/classes/valhalla/model3/classfile/Opcode.java From timo.kinnunen at gmail.com Fri Dec 4 13:18:45 2015 From: timo.kinnunen at gmail.com (Timo Kinnunen) Date: Fri, 4 Dec 2015 14:18:45 +0100 Subject: A thought on value-typing existing Collections Message-ID: <566192b6.93301c0a.fe55e.33bf@mx.google.com> Hi, I recently needed a primitive specialized HashMap and looked into the implementations while deciding on one, eventually going with the https://github.com/OpenHFT/Koloboke library. The generated class hierarchy it uses is obnoxious, but the gist of the implementation can be explained with this pseudo-code: public void put(long key, long value) { long free = getOldFree(); if(key == free) { free = chooseNewFree(); setNewFree(free); } int index = generateHashCode(key); putAtFirstFree(index, free, key, value); } This code will be very easy to convert to use value types. This is because it allows all longs as valid for both keys and values while at the same time it doesn?t use a fixed value as the marker in its implementation. In other words, because it permits every value as a valid input, including the values it?s using for internal bookkeeping! Translated into the references and objects world, the easiest Collections to convert will be those that are internally strongly-typed and then -- a bit paradoxically -- allow null as a valid value as this prevents the use of null as an internal marker. Closely followed by those that use object identities in comparisons. Which also happen to be just the kinds of Collections Java programmers have been told for years to avoid. Oops? Some food for thought for the weekend. -- Have a nice day, Timo Sent from Mail for Windows 10 From maurizio.cimadamore at oracle.com Fri Dec 4 19:24:17 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 04 Dec 2015 19:24:17 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: Ensure that comparison of nulls to value instances is disallowed Message-ID: <201512041924.tB4JOHvJ024306@aojmv0008.oracle.com> Changeset: 2568113d414c Author: mcimadamore Date: 2015-12-04 19:24 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/2568113d414c Fix: Ensure that comparison of nulls to value instances is disallowed Contributed-by: srikanth.adayapalam at oracle.com ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java ! test/tools/javac/valhalla/values/CheckNullAssign.java ! test/tools/javac/valhalla/values/CheckNullAssign.out From maurizio.cimadamore at oracle.com Fri Dec 4 19:25:05 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 04 Dec 2015 19:25:05 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: add tests Message-ID: <201512041925.tB4JP5WY024576@aojmv0008.oracle.com> Changeset: 572d8ea3ca59 Author: mcimadamore Date: 2015-12-04 19:24 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/572d8ea3ca59 Fix: add tests + test/tools/javac/valhalla/values/CheckNullCastable.java + test/tools/javac/valhalla/values/CheckNullCastable.out From sebastian.sickelmann at gmx.de Sat Dec 5 18:36:19 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Sat, 5 Dec 2015 19:36:19 +0100 Subject: Compatible Field Resolution (Very first working prototype) In-Reply-To: <565D9D73.3000206@gmx.de> References: <565AEB85.7020105@gmx.de> <33F75FAA-1BE7-470E-9629-4C78B7AF9B77@oracle.com> <565D9D73.3000206@gmx.de> Message-ID: <56632EA3.9040607@gmx.de> Hi, the very first prototype implemented inside the vm (without byte code instrumentation) is working. For those who want to try it: the changes for this are based on the hs-rt[5] repository and the patches can be found here [0] [1]. For making this first step easy for me it uses the following configure parameters: --with-jvm-interpreter=cpp --with-jvm-variants=zero. To try it out yourself compile the following two source files [2][3] (with an arbitrary java-compiler) public class Example1 { public static void main(String[] args) { SubjectToChange stc = new SubjectToChange(5); System.out.println(stc.publicField); System.out.println(SubjectToChange.publicStaticField); } } public class SubjectToChange { public int publicField; public static int publicStaticField = 42; public SubjectToChange(int value) { this.publicField = value; } } After compiling you should try starting the Example1. Then change just the class SubjectToChange to the following[4] implementation. import java.lang.reflect.Accessor; public class SubjectToChange { private int value; private static int staticValue = 43; public SubjectToChange(int value) { this.value = value; } @Accessor("publicStaticField") public static int getStatic() { return staticValue; } @Accessor("publicStaticField") public static void setStatic(int newValue) { staticValue = newValue; } @Accessor("publicField") public int getPublicField() { return value; } @Accessor("publicField") public void setPublicField(int value) { this.value = value; } } And simple compile only the class SubjectToChange (maybe you use another directory for compilation of the changed version) You have to use the newly created jdk with the patches from [0] and [1]. There is no change in the javac, but you need the implementation of the Accessor-Annotation to compile the changed version off SubjectToChange. I use -source 1.6 -target 1.6 for this one, to later test it also with an older jvm. If you now run the Example1 with the patches jdk9-vm it prints out 5 43 If you run the Example1 with another jdk9 / or jdk8 jvm you get the following expected error: java.lang.NoSuchFieldError: publicField sebastian at Inspi:~/example1$ ~/jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/bin/java -cp orig Example1 5 42 sebastian at Inspi:~/example1$ ~/jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/bin/java -cp change:orig Example1 Exception in thread "main" java.lang.NoSuchFieldError: publicField at Example1.main(Example1.java:4) sebastian at Inspi:~/example1$ ~/hs-rt2/build/linux-x86_64-normal-zero-slowdebug/images/jdk/bin/java -cp orig Example1 5 42 sebastian at Inspi:~/example1$ ~/hs-rt2/build/linux-x86_64-normal-zero-slowdebug/images/jdk/bin/java -cp change:orig Example1 5 43 Hope this gives an first idea what could be done with this. In a follow-up post I want to write some more about the implementation. There are also other ways that should be explored to achieve the same result. Like static postprocessing of jars/modules. Brian mentioned that that there maybe some synergies between this idea and Valhalla. I see some common topics with VarHandles, do you see another special topic in project Valhalla that has something in common with this idea? Hope to get some feedback -- Sebastian [0] http://cr.openjdk.java.net/~sebastian/cfa/firstWorkingExample/jdk.00/ [1] http://cr.openjdk.java.net/~sebastian/cfa/firstWorkingExample/hotspot.00/ [2] https://github.com/picpromusic/incubator/blob/master/jdk/compatibleFieldAccess/testsrc/incubator/tests/Example1.java [3] https://github.com/picpromusic/incubator/blob/master/jdk/compatibleFieldAccess/libsrc/example1/SubjectToChange.java [4] https://github.com/picpromusic/incubator/blob/master/jdk/compatibleFieldAccess/libsrc1/example1/SubjectToChange.java [5] http://hg.openjdk.java.net/jdk9/hs-rt/ On 12/01/2015 02:15 PM, Sebastian Sickelmann wrote: > adding valhalla-dev. > > Thanks to Brian pointing me in that direction. > > On 11/29/2015 05:08 PM, Brian Goetz wrote: >> I think there may be some synergy between this idea and some of the work going on in project Valhalla. So you should (also) bring those ideas there. >> >> Also, you might consider jlink as a vehicle for doing such transformations. >> >> Sent from my iPhone >> >>> On Nov 29, 2015, at 7:11 AM, Sebastian Sickelmann wrote: >>> >>> Hi, >>> >>> some time ago I started a discussion on jdk8-dev [0] about a change in >>> field lookup and resolution to make changes to the visibility of fields >>> possible without losing binary compatibility. In 2011 unfortunately I >>> lost track to take my experiments[1] much further. >>> >>> Now that i get my feet wet again with some openjdk development and >>> learned (hopefully) enough about debugging the jdk with gdb and the jdk >>> itself, i started (a few days ago) my experiment again. This time in the >>> jvm itself based on the cpp-interpreter (zero) of the jdk9/hs-rt repos. >>> Hope to get a of this other type of proof of concept presentable soon. >>> >>> In the meantime I would love to get some thought about the following >>> questions/topics: >>> >>> Q1: Do you think that java(the jvm) would benefit of some way to make >>> changes to the visibility of fields in a binary compatible way? >>> Q2: Do you think that this should be handled at runtime/link-time inside >>> the vm? >>> Q3: Or do you think that this should be handled as early as possible >>> (load-time of classes) --> ex. by exchanging all field access to are not >>> in the same class(/module??) to an indy-call? >>> Q4: Or do you think that a "static prior runtime solution" should be >>> created to update "old" jars/modules? >>> Q5: If the runtime solution is your choice what to you think, should the >>> runtime behavior(lookup and linking of field) of the byte-codes >>> get,put,get-static,put-static also be changed for classes that are >>> compiled for an older jvm and where the jars/modules are signed by an >>> digital certificate? >>> Q6: If not Q5. Should it be allowed by some security-related settings? >>> Q7: What is about reflection functionality. Should this be changed to? >>> Field-Lookups, set / get value of fields? >>> >>> Hope to get some discussion started. Feel free to answer to one or more >>> from the questions / topics above. >>> If you have more questions that should be handled, you are also welcome >>> to post those. >>> Every feedback is welcome, even those you say, all this is a really bad >>> idea. >>> At least for this last mentioned type of feedback I would prefer to get >>> some reasons why you think so. >>> >>> -- >>> Sebastian >>> >>> [0] http://mail.openjdk.java.net/pipermail/jdk8-dev/2011-October/000199.html >>> [1] >>> https://github.com/picpromusic/incubator/tree/master/jdk/compatibleFieldAccess > From maurizio.cimadamore at oracle.com Mon Dec 7 09:58:34 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 7 Dec 2015 09:58:34 +0000 Subject: Result: New Committer: Srikanth Adayapalam Message-ID: <5665584A.7030700@oracle.com> |Voting for Srikanth Adayapalam [1] is now closed. Yes: 3 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. Maurizio [1] - http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-November/001551.html| From forax at univ-mlv.fr Mon Dec 7 11:44:26 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 7 Dec 2015 12:44:26 +0100 (CET) Subject: Result: New Committer: Srikanth Adayapalam In-Reply-To: <5665584A.7030700@oracle.com> References: <5665584A.7030700@oracle.com> Message-ID: <315829976.179365.1449488666431.JavaMail.zimbra@u-pem.fr> Welcome Srikanth :) R?mi ----- Mail original ----- > De: "Maurizio Cimadamore" > ?: valhalla-dev at openjdk.java.net > Cc: registrar at openjdk.java.net > Envoy?: Lundi 7 D?cembre 2015 10:58:34 > Objet: Result: New Committer: Srikanth Adayapalam > > |Voting for Srikanth Adayapalam [1] is now closed. Yes: 3 Veto: 0 > Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is > sufficient to approve the nomination. Maurizio [1] - > http://mail.openjdk.java.net/pipermail/valhalla-dev/2015-November/001551.html| > > From maurizio.cimadamore at oracle.com Mon Dec 7 16:51:07 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 07 Dec 2015 16:51:07 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: Compilation order dependencies causes classes to be erased before types are specialized Message-ID: <201512071651.tB7Gp7Cw013804@aojmv0008.oracle.com> Changeset: 6a8a8a94eaef Author: mcimadamore Date: 2015-12-07 16:50 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/6a8a8a94eaef Fix: Compilation order dependencies causes classes to be erased before types are specialized Fix: calling a __RefOnly method from a nested class crashes javac ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java + test/tools/javac/valhalla/typespec/Auxiliary06.java + test/tools/javac/valhalla/typespec/TestRefOnly11.java From brian.goetz at oracle.com Mon Dec 7 18:04:58 2015 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Mon, 07 Dec 2015 18:04:58 +0000 Subject: hg: valhalla/valhalla/jdk: Fix specializer path where ANEWARRAY on specialized reference arrays was not properly handled Message-ID: <201512071804.tB7I4x1G021287@aojmv0008.oracle.com> Changeset: 8b42a880b426 Author: briangoetz Date: 2015-12-07 13:03 -0500 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/8b42a880b426 Fix specializer path where ANEWARRAY on specialized reference arrays was not properly handled ! src/java.base/share/classes/valhalla/specializer/Specializer.java From maurizio.cimadamore at oracle.com Tue Dec 8 14:42:48 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 08 Dec 2015 14:42:48 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: type-specialization backend overhaul Message-ID: <201512081442.tB8EgmNH027268@aojmv0008.oracle.com> Changeset: 2b24dfe17c16 Author: mcimadamore Date: 2015-12-08 14:42 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/2b24dfe17c16 Enhancement: type-specialization backend overhaul * preserve unerased types as much as possible across the compiler pipeline * push specialization logic into the backend * remove support for specialized class symbols * constant pool support rewrite * explicit symbolic representation for method type types and method handles * simplified Items classes (more to come) ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Lower.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassReader.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Code.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/sym/CreateSymbols.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeInfo.java ! src/jdk.jshell/share/classes/jdk/jshell/SourceCodeAnalysisImpl.java ! test/tools/javac/lambda/TestInvokeDynamic.java ! test/tools/javac/scope/7046348/EagerInterfaceCompletionTest.java ! test/tools/javac/valhalla/typespec/ObjectMethods.out ! test/tools/javac/valhalla/typespec/items/tests/TestRespecialization.java ! test/tools/javap/MethodParameters.java From ali.ebrahimi1781 at gmail.com Tue Dec 8 15:57:01 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Tue, 8 Dec 2015 19:27:01 +0330 Subject: hg: valhalla/valhalla/langtools: Enhancement: type-specialization backend overhaul In-Reply-To: <201512081442.tB8EgmNH027268@aojmv0008.oracle.com> References: <201512081442.tB8EgmNH027268@aojmv0008.oracle.com> Message-ID: Hi, build error. no PoolWriter symbol On Tue, Dec 8, 2015 at 6:12 PM, wrote: > Changeset: 2b24dfe17c16 > Author: mcimadamore > Date: 2015-12-08 14:42 +0000 > URL: > http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/2b24dfe17c16 > > Enhancement: type-specialization backend overhaul > * preserve unerased types as much as possible across the compiler pipeline > * push specialization logic into the backend > * remove support for specialized class symbols > * constant pool support rewrite > * explicit symbolic representation for method type types and method handles > * simplified Items classes (more to come) > > ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.java > ! > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Lower.java > ! > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassReader.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Code.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/sym/CreateSymbols.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeInfo.java > ! src/jdk.jshell/share/classes/jdk/jshell/SourceCodeAnalysisImpl.java > ! test/tools/javac/lambda/TestInvokeDynamic.java > ! test/tools/javac/scope/7046348/EagerInterfaceCompletionTest.java > ! test/tools/javac/valhalla/typespec/ObjectMethods.out > ! test/tools/javac/valhalla/typespec/items/tests/TestRespecialization.java > ! test/tools/javap/MethodParameters.java > > -- Best Regards, Ali Ebrahimi From maurizio.cimadamore at oracle.com Tue Dec 8 16:30:23 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 08 Dec 2015 16:30:23 +0000 Subject: hg: valhalla/valhalla/langtools: forgot to hg add PoolWriter Message-ID: <201512081630.tB8GUN5K007905@aojmv0008.oracle.com> Changeset: e7cdaa508aea Author: mcimadamore Date: 2015-12-08 16:29 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/e7cdaa508aea forgot to hg add PoolWriter ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java + src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolWriter.java From maurizio.cimadamore at oracle.com Tue Dec 8 16:30:49 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 8 Dec 2015 16:30:49 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: type-specialization backend overhaul In-Reply-To: References: <201512081442.tB8EgmNH027268@aojmv0008.oracle.com> Message-ID: <566705B9.1040402@oracle.com> Whoops - my bad; fixed now. Sorry. Maurizio On 08/12/15 15:57, Ali Ebrahimi wrote: > Hi, > build error. no PoolWriter symbol > > On Tue, Dec 8, 2015 at 6:12 PM, > wrote: > > Changeset: 2b24dfe17c16 > Author: mcimadamore > Date: 2015-12-08 14:42 +0000 > URL: > http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/2b24dfe17c16 > > Enhancement: type-specialization backend overhaul > * preserve unerased types as much as possible across the compiler > pipeline > * push specialization logic into the backend > * remove support for specialized class symbols > * constant pool support rewrite > * explicit symbolic representation for method type types and > method handles > * simplified Items classes (more to come) > > ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.java > ! > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Lower.java > ! > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java > ! > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java > ! > src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassReader.java > ! > src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Code.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java > ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java > ! > src/jdk.compiler/share/classes/com/sun/tools/javac/sym/CreateSymbols.java > ! > src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeInfo.java > ! src/jdk.jshell/share/classes/jdk/jshell/SourceCodeAnalysisImpl.java > ! test/tools/javac/lambda/TestInvokeDynamic.java > ! test/tools/javac/scope/7046348/EagerInterfaceCompletionTest.java > ! test/tools/javac/valhalla/typespec/ObjectMethods.out > ! > test/tools/javac/valhalla/typespec/items/tests/TestRespecialization.java > ! test/tools/javap/MethodParameters.java > > > > > -- > > Best Regards, > Ali Ebrahimi From maurizio.cimadamore at oracle.com Tue Dec 8 18:42:10 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 08 Dec 2015 18:42:10 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: Remove dynamic instance tests from Code.java Message-ID: <201512081842.tB8IgF5a020711@aojmv0008.oracle.com> Changeset: 6acf2c8bf289 Author: mcimadamore Date: 2015-12-08 18:41 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/6acf2c8bf289 Enhancement: Remove dynamic instance tests from Code.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Code.java From brian.goetz at oracle.com Thu Dec 10 19:49:02 2015 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Thu, 10 Dec 2015 19:49:02 +0000 Subject: hg: valhalla/valhalla/jdk: Further progress towards anyfying ArrayList, HashMap Message-ID: <201512101949.tBAJn2Rt010891@aojmv0008.oracle.com> Changeset: 2688b782367b Author: briangoetz Date: 2015-12-10 14:48 -0500 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/2688b782367b Further progress towards anyfying ArrayList, HashMap ! src/java.base/share/classes/java/anyutil/AbstractCollection.java ! src/java.base/share/classes/java/anyutil/AbstractList.java ! src/java.base/share/classes/java/anyutil/AbstractMap.java ! src/java.base/share/classes/java/anyutil/AbstractSet.java + src/java.base/share/classes/java/anyutil/Any.java ! src/java.base/share/classes/java/anyutil/ArrayList.java ! src/java.base/share/classes/java/anyutil/Collection.java ! src/java.base/share/classes/java/anyutil/HashMap.java ! src/java.base/share/classes/java/anyutil/HashSet.java ! src/java.base/share/classes/java/anyutil/LinkedHashMap.java ! src/java.base/share/classes/java/anyutil/Map.java ! src/java.base/share/classes/java/anyutil/NavigableMap.java ! src/java.base/share/classes/java/anyutil/NavigableSet.java ! src/java.base/share/classes/java/anyutil/SortedMap.java ! src/java.base/share/classes/java/anyutil/SortedSet.java ! src/java.base/share/classes/java/anyutil/stream/Collectors.java + test/valhalla/test/valhalla/anyutil/CollectionsTest.java From pbenedict at apache.org Thu Dec 10 22:33:04 2015 From: pbenedict at apache.org (Paul Benedict) Date: Thu, 10 Dec 2015 16:33:04 -0600 Subject: Thoughts on peeling and readability Message-ID: I truly like the peeling concept. Kudos Brian, Maurizio, et al. My only difficulty is reading the source code that contains the peeling. When I read it, I become concerned that mixing value-based blocks and an object-based blocks diminishes comprehension. In a way, I feel transported into C with preprocessor directives. :-) I wish there was cleaner separation. Three points: 1) In particular, I find reading most harmed when __WhereRef or ___WhereVal is within a function. I don't like these blocks at all. 2) Declared at the method level, reading is easy -- because a method is complete and distinct. This may be, in fact, the happy medium and the most preferred way of coding it. 3) At the other extreme, have you explored allowing developers to write semi-subclasses? Sorry for the poor term, but I don't know what else to call it. Some of the Collection changes are incredibly big, and almost seem like half-and-half code. I am concerned that when the division becomes so enormous yet equal, maintainability will be difficult. Perhaps something like this can be explored: public abstract class AbstractCollection implements Collection { // shared } // in same compilation unit __WhereRef(E) AbstractCollection { public String toString() { ... } } // in same compilation unit __WhereVal(E) AbstractCollection { public String toString() { ... } } Cheers, Paul From palo.marton at gmail.com Fri Dec 11 08:21:23 2015 From: palo.marton at gmail.com (Palo Marton) Date: Fri, 11 Dec 2015 09:21:23 +0100 Subject: Thoughts on peeling and readability In-Reply-To: References: Message-ID: ad 1), 2) - Current syntax is totally provisional and not important in the current phase of valhalla project. There are other more important things that have have to be solved first, syntax is left for later. On Thu, Dec 10, 2015 at 11:33 PM, Paul Benedict wrote: > I truly like the peeling concept. Kudos Brian, Maurizio, et al. > > My only difficulty is reading the source code that contains the peeling. > When I read it, I become concerned that mixing value-based blocks and an > object-based blocks diminishes comprehension. In a way, I feel transported > into C with preprocessor directives. :-) I wish there was cleaner > separation. > > Three points: > > 1) In particular, I find reading most harmed when __WhereRef or ___WhereVal > is within a function. I don't like these blocks at all. > > 2) Declared at the method level, reading is easy -- because a method is > complete and distinct. This may be, in fact, the happy medium and the most > preferred way of coding it. > > 3) At the other extreme, have you explored allowing developers to write > semi-subclasses? Sorry for the poor term, but I don't know what else to > call it. Some of the Collection changes are incredibly big, and almost seem > like half-and-half code. I am concerned that when the division becomes so > enormous yet equal, maintainability will be difficult. Perhaps something > like this can be explored: > > public abstract class AbstractCollection implements Collection { > // shared > } > > // in same compilation unit > __WhereRef(E) AbstractCollection { > public String toString() { ... } > } > > // in same compilation unit > __WhereVal(E) AbstractCollection { > public String toString() { ... } > } > > Cheers, > Paul > From maurizio.cimadamore at oracle.com Fri Dec 11 10:52:19 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 11 Dec 2015 10:52:19 +0000 Subject: Thoughts on peeling and readability In-Reply-To: References: Message-ID: <566AAAE3.7090906@oracle.com> Hi Paul, thanks for the comments; we are aware that the syntax situation is a bit sad at the moment, and of course we wouldn't wish such a syntax to be forced on any user. But, in terms of expressiveness, we believe that the current prototype is powerful enough - i.e. it lets you do most of the things that are needed in order to workaround most of the problems we faced when trying to anyfy the collection library. So, rest assured - the syntax tokens used currently are mere hooks for the compiler to get it to generate what we want. We are aware that the current approach is both too verbose and also very alien to the Java developer and we have already few promising alternatives which could retain same expressive power while being way more friendly to Java developers (unfortunately we cannot share much at this point in time). Your idea of semi-subclasses is not that different from the concept of layering that was discussed few months ago in this list; while arguably cleaner, that too sparked some concerns about perceived complexity; it's definitively an option on the table, but we'd like something simpler. Thanks Maurizio On 10/12/15 22:33, Paul Benedict wrote: > I truly like the peeling concept. Kudos Brian, Maurizio, et al. > > My only difficulty is reading the source code that contains the peeling. > When I read it, I become concerned that mixing value-based blocks and an > object-based blocks diminishes comprehension. In a way, I feel transported > into C with preprocessor directives. :-) I wish there was cleaner > separation. > > Three points: > > 1) In particular, I find reading most harmed when __WhereRef or ___WhereVal > is within a function. I don't like these blocks at all. > > 2) Declared at the method level, reading is easy -- because a method is > complete and distinct. This may be, in fact, the happy medium and the most > preferred way of coding it. > > 3) At the other extreme, have you explored allowing developers to write > semi-subclasses? Sorry for the poor term, but I don't know what else to > call it. Some of the Collection changes are incredibly big, and almost seem > like half-and-half code. I am concerned that when the division becomes so > enormous yet equal, maintainability will be difficult. Perhaps something > like this can be explored: > > public abstract class AbstractCollection implements Collection { > // shared > } > > // in same compilation unit > __WhereRef(E) AbstractCollection { > public String toString() { ... } > } > > // in same compilation unit > __WhereVal(E) AbstractCollection { > public String toString() { ... } > } > > Cheers, > Paul From brian.goetz at oracle.com Fri Dec 11 13:57:22 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 11 Dec 2015 08:57:22 -0500 Subject: Thoughts on peeling and readability In-Reply-To: References: Message-ID: <566AD642.4050208@oracle.com> First, remember that the current syntax is *deliberately* awful. The sole purpose of the current round is to determine if: there a model that (a) can handle the migration needs of the current libraries, (b) can be set firmly on a theoretical foundation, and (c) is not too foreign to the existing mindset of Java developers. We've already gone through several rounds of refining the model and will go through several more. Once we're satisfied with the above, then it becomes sensible to start to think about how the code looks! The good news is that we might actually be getting there (unlike in the past few attempts.) > Some of the Collection changes are incredibly big They look that way now, but in the end, they won't be. Most of the changes look "big" for purely accidental reasons. About 75% of the lines of diff are because the compiler doesn't yet support inner classes, and so these have to be manually desugared, for example. And many of the implementations that are currently peeled as entire methods will instead be factored down into smaller peeled methods or into direct language support for type-dependent operation (e.g., null checks) so that no peeling is required. So, don't freak out! On 12/10/2015 5:33 PM, Paul Benedict wrote: > I truly like the peeling concept. Kudos Brian, Maurizio, et al. > > My only difficulty is reading the source code that contains the peeling. > When I read it, I become concerned that mixing value-based blocks and an > object-based blocks diminishes comprehension. In a way, I feel transported > into C with preprocessor directives. :-) I wish there was cleaner > separation. > > Three points: > > 1) In particular, I find reading most harmed when __WhereRef or ___WhereVal > is within a function. I don't like these blocks at all. > > 2) Declared at the method level, reading is easy -- because a method is > complete and distinct. This may be, in fact, the happy medium and the most > preferred way of coding it. > > 3) At the other extreme, have you explored allowing developers to write > semi-subclasses? Sorry for the poor term, but I don't know what else to > call it. Some of the Collection changes are incredibly big, and almost seem > like half-and-half code. I am concerned that when the division becomes so > enormous yet equal, maintainability will be difficult. Perhaps something > like this can be explored: > > public abstract class AbstractCollection implements Collection { > // shared > } > > // in same compilation unit > __WhereRef(E) AbstractCollection { > public String toString() { ... } > } > > // in same compilation unit > __WhereVal(E) AbstractCollection { > public String toString() { ... } > } > > Cheers, > Paul From maurizio.cimadamore at oracle.com Fri Dec 11 14:21:21 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Dec 2015 14:21:21 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: refactor ClassReader to use Pool, PoolReader Message-ID: <201512111421.tBBELMsh018531@aojmv0008.oracle.com> Changeset: 0c84a5536918 Author: mcimadamore Date: 2015-12-11 14:20 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/0c84a5536918 Enhancement: refactor ClassReader to use Pool, PoolReader ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassReader.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Code.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java + src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolReader.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties ! src/jdk.compiler/share/classes/com/sun/tools/javac/util/ByteBuffer.java ! test/tools/javac/diags/examples.not-yet.txt From paul.sandoz at oracle.com Fri Dec 11 15:00:21 2015 From: paul.sandoz at oracle.com (paul.sandoz at oracle.com) Date: Fri, 11 Dec 2015 15:00:21 +0000 Subject: hg: valhalla/valhalla/jdk: Tweak List/AbstractList/ArrayList for remove(int ) -> removeAt(int ) Message-ID: <201512111500.tBBF0Lep003461@aojmv0008.oracle.com> Changeset: eef734f30f61 Author: psandoz Date: 2015-12-11 15:53 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/eef734f30f61 Tweak List/AbstractList/ArrayList for remove(int ) -> removeAt(int ) ! src/java.base/share/classes/java/anyutil/AbstractList.java ! src/java.base/share/classes/java/anyutil/ArrayList.java ! src/java.base/share/classes/java/anyutil/List.java ! test/valhalla/test/valhalla/anyutil/CollectionsTest.java From paul.sandoz at oracle.com Fri Dec 11 15:25:24 2015 From: paul.sandoz at oracle.com (paul.sandoz at oracle.com) Date: Fri, 11 Dec 2015 15:25:24 +0000 Subject: hg: valhalla/valhalla/jdk: AbstractSequentialList.java/LinkedList remove(int ) -> removeAt(int ) Message-ID: <201512111525.tBBFPPXv013129@aojmv0008.oracle.com> Changeset: 1dcd5ede7692 Author: psandoz Date: 2015-12-11 16:24 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/1dcd5ede7692 AbstractSequentialList.java/LinkedList remove(int ) -> removeAt(int ) AbstractList remove(int) can defer to removeAt(int ), kind of like an explicit bridge, where as it's the reverse with the default method on List. ! src/java.base/share/classes/java/anyutil/AbstractList.java ! src/java.base/share/classes/java/anyutil/AbstractSequentialList.java ! src/java.base/share/classes/java/anyutil/LinkedList.java ! test/valhalla/test/valhalla/anyutil/CollectionsTest.java From vitalyd at gmail.com Fri Dec 11 15:56:32 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 11 Dec 2015 10:56:32 -0500 Subject: Thoughts on peeling and readability In-Reply-To: <566AD642.4050208@oracle.com> References: <566AD642.4050208@oracle.com> Message-ID: Brian, Like Paul, I also like the peeling/specialization approach. I just wanted to quickly ask if there's still a plan to allow specialization of storage based on concrete type - think ArrayList being a bit vector internally, as an example. Any current thoughts/plans? sent from my phone On Dec 11, 2015 8:57 AM, "Brian Goetz" wrote: > First, remember that the current syntax is *deliberately* awful. The sole > purpose of the current round is to determine if: there a model that (a) can > handle the migration needs of the current libraries, (b) can be set firmly > on a theoretical foundation, and (c) is not too foreign to the existing > mindset of Java developers. > > We've already gone through several rounds of refining the model and will > go through several more. Once we're satisfied with the above, then it > becomes sensible to start to think about how the code looks! The good news > is that we might actually be getting there (unlike in the past few > attempts.) > > Some of the Collection changes are incredibly big >> > > They look that way now, but in the end, they won't be. Most of the > changes look "big" for purely accidental reasons. About 75% of the lines > of diff are because the compiler doesn't yet support inner classes, and so > these have to be manually desugared, for example. And many of the > implementations that are currently peeled as entire methods will instead be > factored down into smaller peeled methods or into direct language support > for type-dependent operation (e.g., null checks) so that no peeling is > required. > > So, don't freak out! > > > On 12/10/2015 5:33 PM, Paul Benedict wrote: > >> I truly like the peeling concept. Kudos Brian, Maurizio, et al. >> >> My only difficulty is reading the source code that contains the peeling. >> When I read it, I become concerned that mixing value-based blocks and an >> object-based blocks diminishes comprehension. In a way, I feel transported >> into C with preprocessor directives. :-) I wish there was cleaner >> separation. >> >> Three points: >> >> 1) In particular, I find reading most harmed when __WhereRef or >> ___WhereVal >> is within a function. I don't like these blocks at all. >> >> 2) Declared at the method level, reading is easy -- because a method is >> complete and distinct. This may be, in fact, the happy medium and the most >> preferred way of coding it. >> >> 3) At the other extreme, have you explored allowing developers to write >> semi-subclasses? Sorry for the poor term, but I don't know what else to >> call it. Some of the Collection changes are incredibly big, and almost >> seem >> like half-and-half code. I am concerned that when the division becomes so >> enormous yet equal, maintainability will be difficult. Perhaps something >> like this can be explored: >> >> public abstract class AbstractCollection implements Collection { >> // shared >> } >> >> // in same compilation unit >> __WhereRef(E) AbstractCollection { >> public String toString() { ... } >> } >> >> // in same compilation unit >> __WhereVal(E) AbstractCollection { >> public String toString() { ... } >> } >> >> Cheers, >> Paul >> > > From brian.goetz at oracle.com Fri Dec 11 16:28:04 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 11 Dec 2015 11:28:04 -0500 Subject: Thoughts on peeling and readability In-Reply-To: References: <566AD642.4050208@oracle.com> Message-ID: <566AF994.7070209@oracle.com> Yes, there are thoughts and plans :) But we're focusing first on the "how do I write a fully generic ArrayList" first, and then we'll turn to how this could be further hand-optimized for specific instantiations. (It's worth noting, though, there are actually not that many concrete examples. ArrayList is an obvious one, as is HashMap (the literature is full of optimized Map implementations for this case.) And things like Optional, which can have a more compact representation than Optional. Beyond this, the "obvious" examples tail off pretty quickly.) On 12/11/2015 10:56 AM, Vitaly Davidovich wrote: > > Brian, > > Like Paul, I also like the peeling/specialization approach. I just > wanted to quickly ask if there's still a plan to allow specialization > of storage based on concrete type - think ArrayList being a > bit vector internally, as an example. Any current thoughts/plans? > > sent from my phone > > On Dec 11, 2015 8:57 AM, "Brian Goetz" > wrote: > > First, remember that the current syntax is *deliberately* awful. > The sole purpose of the current round is to determine if: there a > model that (a) can handle the migration needs of the current > libraries, (b) can be set firmly on a theoretical foundation, and > (c) is not too foreign to the existing mindset of Java developers. > > We've already gone through several rounds of refining the model > and will go through several more. Once we're satisfied with the > above, then it becomes sensible to start to think about how the > code looks! The good news is that we might actually be getting > there (unlike in the past few attempts.) > > Some of the Collection changes are incredibly big > > > They look that way now, but in the end, they won't be. Most of > the changes look "big" for purely accidental reasons. About 75% of > the lines of diff are because the compiler doesn't yet support > inner classes, and so these have to be manually desugared, for > example. And many of the implementations that are currently peeled > as entire methods will instead be factored down into smaller > peeled methods or into direct language support for type-dependent > operation (e.g., null checks) so that no peeling is required. > > So, don't freak out! > > > On 12/10/2015 5:33 PM, Paul Benedict wrote: > > I truly like the peeling concept. Kudos Brian, Maurizio, et al. > > My only difficulty is reading the source code that contains > the peeling. > When I read it, I become concerned that mixing value-based > blocks and an > object-based blocks diminishes comprehension. In a way, I feel > transported > into C with preprocessor directives. :-) I wish there was cleaner > separation. > > Three points: > > 1) In particular, I find reading most harmed when __WhereRef > or ___WhereVal > is within a function. I don't like these blocks at all. > > 2) Declared at the method level, reading is easy -- because a > method is > complete and distinct. This may be, in fact, the happy medium > and the most > preferred way of coding it. > > 3) At the other extreme, have you explored allowing developers > to write > semi-subclasses? Sorry for the poor term, but I don't know > what else to > call it. Some of the Collection changes are incredibly big, > and almost seem > like half-and-half code. I am concerned that when the division > becomes so > enormous yet equal, maintainability will be difficult. Perhaps > something > like this can be explored: > > public abstract class AbstractCollection implements > Collection { > // shared > } > > // in same compilation unit > __WhereRef(E) AbstractCollection { > public String toString() { ... } > } > > // in same compilation unit > __WhereVal(E) AbstractCollection { > public String toString() { ... } > } > > Cheers, > Paul > > From vitalyd at gmail.com Fri Dec 11 16:36:15 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 11 Dec 2015 11:36:15 -0500 Subject: Thoughts on peeling and readability In-Reply-To: <566AF994.7070209@oracle.com> References: <566AD642.4050208@oracle.com> <566AF994.7070209@oracle.com> Message-ID: Awesome! Fully understand there are more fundamental/basic issues to sort out, as you said, but wanted to see what you guys are thinking. I have some internal cases where I could make use of this quite a bit; certain types have characteristics that enable dense/creative ways of storing them. sent from my phone On Dec 11, 2015 11:28 AM, "Brian Goetz" wrote: > Yes, there are thoughts and plans :) But we're focusing first on the "how > do I write a fully generic ArrayList" first, and then we'll turn to how > this could be further hand-optimized for specific instantiations. > > (It's worth noting, though, there are actually not that many concrete > examples. ArrayList is an obvious one, as is HashMap > (the literature is full of optimized Map implementations for this case.) > And things like Optional, which can have a more compact representation > than Optional. Beyond this, the "obvious" examples tail off pretty > quickly.) > > On 12/11/2015 10:56 AM, Vitaly Davidovich wrote: > > Brian, > > Like Paul, I also like the peeling/specialization approach. I just wanted > to quickly ask if there's still a plan to allow specialization of storage > based on concrete type - think ArrayList being a bit vector > internally, as an example. Any current thoughts/plans? > > sent from my phone > On Dec 11, 2015 8:57 AM, "Brian Goetz" wrote: > >> First, remember that the current syntax is *deliberately* awful. The sole >> purpose of the current round is to determine if: there a model that (a) can >> handle the migration needs of the current libraries, (b) can be set firmly >> on a theoretical foundation, and (c) is not too foreign to the existing >> mindset of Java developers. >> >> We've already gone through several rounds of refining the model and will >> go through several more. Once we're satisfied with the above, then it >> becomes sensible to start to think about how the code looks! The good news >> is that we might actually be getting there (unlike in the past few >> attempts.) >> >> Some of the Collection changes are incredibly big >>> >> >> They look that way now, but in the end, they won't be. Most of the >> changes look "big" for purely accidental reasons. About 75% of the lines >> of diff are because the compiler doesn't yet support inner classes, and so >> these have to be manually desugared, for example. And many of the >> implementations that are currently peeled as entire methods will instead be >> factored down into smaller peeled methods or into direct language support >> for type-dependent operation (e.g., null checks) so that no peeling is >> required. >> >> So, don't freak out! >> >> >> On 12/10/2015 5:33 PM, Paul Benedict wrote: >> >>> I truly like the peeling concept. Kudos Brian, Maurizio, et al. >>> >>> My only difficulty is reading the source code that contains the peeling. >>> When I read it, I become concerned that mixing value-based blocks and an >>> object-based blocks diminishes comprehension. In a way, I feel >>> transported >>> into C with preprocessor directives. :-) I wish there was cleaner >>> separation. >>> >>> Three points: >>> >>> 1) In particular, I find reading most harmed when __WhereRef or >>> ___WhereVal >>> is within a function. I don't like these blocks at all. >>> >>> 2) Declared at the method level, reading is easy -- because a method is >>> complete and distinct. This may be, in fact, the happy medium and the >>> most >>> preferred way of coding it. >>> >>> 3) At the other extreme, have you explored allowing developers to write >>> semi-subclasses? Sorry for the poor term, but I don't know what else to >>> call it. Some of the Collection changes are incredibly big, and almost >>> seem >>> like half-and-half code. I am concerned that when the division becomes so >>> enormous yet equal, maintainability will be difficult. Perhaps something >>> like this can be explored: >>> >>> public abstract class AbstractCollection implements Collection >>> { >>> // shared >>> } >>> >>> // in same compilation unit >>> __WhereRef(E) AbstractCollection { >>> public String toString() { ... } >>> } >>> >>> // in same compilation unit >>> __WhereVal(E) AbstractCollection { >>> public String toString() { ... } >>> } >>> >>> Cheers, >>> Paul >>> >> >> > From maurizio.cimadamore at oracle.com Fri Dec 11 18:51:32 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Dec 2015 18:51:32 +0000 Subject: hg: valhalla/valhalla/jdk: Fix: Specializer.advanceType() doesn't erase unspecialized vars Message-ID: <201512111851.tBBIpWHE012884@aojmv0008.oracle.com> Changeset: f3e0ef836124 Author: mcimadamore Date: 2015-12-11 18:51 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/f3e0ef836124 Fix: Specializer.advanceType() doesn't erase unspecialized vars ! src/java.base/share/classes/valhalla/specializer/Specializer.java From maurizio.cimadamore at oracle.com Fri Dec 11 18:53:31 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Dec 2015 18:53:31 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: generic virtualized methods should retain their type-variables Message-ID: <201512111853.tBBIrVvA013856@aojmv0008.oracle.com> Changeset: b6b09873c7b2 Author: mcimadamore Date: 2015-12-11 18:53 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/b6b09873c7b2 Fix: generic virtualized methods should retain their type-variables Fix: unspecialized calls to generic methods should not leak type-variables in the BMA ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java + test/tools/javac/valhalla/typespec/items/tests/TestNonSpecializedMethod.java ! test/tools/javac/valhalla/typespec/items/tests/TestRespecialization.java From twhitmore.nz at gmail.com Sat Dec 12 00:22:52 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Sat, 12 Dec 2015 13:22:52 +1300 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: Hi Vitaly, > I just wanted to quickly ask if there's still a plan to allow specialization of storage > based on concrete type - think ArrayList being a bit vector > internally, as an example. Any current thoughts/plans? The clear conclusion I came to, was that specialization of storage belonged in a separate class. And that it would be a design error to try and misuse genericization/ value-type handling for that different purpose. If you want a BooleanBitList, create a separate class for it. Fundamental differences in how storage is addressed == fundamental structural change == different class. Clear & simple design principle, no? Otherwise we end up with a C-style preprocessor, the ability to conditionally code anything, and the notion of 'class' becomes meaningless. Regards, Thomas From vitalyd at gmail.com Sat Dec 12 00:46:53 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 11 Dec 2015 19:46:53 -0500 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: sent from my phone On Dec 11, 2015 7:23 PM, "Thomas W" wrote: > > Hi Vitaly, > > > I just wanted to quickly ask if there's still a plan to allow > specialization of storage > > based on concrete type - think ArrayList being a bit vector > > internally, as an example. Any current thoughts/plans? > > The clear conclusion I came to, was that specialization of storage belonged > in a separate class. And that it would be a design error to try and misuse > genericization/ value-type handling for that different purpose. > > If you want a BooleanBitList, create a separate class for it. But why? I have a generic class ArrayList that satisfies contract of whomever wants to use it - why define another abstraction just to support better/more efficient storage for a given *type*? There's no capability/semantic change here, it's still an ArrayList. The world today requires that but it's unnecessary; storage is an implementation detail that shouldn't necessarily "leak" into the type system. > Fundamental differences in how storage is addressed == fundamental > structural change == different class. > I don't consider this a fundamental change in this example. LinkedList is a fundamental change from ArrayList because it uses dynamic storage whereas ArrayList is fixed size (until expansion) and has a few other operational differences. To me, that is fundamental. This will be subjective I suspect. > Clear & simple design principle, no? Otherwise we end up with a C-style > preprocessor, the ability to conditionally code anything, and the notion of > 'class' becomes meaningless. Generic types, by definition, are incomplete until you instantiate them with a concrete type. They will be described by whatever contract they document, but using the most appropriate storage model when actually knowing the type seems like a good thing, no? Defining a new class altogether seems wasteful and unnecessary for such cases. > > Regards, > Thomas From vitalyd at gmail.com Sat Dec 12 00:51:41 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 11 Dec 2015 19:51:41 -0500 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: And of course, if you create a new type someone has to know about it to use it. If you specialize a generic type, they only need to know about the generic type. There's no need to create factories for every single specialization, whose set is dynamic over time. Caller code doesn't change here. sent from my phone On Dec 11, 2015 7:46 PM, "Vitaly Davidovich" wrote: > sent from my phone > On Dec 11, 2015 7:23 PM, "Thomas W" wrote: > > > > Hi Vitaly, > > > > > I just wanted to quickly ask if there's still a plan to allow > > specialization of storage > > > based on concrete type - think ArrayList being a bit vector > > > internally, as an example. Any current thoughts/plans? > > > > The clear conclusion I came to, was that specialization of storage > belonged > > in a separate class. And that it would be a design error to try and > misuse > > genericization/ value-type handling for that different purpose. > > > > If you want a BooleanBitList, create a separate class for it. > > But why? I have a generic class ArrayList that satisfies contract of > whomever wants to use it - why define another abstraction just to support > better/more efficient storage for a given *type*? There's no > capability/semantic change here, it's still an ArrayList. > > The world today requires that but it's unnecessary; storage is an > implementation detail that shouldn't necessarily "leak" into the type > system. > > > Fundamental differences in how storage is addressed == fundamental > > structural change == different class. > > > > I don't consider this a fundamental change in this example. LinkedList is > a fundamental change from ArrayList because it uses dynamic storage whereas > ArrayList is fixed size (until expansion) and has a few other operational > differences. To me, that is fundamental. This will be subjective I > suspect. > > > Clear & simple design principle, no? Otherwise we end up with a C-style > > preprocessor, the ability to conditionally code anything, and the notion > of > > 'class' becomes meaningless. > > Generic types, by definition, are incomplete until you instantiate them > with a concrete type. They will be described by whatever contract they > document, but using the most appropriate storage model when actually > knowing the type seems like a good thing, no? Defining a new class > altogether seems wasteful and unnecessary for such cases. > > > > > Regards, > > Thomas > From richard.warburton at gmail.com Sat Dec 12 09:29:50 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Sat, 12 Dec 2015 10:29:50 +0100 Subject: Thoughts on peeling and readability In-Reply-To: <566AF994.7070209@oracle.com> References: <566AD642.4050208@oracle.com> <566AF994.7070209@oracle.com> Message-ID: Hi, Yes, there are thoughts and plans :) But we're focusing first on the "how > do I write a fully generic ArrayList" first, and then we'll turn to how > this could be further hand-optimized for specific instantiations. > > (It's worth noting, though, there are actually not that many concrete > examples. ArrayList is an obvious one, as is HashMap > (the literature is full of optimized Map implementations for this case.) > And things like Optional, which can have a more compact representation > than Optional. Beyond this, the "obvious" examples tail off pretty > quickly. > Are you able to comment on whether you plan to offer more flexible specialization plans than just specific instantiations? For example is providing a specialization for the generic type HashMap where K = V on the cards? regards, Richard Warburton http://insightfullogic.com @RichardWarburto From timo.kinnunen at gmail.com Sat Dec 12 09:43:31 2015 From: timo.kinnunen at gmail.com (Timo Kinnunen) Date: Sat, 12 Dec 2015 10:43:31 +0100 Subject: Thoughts on peeling and readability In-Reply-To: <566AAAE3.7090906@oracle.com> References: <566AAAE3.7090906@oracle.com> Message-ID: <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> Hi, One thing that I don?t remember seeing is any syntax for constructing arbitrary values in generic code without having to know about the precise field layouts and what the meaning of such fields is. Something like T.default but for values other than 0. Perhaps T.default(12345) or some such? Or maybe this is slated to go with bytecode type specialization? What sort of syntax is envisioned to be driving that anyways? -- Have a nice day, Timo Sent from Mail for Windows 10 From: Maurizio Cimadamore Sent: Friday, December 11, 2015 11:52 To: Paul Benedict;valhalla-dev at openjdk.java.net Subject: Re: Thoughts on peeling and readability Hi Paul, thanks for the comments; we are aware that the syntax situation is a bit sad at the moment, and of course we wouldn't wish such a syntax to be forced on any user. But, in terms of expressiveness, we believe that the current prototype is powerful enough - i.e. it lets you do most of the things that are needed in order to workaround most of the problems we faced when trying to anyfy the collection library. So, rest assured - the syntax tokens used currently are mere hooks for the compiler to get it to generate what we want. We are aware that the current approach is both too verbose and also very alien to the Java developer and we have already few promising alternatives which could retain same expressive power while being way more friendly to Java developers (unfortunately we cannot share much at this point in time). Your idea of semi-subclasses is not that different from the concept of layering that was discussed few months ago in this list; while arguably cleaner, that too sparked some concerns about perceived complexity; it's definitively an option on the table, but we'd like something simpler. Thanks Maurizio On 10/12/15 22:33, Paul Benedict wrote: > I truly like the peeling concept. Kudos Brian, Maurizio, et al. > > My only difficulty is reading the source code that contains the peeling. > When I read it, I become concerned that mixing value-based blocks and an > object-based blocks diminishes comprehension. In a way, I feel transported > into C with preprocessor directives. :-) I wish there was cleaner > separation. > > Three points: > > 1) In particular, I find reading most harmed when __WhereRef or ___WhereVal > is within a function. I don't like these blocks at all. > > 2) Declared at the method level, reading is easy -- because a method is > complete and distinct. This may be, in fact, the happy medium and the most > preferred way of coding it. > > 3) At the other extreme, have you explored allowing developers to write > semi-subclasses? Sorry for the poor term, but I don't know what else to > call it. Some of the Collection changes are incredibly big, and almost seem > like half-and-half code. I am concerned that when the division becomes so > enormous yet equal, maintainability will be difficult. Perhaps something > like this can be explored: > > public abstract class AbstractCollection implements Collection { > // shared > } > > // in same compilation unit > __WhereRef(E) AbstractCollection { > public String toString() { ... } > } > > // in same compilation unit > __WhereVal(E) AbstractCollection { > public String toString() { ... } > } > > Cheers, > Paul From brian.goetz at oracle.com Sat Dec 12 17:14:54 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 12 Dec 2015 12:14:54 -0500 Subject: Thoughts on peeling and readability In-Reply-To: References: <566AD642.4050208@oracle.com> <566AF994.7070209@oracle.com> Message-ID: <566C560E.1090602@oracle.com> Without being drawn into a "draw the line" exercise ... that's almost certainly on the other side of the line. "Just specific instantiations" is a pretty compelling place to start -- and probably end -- because they map directly to the underlying VM type system. HashMap maps to a distinct runtime type; HashMap also does; HashMap> does not. (Also, if you allow separate specializations of HashMap and HashMap, which do you use for HashMap? Now you need runtime "most specific" analysis. Yech.) Once you cross the line of "specific instantiations", now you're asking the runtime to redo a lot of computation that we'd like to leave in the static compiler. This seems to be entering the "bad return on complexity" portion of the curve. On 12/12/2015 4:29 AM, Richard Warburton wrote: > Hi, > > Yes, there are thoughts and plans :) But we're focusing first on > the "how do I write a fully generic ArrayList" first, and then > we'll turn to how this could be further hand-optimized for > specific instantiations. > > (It's worth noting, though, there are actually not that many > concrete examples. ArrayList is an obvious one, as is > HashMap (the literature is full of optimized Map > implementations for this case.) And things like Optional, > which can have a more compact representation than Optional. > Beyond this, the "obvious" examples tail off pretty quickly. > > > Are you able to comment on whether you plan to offer more flexible > specialization plans than just specific instantiations? For example is > providing a specialization for the generic type HashMap where K > = V on the cards? > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto From brian.goetz at oracle.com Sat Dec 12 17:20:45 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 12 Dec 2015 12:20:45 -0500 Subject: Thoughts on peeling and readability In-Reply-To: <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> Message-ID: <566C576D.2020806@oracle.com> Precise layout and bit control of values are anti-goals of Valhalla, so we're not really exploring this direction at this time. The problem with approaches like the one you suggest is they fall apart as soon as you leave the realm of "primitives modeled as values." What about values that have refs in them? What about values whose representations are private? Their implementation is supposed to be in sole control of their representation. This runs contrary to the "codes like a class" dictum. On 12/12/2015 4:43 AM, Timo Kinnunen wrote: > Hi, > > One thing that I don?t remember seeing is any syntax for constructing arbitrary values in generic code without having to know about the precise field layouts and what the meaning of such fields is. Something like T.default but for values other than 0. Perhaps T.default(12345) or some such? > > Or maybe this is slated to go with bytecode type specialization? What sort of syntax is envisioned to be driving that anyways? > > > > > From timo.kinnunen at gmail.com Sat Dec 12 22:44:17 2015 From: timo.kinnunen at gmail.com (Timo Kinnunen) Date: Sat, 12 Dec 2015 23:44:17 +0100 Subject: Thoughts on peeling and readability In-Reply-To: <566C576D.2020806@oracle.com> References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> <566C576D.2020806@oracle.com> Message-ID: <566ca345.55b21c0a.b1d90.4516@mx.google.com> Field layout and bit fiddling isn?t exactly what I was thinking. Rather I was thinking something like Float.floatToRawIntBits() and Double.doubleToRawLongBits(), but without having to know about the types Float and Double or how many bits are in their raw bits. So something like this syntax: static T nextUp (T value) { rawBits = T.toRawBits(value); T nextValue = T.fromRawBits(rawBits + 1); return nextValue; } This should fit in Valhalla reasonably well, as it is just a generalization of T.default with its complement operation included. And as it is, all of the problems you listed already apply to T.default. For example, a value type with one long field: If the long value in the field is a handle pointing to a memory-mapped buffer then any use of a default value of such a type could cause a crash. Which can include asking a properly constructed value if it is equal to any of the values in an array you have. -- Have a nice day, Timo Sent from Mail for Windows 10 From: Brian Goetz Sent: Saturday, December 12, 2015 18:21 To: Timo Kinnunen;Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net Subject: Re: Thoughts on peeling and readability Precise layout and bit control of values are anti-goals of Valhalla, so we're not really exploring this direction at this time. The problem with approaches like the one you suggest is they fall apart as soon as you leave the realm of "primitives modeled as values." What about values that have refs in them? What about values whose representations are private? Their implementation is supposed to be in sole control of their representation. This runs contrary to the "codes like a class" dictum. On 12/12/2015 4:43 AM, Timo Kinnunen wrote: > Hi, > > One thing that I don?t remember seeing is any syntax for constructing arbitrary values in generic code without having to know about the precise field layouts and what the meaning of such fields is. Something like T.default but for values other than 0. Perhaps T.default(12345) or some such? > > Or maybe this is slated to go with bytecode type specialization? What sort of syntax is envisioned to be driving that anyways? > > > > > From brian.goetz at oracle.com Sun Dec 13 00:20:10 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 12 Dec 2015 19:20:10 -0500 Subject: Thoughts on peeling and readability In-Reply-To: <566ca345.55b21c0a.b1d90.4516@mx.google.com> References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> <566C576D.2020806@oracle.com> <566ca345.55b21c0a.b1d90.4516@mx.google.com> Message-ID: <39F28927-7C94-47B6-B8D4-EB69448E9917@oracle.com> No. In general, to/from raw bit operations are not safe except in a few corner cases (like int and long.) Values are not uncontrolled buckets of bits. On the other hand, T.default *is* safe, because every type has a default bit pattern which is the initialization of any otherwise uninitialized field or array element. (It so happens that this default bit pattern corresponds to all zero bits for all types, though this is mostly a convenience for VM implementors.) For a composite value, the default value is comprised of the default value for all fields. By *definition*, the all-zero bit pattern is a valid element of all value types. However, there is no guarantee that any other bit pattern is valid for any given value type. If a particular value type wants to expose to/from raw bit constructors, that?s fine ? but you?re asking for a language feature that applies to *all* values ? and there is no guarantee that this is a safe operation for all values. On Dec 12, 2015, at 5:44 PM, Timo Kinnunen wrote: > Field layout and bit fiddling isn?t exactly what I was thinking. Rather I was thinking something like Float.floatToRawIntBits() and Double.doubleToRawLongBits(), but without having to know about the types Float and Double or how many bits are in their raw bits. So something like this syntax: > > static T nextUp (T value) { > rawBits = T.toRawBits(value); > T nextValue = T.fromRawBits(rawBits + 1); > return nextValue; > } > > This should fit in Valhalla reasonably well, as it is just a generalization of T.default with its complement operation included. And as it is, all of the problems you listed already apply to T.default. For example, a value type with one long field: If the long value in the field is a handle pointing to a memory-mapped buffer then any use of a default value of such a type could cause a crash. Which can include asking a properly constructed value if it is equal to any of the values in an array you have. > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > > From: Brian Goetz > Sent: Saturday, December 12, 2015 18:21 > To: Timo Kinnunen;Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net > Subject: Re: Thoughts on peeling and readability > > > Precise layout and bit control of values are anti-goals of Valhalla, so > we're not really exploring this direction at this time. > > The problem with approaches like the one you suggest is they fall apart > as soon as you leave the realm of "primitives modeled as values." What > about values that have refs in them? What about values whose > representations are private? Their implementation is supposed to be in > sole control of their representation. This runs contrary to the "codes > like a class" dictum. > > > > > > > On 12/12/2015 4:43 AM, Timo Kinnunen wrote: > > Hi, > > > > One thing that I don?t remember seeing is any syntax for constructing arbitrary values in generic code without having to know about the precise field layouts and what the meaning of such fields is. Something like T.default but for values other than 0. Perhaps T.default(12345) or some such? > > > > Or maybe this is slated to go with bytecode type specialization? What sort of syntax is envisioned to be driving that anyways? > > > > > > > > > > From timo.kinnunen at gmail.com Sun Dec 13 18:32:52 2015 From: timo.kinnunen at gmail.com (Timo Kinnunen) Date: Sun, 13 Dec 2015 19:32:52 +0100 Subject: Thoughts on peeling and readability In-Reply-To: <39F28927-7C94-47B6-B8D4-EB69448E9917@oracle.com> References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> <566C576D.2020806@oracle.com> <566ca345.55b21c0a.b1d90.4516@mx.google.com> <39F28927-7C94-47B6-B8D4-EB69448E9917@oracle.com> Message-ID: <566db9d9.296dc20a.acd5f.fffff668@mx.google.com> Well, it?s ints and longs, and all primitive types copyable around as ints and longs, and all objects serializable to and from arrays of ints and longs, and all arrays of such, and all values made of such, and all arrays of such values, and all values made of such values, aaand I?m probably missing a dimension or two somewhere. These values are just as valid regardless of which bit patterns, all-zero or not, were used to construct them. They are safe to be copied around and, if you implemented them yourself, hashCode, equals, toString and any component-wise operations could also be done safely. Such operations simply can?t call any foreign code of any of the value or reference types involved. We don?t expect that we can take an arbitrary Object, use reflection to zero out its fields and then be able to call its instance methods like nothing had happened either. So, for reference types these operations would have to done using reflection, for value types VarHandles might give better performance. Ultimately I guess it all depends on whether T.default invokes some constructor or not. If it doesn?t or if the constructor is specified to always succeed trivially then we have reinvented null. Our new nulls trade off a large number of NPEs for silent invariant violations. This could be a good tradeoff but without knowing about the consequences of the violations beforehand it?s gonna be hard to say for certain. The good news is that hey, null is back! If T.default executes a constructor that can refuse an all-zero bit pattern, then we have reinvented serialization for value types and are requiring all value types support it. Our serialization protocol only recognizes one input value and can only deserialize, so it?s a bit useless. But with the addition of the missing serialize-function we can then define transforms for long[] <-> val <-> long[] and will have a quite general and capable system already. Or are we gonna just include the worst parts from both? -- Have a nice day, Timo Sent from Mail for Windows 10 From: Brian Goetz Sent: Sunday, December 13, 2015 01:20 To: Timo Kinnunen Cc: Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net Subject: Re: Thoughts on peeling and readability No. ?In general, to/from raw bit operations are not safe except in a few corner cases (like int and long.) ?Values are not uncontrolled buckets of bits. ? On the other hand, T.default *is* safe, because every type has a default bit pattern which is the initialization of any otherwise uninitialized field or array element. ?(It so happens that this default bit pattern corresponds to all zero bits for all types, though this is mostly a convenience for VM implementors.) ?For a composite value, the default value is comprised of the default value for all fields. ?By *definition*, the all-zero bit pattern is a valid element of all value types. ?However, there is no guarantee that any other bit pattern is valid for any given value type. ? If a particular value type wants to expose to/from raw bit constructors, that?s fine ? but you?re asking for a language feature that applies to *all* values ? and there is no guarantee that this is a safe operation for all values. ? On Dec 12, 2015, at 5:44 PM, Timo Kinnunen wrote: Field layout and bit fiddling isn?t exactly what I was thinking. Rather I was thinking something like Float.floatToRawIntBits() and Double.doubleToRawLongBits(), but without having to know about the types Float and Double or how many bits are in their raw bits. So something like this syntax: ? ??????????????? static T nextUp (T value) { ??????????????? ??????????????? rawBits = T.toRawBits(value); ??????????????? ??????????????? T nextValue = T.fromRawBits(rawBits + 1); ??????????????????????????????? return nextValue; ??????????????? } ? This should fit in Valhalla reasonably well, as it is just a generalization of T.default with its complement operation included. And as it is, all of the problems you listed already apply to T.default. For example, a value type with one long field: If the long value in the field is a handle pointing to a memory-mapped buffer then any use of a default value of such a type could cause a crash. Which can include asking a properly constructed value if it is equal to any of the values in an array you have. --? Have a nice day,? Timo Sent from Mail for Windows 10 ? ? From:?Brian Goetz Sent:?Saturday, December 12, 2015 18:21 To:?Timo Kinnunen;Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net Subject:?Re: Thoughts on peeling and readability ? ? Precise layout and bit control of values are anti-goals of Valhalla, so we're not really exploring this direction at this time. ? The problem with approaches like the one you suggest is they fall apart as soon as you leave the realm of "primitives modeled as values."? What about values that have refs in them?? What about values whose representations are private?? Their implementation is supposed to be in sole control of their representation.? This runs contrary to the "codes like a class" dictum. ? ? ? ? ? ? On 12/12/2015 4:43 AM, Timo Kinnunen wrote: > Hi, >? > One thing that I don?t remember seeing is any syntax for constructing arbitrary values in generic code without having to know about the precise field layouts and what the meaning of such fields is. Something like T.default but for values other than 0. Perhaps T.default(12345) or some such? >? > Or maybe this is slated to go with bytecode type specialization? What sort of syntax is envisioned to be driving that anyways? >? >? >? >? > From brian.goetz at oracle.com Sun Dec 13 19:08:10 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 13 Dec 2015 14:08:10 -0500 Subject: Thoughts on peeling and readability In-Reply-To: <566db9d9.296dc20a.acd5f.fffff668@mx.google.com> References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> <566C576D.2020806@oracle.com> <566ca345.55b21c0a.b1d90.4516@mx.google.com> <39F28927-7C94-47B6-B8D4-EB69448E9917@oracle.com> <566db9d9.296dc20a.acd5f.fffff668@mx.google.com> Message-ID: <566DC21A.8020004@oracle.com> Primitives like int and long are "special" in that all bit patterns are valid and there are no integrity constraints that would prevent a client from requesting a specific bit pattern. But this is the special case, not the general case. A linguistic construct like T.default would have to work for *all* T, not just the special cases. There's nothing to stop you from writing an implementation that takes advantage of knowledge of specific types like int; there's a range of options there. What we're uninterested in doing is allowing clients to have unsafe bit-level access to the representation of all value types. There's also nothing to stop you from writing value types that allow raw-bit operations; we're just not going to require that every value type support that (which is what asking for a bit-oriented T.default would be.) T.default will almost certainly not go through a constructor; the VM will zero out the bits as it does with the existing built-in types (the eight primitive types plus references.) This process is pretty obvious from both a specification and implementation perspective, but it does create some responsibility for writers of value types -- specifically the need to deal with the default bit pattern. As our friends in the .NET community have discovered, trying to enforce that the no-arg ctor is always executed before a value is exposed is a game of whack-a-mole, so having T.default go through a constructor simply reduces the probability of the "implicit null" surprise, but doesn't banish it. One can be as snarky as one likes about the tradeoffs ("reinvented null" vs "reinvented serialization"), but the reality here is that there are risks lurking around both corners. Personally I like the tradeoff we're converging towards, but we are aware it is not perfect. Stepping back, rather than arguing the merits or demerits of a particular solution (which at this point we've more than exhausted), it's far more helpful to talk about the problem instead of the solution. So, let me ask -- what use cases are you concerned about, other then the manufacture of multiple sentinels for use inside data structure implementations? (I suspect in the end you will find that you will be able to accomplish what you need with the tools available, but the "let me specify the bit pattern for an arbitrary value type" approach is not the way to get there.) On 12/13/2015 1:32 PM, Timo Kinnunen wrote: > > Well, it?s ints and longs, and all primitive types copyable around as > ints and longs, and all objects serializable to and from arrays of > ints and longs, and all arrays of such, and all values made of such, > and all arrays of such values, and all values made of such values, > aaand I?m probably missing a dimension or two somewhere. > > These values are just as valid regardless of which bit patterns, > all-zero or not, were used to construct them. They are safe to be > copied around and, if you implemented them yourself, hashCode, equals, > toString and any component-wise operations could also be done safely. > Such operations simply can?t call any foreign code of any of the value > or reference types involved. We don?t expect that we can take an > arbitrary Object, use reflection to zero out its fields and then be > able to call its instance methods like nothing had happened either. > So, for reference types these operations would have to done using > reflection, for value types VarHandles might give better performance. > > Ultimately I guess it all depends on whether T.default invokes some > constructor or not. If it doesn?t or if the constructor is specified > to always succeed trivially then we have reinvented null. Our new > nulls trade off a large number of NPEs for silent invariant > violations. This could be a good tradeoff but without knowing about > the consequences of the violations beforehand it?s gonna be hard to > say for certain. The good news is that hey, null is back! > > If T.default executes a constructor that can refuse an all-zero bit > pattern, then we have reinvented serialization for value types and are > requiring all value types support it. Our serialization protocol only > recognizes one input value and can only deserialize, so it?s a bit > useless. But with the addition of the missing serialize-function we > can then define transforms for long[] <-> val <-> long[] and will have > a quite general and capable system already. > > Or are we gonna just include the worst parts from both? > > > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > *From: *Brian Goetz > *Sent: *Sunday, December 13, 2015 01:20 > *To: *Timo Kinnunen > *Cc: *Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net > *Subject: *Re: Thoughts on peeling and readability > > No. In general, to/from raw bit operations are not safe except in a > few corner cases (like int and long.) Values are not uncontrolled > buckets of bits. > > On the other hand, T.default *is* safe, because every type has a > default bit pattern which is the initialization of any otherwise > uninitialized field or array element. (It so happens that this > default bit pattern corresponds to all zero bits for all types, though > this is mostly a convenience for VM implementors.) For a composite > value, the default value is comprised of the default value for all > fields. By *definition*, the all-zero bit pattern is a valid element > of all value types. However, there is no guarantee that any other bit > pattern is valid for any given value type. > > If a particular value type wants to expose to/from raw bit > constructors, that?s fine ? but you?re asking for a language feature > that applies to *all* values ? and there is no guarantee that this is > a safe operation for all values. > > On Dec 12, 2015, at 5:44 PM, Timo Kinnunen > wrote: > > > > Field layout and bit fiddling isn?t exactly what I was thinking. > Rather I was thinking something like Float.floatToRawIntBits() and > Double.doubleToRawLongBits(), but without having to know about the > types Float and Double or how many bits are in their raw bits. So > something like this syntax: > > static T nextUp (T value) { > > rawBits = T.toRawBits(value); > > T nextValue = > T.fromRawBits(rawBits + 1); > > return nextValue; > > } > > This should fit in Valhalla reasonably well, as it is just a > generalization of T.default with its complement operation > included. And as it is, all of the problems you listed already > apply to T.default. For example, a value type with one long field: > If the long value in the field is a handle pointing to a > memory-mapped buffer then any use of a default value of such a > type could cause a crash. Which can include asking a properly > constructed value if it is equal to any of the values in an array > you have. > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > *From:*Brian Goetz > *Sent:*Saturday, December 12, 2015 18:21 > *To:*Timo Kinnunen;Maurizio Cimadamore;Paul > Benedict;valhalla-dev at openjdk.java.net > > *Subject:*Re: Thoughts on peeling and readability > > Precise layout and bit control of values are anti-goals of > Valhalla, so > > we're not really exploring this direction at this time. > > The problem with approaches like the one you suggest is they fall > apart > > as soon as you leave the realm of "primitives modeled as values." > What > > about values that have refs in them? What about values whose > > representations are private? Their implementation is supposed to > be in > > sole control of their representation. This runs contrary to the > "codes > > like a class" dictum. > > On 12/12/2015 4:43 AM, Timo Kinnunen wrote: > > > Hi, > > > > > > One thing that I don?t remember seeing is any syntax for > constructing arbitrary values in generic code without having to > know about the precise field layouts and what the meaning of such > fields is. Something like T.default but for values other than 0. > Perhaps T.default(12345) or some such? > > > > > > Or maybe this is slated to go with bytecode type specialization? > What sort of syntax is envisioned to be driving that anyways? > > > > > > > > > > > > > > > > From vitalyd at gmail.com Sun Dec 13 19:23:03 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Sun, 13 Dec 2015 14:23:03 -0500 Subject: Thoughts on peeling and readability In-Reply-To: <566DC21A.8020004@oracle.com> References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> <566C576D.2020806@oracle.com> <566ca345.55b21c0a.b1d90.4516@mx.google.com> <39F28927-7C94-47B6-B8D4-EB69448E9917@oracle.com> <566db9d9.296dc20a.acd5f.fffff668@mx.google.com> <566DC21A.8020004@oracle.com> Message-ID: The one thing I always found lacking in C# is the limited constraints on generic types' constructor. You can state that a T must have a public default ctor, but cannot state more than that. For example, if you want to initialize T instances generically from an int, you cannot. If you could provide such a constraint though, then you can use that facility as a way to provide conversion from primitives to the T. But even more generally, constraints that aren't just type based is a nice facility. Not sure if that's even on the table at this point, but wanted to throw that out there. On Sunday, December 13, 2015, Brian Goetz wrote: > Primitives like int and long are "special" in that all bit patterns are > valid and there are no integrity constraints that would prevent a client > from requesting a specific bit pattern. But this is the special case, not > the general case. A linguistic construct like T.default would have to work > for *all* T, not just the special cases. > > There's nothing to stop you from writing an implementation that takes > advantage of knowledge of specific types like int; there's a range of > options there. What we're uninterested in doing is allowing clients to > have unsafe bit-level access to the representation of all value types. > There's also nothing to stop you from writing value types that allow > raw-bit operations; we're just not going to require that every value type > support that (which is what asking for a bit-oriented T.default would be.) > > T.default will almost certainly not go through a constructor; the VM will > zero out the bits as it does with the existing built-in types (the eight > primitive types plus references.) This process is pretty obvious from both > a specification and implementation perspective, but it does create some > responsibility for writers of value types -- specifically the need to deal > with the default bit pattern. > > As our friends in the .NET community have discovered, trying to enforce > that the no-arg ctor is always executed before a value is exposed is a game > of whack-a-mole, so having T.default go through a constructor simply > reduces the probability of the "implicit null" surprise, but doesn't banish > it. One can be as snarky as one likes about the tradeoffs ("reinvented > null" vs "reinvented serialization"), but the reality here is that there > are risks lurking around both corners. Personally I like the tradeoff > we're converging towards, but we are aware it is not perfect. > > > Stepping back, rather than arguing the merits or demerits of a particular > solution (which at this point we've more than exhausted), it's far more > helpful to talk about the problem instead of the solution. So, let me ask > -- what use cases are you concerned about, other then the manufacture of > multiple sentinels for use inside data structure implementations? > > (I suspect in the end you will find that you will be able to accomplish > what you need with the tools available, but the "let me specify the bit > pattern for an arbitrary value type" approach is not the way to get there.) > > > On 12/13/2015 1:32 PM, Timo Kinnunen wrote: > >> >> Well, it?s ints and longs, and all primitive types copyable around as >> ints and longs, and all objects serializable to and from arrays of ints and >> longs, and all arrays of such, and all values made of such, and all arrays >> of such values, and all values made of such values, aaand I?m probably >> missing a dimension or two somewhere. >> >> These values are just as valid regardless of which bit patterns, all-zero >> or not, were used to construct them. They are safe to be copied around and, >> if you implemented them yourself, hashCode, equals, toString and any >> component-wise operations could also be done safely. Such operations simply >> can?t call any foreign code of any of the value or reference types >> involved. We don?t expect that we can take an arbitrary Object, use >> reflection to zero out its fields and then be able to call its instance >> methods like nothing had happened either. So, for reference types these >> operations would have to done using reflection, for value types VarHandles >> might give better performance. >> >> Ultimately I guess it all depends on whether T.default invokes some >> constructor or not. If it doesn?t or if the constructor is specified to >> always succeed trivially then we have reinvented null. Our new nulls trade >> off a large number of NPEs for silent invariant violations. This could be a >> good tradeoff but without knowing about the consequences of the violations >> beforehand it?s gonna be hard to say for certain. The good news is that >> hey, null is back! >> >> If T.default executes a constructor that can refuse an all-zero bit >> pattern, then we have reinvented serialization for value types and are >> requiring all value types support it. Our serialization protocol only >> recognizes one input value and can only deserialize, so it?s a bit useless. >> But with the addition of the missing serialize-function we can then define >> transforms for long[] <-> val <-> long[] and will have a quite general and >> capable system already. >> >> Or are we gonna just include the worst parts from both? >> >> >> >> >> >> >> -- >> Have a nice day, >> Timo >> >> Sent from Mail for Windows 10 >> >> >> *From: *Brian Goetz >> *Sent: *Sunday, December 13, 2015 01:20 >> *To: *Timo Kinnunen >> *Cc: *Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net >> *Subject: *Re: Thoughts on peeling and readability >> >> No. In general, to/from raw bit operations are not safe except in a few >> corner cases (like int and long.) Values are not uncontrolled buckets of >> bits. >> >> On the other hand, T.default *is* safe, because every type has a default >> bit pattern which is the initialization of any otherwise uninitialized >> field or array element. (It so happens that this default bit pattern >> corresponds to all zero bits for all types, though this is mostly a >> convenience for VM implementors.) For a composite value, the default value >> is comprised of the default value for all fields. By *definition*, the >> all-zero bit pattern is a valid element of all value types. However, there >> is no guarantee that any other bit pattern is valid for any given value >> type. >> >> If a particular value type wants to expose to/from raw bit constructors, >> that?s fine ? but you?re asking for a language feature that applies to >> *all* values ? and there is no guarantee that this is a safe operation for >> all values. >> >> On Dec 12, 2015, at 5:44 PM, Timo Kinnunen > > wrote: >> >> >> >> Field layout and bit fiddling isn?t exactly what I was thinking. >> Rather I was thinking something like Float.floatToRawIntBits() and >> Double.doubleToRawLongBits(), but without having to know about the >> types Float and Double or how many bits are in their raw bits. So >> something like this syntax: >> >> static T nextUp (T value) { >> >> rawBits = T.toRawBits(value); >> >> T nextValue = >> T.fromRawBits(rawBits + 1); >> >> return nextValue; >> >> } >> >> This should fit in Valhalla reasonably well, as it is just a >> generalization of T.default with its complement operation >> included. And as it is, all of the problems you listed already >> apply to T.default. For example, a value type with one long field: >> If the long value in the field is a handle pointing to a >> memory-mapped buffer then any use of a default value of such a >> type could cause a crash. Which can include asking a properly >> constructed value if it is equal to any of the values in an array >> you have. >> >> >> >> >> -- >> Have a nice day, >> Timo >> >> Sent from Mail for Windows 10 >> >> >> *From:*Brian Goetz >> *Sent:*Saturday, December 12, 2015 18:21 >> *To:*Timo Kinnunen;Maurizio Cimadamore;Paul >> Benedict;valhalla-dev at openjdk.java.net >> >> *Subject:*Re: Thoughts on peeling and readability >> >> Precise layout and bit control of values are anti-goals of >> Valhalla, so >> >> we're not really exploring this direction at this time. >> >> The problem with approaches like the one you suggest is they fall >> apart >> >> as soon as you leave the realm of "primitives modeled as values." >> What >> >> about values that have refs in them? What about values whose >> >> representations are private? Their implementation is supposed to >> be in >> >> sole control of their representation. This runs contrary to the >> "codes >> >> like a class" dictum. >> >> On 12/12/2015 4:43 AM, Timo Kinnunen wrote: >> >> > Hi, >> >> > >> >> > One thing that I don?t remember seeing is any syntax for >> constructing arbitrary values in generic code without having to >> know about the precise field layouts and what the meaning of such >> fields is. Something like T.default but for values other than 0. >> Perhaps T.default(12345) or some such? >> >> > >> >> > Or maybe this is slated to go with bytecode type specialization? >> What sort of syntax is envisioned to be driving that anyways? >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > -- Sent from my phone From twhitmore.nz at gmail.com Mon Dec 14 07:15:54 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Mon, 14 Dec 2015 20:15:54 +1300 Subject: valhalla-dev Digest, Vol 17, Issue 5 In-Reply-To: References: Message-ID: Hi Timo, You seem to be asking about bit-level patterns -- is that for the purpose of 'sentinels'? My position is with Brian, that exposing the bit-level representation is completely undesirable. T.default is intended to provide a good 'default value' for eg. empty cells in a collection to default to. Previously, however, I did propose a 'T.sentinel' capability, which would have provided a "reasonable" sentinel value for most types. Reasonable, not perfect, since sentinels are a compromise & prevent use of the complete value-space for the type. Nevertheless, I believed there was substantial usefulness & value in having such a capability available. Part of the rationale is that "null" for references has two common uses; 1) as a default, 2) as a sentinel. For numeric primitives, by comparison, the default (zero) is far too important as a value to be a reasonable sentinel. I would obviously not see the standard Java collection implementations using these, but for specialized/ custom collections, I'd still advocate the value of exposing that. int.sentinel = Integer.MIN_VALUE double.sentinel = NaN Object.sentinel = null etc. 'boolean' would be the only case not well supported. Would you support that as useful, Timo? Regards, Thomas From twhitmore.nz at gmail.com Mon Dec 14 07:41:52 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Mon, 14 Dec 2015 20:41:52 +1300 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: Hi Vitaly, people [If you want a BooleanBitList, create a separate class for it.] > But why? I have a generic class ArrayList that satisfies contract of whomever wants to use it ArrayList is an *implementation*, java.util.List is a contract. Now the List contract can be filled by anybody -- ArrayList, ArrayList using a byte-per-cell, BooleanBitList using a bit-per-cell. The "contract" you referred to (but referencing the incorrect type) gives you full flexibility, to have whatever storage & addressing are provided by the implementation. An "implementation" on the other hand, to be meaningful, has to actually make some things concrete. In a Collection, storage is likely to be that. I am happy for bytewise addressing & cell width (in bytes) to be abstracted in the service of generic specialization; but you're trying to cross a really clear boundary, when you change the CPU instructions, storage addressing etc etc etc to address memory below the "byte" level. I'm pretty sure you can see there's a boundary, too -- Good design is about choosing the right boundaries & respecting them, such that a useful, meaningful & efficient abstraction is presented at the higher level. > Why define another abstraction just to support better/more efficient storage for a given *type*? There's no capability/semantic > change here, it's still an ArrayList. Another abstraction? Classes have existed since Java 1.0, hardly new. We should use them, they work well for things such as types providing different implementations of an interface. And to correct your second sentence: there's no capability/semantic change, it's still a *List*. But there would be _fundamental difference in implementation_, so it should be a different concrete type. > The world today requires that but it's unnecessary; storage is an implementation detail that shouldn't necessarily "leak" into the > type system. The purpose of a concrete Collection implementation is to implement storage. 'List' keeps storage out of the type. ArrayList, LinkedList & the putative BooleanBitList implement it. It's probably mistaken to claim that implementation is "just a detail" that can (or should be) abstracted out of the type system. The fundamental purpose of concrete classes is to organize _concrete code which does something_, aka implementation, since without implementation a system does nothing. Yes, a preprocessor can do "anything". Is it a good idea? No. Regards, Thomas From maurizio.cimadamore at oracle.com Mon Dec 14 13:32:45 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 14 Dec 2015 13:32:45 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: Add PoolReader support for Valhalla new pool entries Message-ID: <201512141332.tBEDWjWA015938@aojmv0008.oracle.com> Changeset: 009918773810 Author: mcimadamore Date: 2015-12-14 13:32 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/009918773810 Enhancement: Add PoolReader support for Valhalla new pool entries ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolReader.java + test/tools/javac/valhalla/typespec/separate06/Foo.java + test/tools/javac/valhalla/typespec/separate06/Separate06.java From maurizio.cimadamore at oracle.com Mon Dec 14 13:40:03 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 14 Dec 2015 13:40:03 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: remove unused imports in PoolReader Message-ID: <201512141340.tBEDe4fD018374@aojmv0008.oracle.com> Changeset: 3cc4be0968e4 Author: mcimadamore Date: 2015-12-14 13:39 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/3cc4be0968e4 Fix: remove unused imports in PoolReader ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolReader.java From vitalyd at gmail.com Mon Dec 14 14:56:49 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 14 Dec 2015 09:56:49 -0500 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: On Monday, December 14, 2015, Thomas W wrote: > Hi Vitaly, people > > [If you want a BooleanBitList, create a separate class for it.] > > But why? I have a generic class ArrayList that satisfies contract of > whomever wants to use it > > ArrayList is an *implementation*, java.util.List is a contract. Now the > List contract can be filled by anybody -- ArrayList, > ArrayList using a byte-per-cell, BooleanBitList using a > bit-per-cell. > > The "contract" you referred to (but referencing the incorrect type) gives > you full flexibility, to have whatever storage & addressing are provided by > the implementation. An "implementation" on the other hand, to be > meaningful, has to actually make some things concrete. In a Collection, > storage is likely to be that. > > Nope, I referenced the right type :). ArrayList fulfills the higher level List contract but it has implementation level contract that people rely on and leverage intentionally. > I am happy for bytewise addressing & cell width (in bytes) to be > abstracted in the service of generic specialization; but you're trying to > cross a really clear boundary, when you change the CPU instructions, > storage addressing etc etc etc to address memory below the "byte" level. > Perhaps we'll have to disagree here, I don't think there's a clear boundary here. > I'm pretty sure you can see there's a boundary, too -- Good design is > about choosing the right boundaries & respecting them, such that a useful, > meaningful & efficient abstraction is presented at the higher level. > Agreed in principle. > > Why define another abstraction just to support better/more efficient > storage for a given *type*? There's no capability/semantic > > change here, it's still an ArrayList. > > Another abstraction? Classes have existed since Java 1.0, hardly new. We > should use them, they work well for things such as types providing > different implementations of an interface. > Abstraction is meant in general sense here. You're providing a named type here that user will be aware of one way or another; either the type is exposed directly or hidden behind some factory that returns it. Unless of course the factory takes a Class and switches on known types, but we'll probably agree that's not a good answer. > > And to correct your second sentence: there's no capability/semantic > change, it's still a *List*. But there would be _fundamental difference in > implementation_, so it should be a different concrete type. > What is the fundamental difference? > > > The world today requires that but it's unnecessary; storage is an > implementation detail that shouldn't necessarily "leak" into the > > type system. > > The purpose of a concrete Collection implementation is to implement > storage. 'List' keeps storage out of the type. ArrayList, LinkedList & the > putative BooleanBitList implement it. > > It's probably mistaken to claim that implementation is "just a detail" > that can (or should be) abstracted out of the type system. The fundamental > purpose of concrete classes is to organize _concrete code which does > something_, aka implementation, since without implementation a system does > nothing. > Again, agree in principle. I'm just not seeing the fundamental difference here that warrants this. > Yes, a preprocessor can do "anything". Is it a good idea? No. > Don't think this is anything close to being a preprocessor. Let me ask you this - how do you propose users make use of more optimal ArrayList-like implementations? How do you expose your BooleanBitList? How do you expose more efficient implementations in the future when you didn't envision a particular specialization today? How does user make use of it without touching their source code? > > > Regards, > Thomas > -- Sent from my phone From brian.goetz at oracle.com Mon Dec 14 15:06:34 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 14 Dec 2015 10:06:34 -0500 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: What Vitaly is saying is that being able to control the operation of new Foo and have that evaluate to something that matches the contract of Foo, but not necessarily the same representation of other parameterizations of Foo, gives library designers more power to deliver useful and performant libraries without the user having to worry about it directly. This is a valid point, and one we?re willing to explore, though have not committed to anything yet. Note that we almost certainly would *not* allow use-site control over this ? a client will not be able to say ?I want to silently override the definition of ArrayList?, just as they cannot inject default methods into ArrayList. Thomas? concern is also a valid one ? does this break parametricity? Also worth exploring. Bottom line is: this is within the realm of possible, but by no means guaranteed. On Dec 14, 2015, at 9:56 AM, Vitaly Davidovich wrote: > On Monday, December 14, 2015, Thomas W wrote: > >> Hi Vitaly, people >> >> [If you want a BooleanBitList, create a separate class for it.] >>> But why? I have a generic class ArrayList that satisfies contract of >> whomever wants to use it >> >> ArrayList is an *implementation*, java.util.List is a contract. Now the >> List contract can be filled by anybody -- ArrayList, >> ArrayList using a byte-per-cell, BooleanBitList using a >> bit-per-cell. >> >> The "contract" you referred to (but referencing the incorrect type) gives >> you full flexibility, to have whatever storage & addressing are provided by >> the implementation. An "implementation" on the other hand, to be >> meaningful, has to actually make some things concrete. In a Collection, >> storage is likely to be that. >> >> > Nope, I referenced the right type :). ArrayList fulfills the higher level > List contract but it has implementation level contract that people rely on > and leverage intentionally. > > >> I am happy for bytewise addressing & cell width (in bytes) to be >> abstracted in the service of generic specialization; but you're trying to >> cross a really clear boundary, when you change the CPU instructions, >> storage addressing etc etc etc to address memory below the "byte" level. >> > > Perhaps we'll have to disagree here, I don't think there's a clear boundary > here. > > >> I'm pretty sure you can see there's a boundary, too -- Good design is >> about choosing the right boundaries & respecting them, such that a useful, >> meaningful & efficient abstraction is presented at the higher level. >> > > Agreed in principle. > > >>> Why define another abstraction just to support better/more efficient >> storage for a given *type*? There's no capability/semantic >>> change here, it's still an ArrayList. >> >> Another abstraction? Classes have existed since Java 1.0, hardly new. We >> should use them, they work well for things such as types providing >> different implementations of an interface. >> > > Abstraction is meant in general sense here. You're providing a named type > here that user will be aware of one way or another; either the type is > exposed directly or hidden behind some factory that returns it. Unless of > course the factory takes a Class and switches on known types, but we'll > probably agree that's not a good answer. > >> >> And to correct your second sentence: there's no capability/semantic >> change, it's still a *List*. But there would be _fundamental difference in >> implementation_, so it should be a different concrete type. >> > > What is the fundamental difference? > >> >>> The world today requires that but it's unnecessary; storage is an >> implementation detail that shouldn't necessarily "leak" into the >>> type system. >> >> The purpose of a concrete Collection implementation is to implement >> storage. 'List' keeps storage out of the type. ArrayList, LinkedList & the >> putative BooleanBitList implement it. >> >> It's probably mistaken to claim that implementation is "just a detail" >> that can (or should be) abstracted out of the type system. The fundamental >> purpose of concrete classes is to organize _concrete code which does >> something_, aka implementation, since without implementation a system does >> nothing. >> > > Again, agree in principle. I'm just not seeing the fundamental difference > here that warrants this. > >> Yes, a preprocessor can do "anything". Is it a good idea? No. >> > > Don't think this is anything close to being a preprocessor. > > Let me ask you this - how do you propose users make use of more optimal > ArrayList-like implementations? How do you expose your BooleanBitList? How > do you expose more efficient implementations in the future when you didn't > envision a particular specialization today? How does user make use of it > without touching their source code? > >> >> >> Regards, >> Thomas >> > > > -- > Sent from my phone From vitalyd at gmail.com Mon Dec 14 15:24:55 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 14 Dec 2015 10:24:55 -0500 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: On Monday, December 14, 2015, Brian Goetz wrote: > What Vitaly is saying is that being able to control the operation of > > new Foo > > and have that evaluate to something that matches the contract of Foo, but > not necessarily the same representation of other parameterizations of Foo, > gives library designers more power to deliver useful and performant > libraries without the user having to worry about it directly. > > This is a valid point, and one we?re willing to explore, though have not > committed to anything yet. > That's right. This is really the crux of my position, the other stuff is more subjective and philosophical. There may be some (micro) performance difference artifacts as well between the two approaches, but I'll leave that alone here. > Note that we almost certainly would *not* allow use-site control over this > ? a client will not be able to say ?I want to silently override the > definition of ArrayList?, just as they cannot inject default methods > into ArrayList. > > Agreed. > Thomas? concern is also a valid one ? does this break parametricity? Also > worth exploring. > I see these things as being somewhat related but orthogonal. > > Bottom line is: this is within the realm of possible, but by no means > guaranteed. > > On Dec 14, 2015, at 9:56 AM, Vitaly Davidovich > wrote: > > On Monday, December 14, 2015, Thomas W > wrote: > > Hi Vitaly, people > > [If you want a BooleanBitList, create a separate class for it.] > > But why? I have a generic class ArrayList that satisfies contract of > > whomever wants to use it > > ArrayList is an *implementation*, java.util.List is a contract. Now the > List contract can be filled by anybody -- ArrayList, > ArrayList using a byte-per-cell, BooleanBitList using a > bit-per-cell. > > The "contract" you referred to (but referencing the incorrect type) gives > you full flexibility, to have whatever storage & addressing are provided by > the implementation. An "implementation" on the other hand, to be > meaningful, has to actually make some things concrete. In a Collection, > storage is likely to be that. > > > Nope, I referenced the right type :). ArrayList fulfills the higher level > List contract but it has implementation level contract that people rely on > and leverage intentionally. > > > I am happy for bytewise addressing & cell width (in bytes) to be > abstracted in the service of generic specialization; but you're trying to > cross a really clear boundary, when you change the CPU instructions, > storage addressing etc etc etc to address memory below the "byte" level. > > > Perhaps we'll have to disagree here, I don't think there's a clear boundary > here. > > > I'm pretty sure you can see there's a boundary, too -- Good design is > about choosing the right boundaries & respecting them, such that a useful, > meaningful & efficient abstraction is presented at the higher level. > > > Agreed in principle. > > > Why define another abstraction just to support better/more efficient > > storage for a given *type*? There's no capability/semantic > > change here, it's still an ArrayList. > > > Another abstraction? Classes have existed since Java 1.0, hardly new. We > should use them, they work well for things such as types providing > different implementations of an interface. > > > Abstraction is meant in general sense here. You're providing a named type > here that user will be aware of one way or another; either the type is > exposed directly or hidden behind some factory that returns it. Unless of > course the factory takes a Class and switches on known types, but we'll > probably agree that's not a good answer. > > > And to correct your second sentence: there's no capability/semantic > change, it's still a *List*. But there would be _fundamental difference in > implementation_, so it should be a different concrete type. > > > What is the fundamental difference? > > > The world today requires that but it's unnecessary; storage is an > > implementation detail that shouldn't necessarily "leak" into the > > type system. > > > The purpose of a concrete Collection implementation is to implement > storage. 'List' keeps storage out of the type. ArrayList, LinkedList & the > putative BooleanBitList implement it. > > It's probably mistaken to claim that implementation is "just a detail" > that can (or should be) abstracted out of the type system. The fundamental > purpose of concrete classes is to organize _concrete code which does > something_, aka implementation, since without implementation a system does > nothing. > > > Again, agree in principle. I'm just not seeing the fundamental difference > here that warrants this. > > Yes, a preprocessor can do "anything". Is it a good idea? No. > > > Don't think this is anything close to being a preprocessor. > > Let me ask you this - how do you propose users make use of more optimal > ArrayList-like implementations? How do you expose your BooleanBitList? How > do you expose more efficient implementations in the future when you didn't > envision a particular specialization today? How does user make use of it > without touching their source code? > > > > Regards, > Thomas > > > > -- > Sent from my phone > > > -- Sent from my phone From Mohammad.Rezaei at gs.com Mon Dec 14 15:59:11 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Mon, 14 Dec 2015 10:59:11 -0500 Subject: valhalla-dev Digest, Vol 17, Issue 5 In-Reply-To: References: Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D886@GSCMAMP06EX.firmwide.corp.gs.com> >Previously, however, I did propose a 'T.sentinel' capability, which would >have provided a "reasonable" sentinel value for most types. Reasonable, not >perfect, since sentinels are a compromise & prevent use of the complete >value-space for the type. Having a sentinel value does *not* preclude that value from being used as a valid value for that type. A properly designed data structure will use a sentinel in a completely encapsulated way, such that the user of the data structure will never need to know what the sentinel is. For example, most primitive hash structures (e.g. trove, gs collections) use one or more sentinels to represent FREE, REMOVED, etc. The values are implementation dependent, but the user can still store/retrieve those values from these structures. Thanks Moh From paul.sandoz at oracle.com Mon Dec 14 16:04:10 2015 From: paul.sandoz at oracle.com (paul.sandoz at oracle.com) Date: Mon, 14 Dec 2015 16:04:10 +0000 Subject: hg: valhalla/valhalla/jdk: Any'ify Queue/Deque/AbstractQueue/ArrayDeque. Message-ID: <201512141604.tBEG4FPn027226@aojmv0008.oracle.com> Changeset: f65e797d198a Author: psandoz Date: 2015-12-14 16:59 +0100 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/f65e797d198a Any'ify Queue/Deque/AbstractQueue/ArrayDeque. Clone the java/util/Collection/MOAT test and use for testing ref instances of any'fied collection. Close the java/util/Deque/ChorusLine and use for testing ref and value instances of ArrayDeque. Fix various bugs to get the tests to run. ! src/java.base/share/classes/java/anyutil/AbstractCollection.java ! src/java.base/share/classes/java/anyutil/AbstractQueue.java ! src/java.base/share/classes/java/anyutil/AbstractSet.java ! src/java.base/share/classes/java/anyutil/ArrayDeque.java ! src/java.base/share/classes/java/anyutil/Deque.java ! src/java.base/share/classes/java/anyutil/Iterable.java ! src/java.base/share/classes/java/anyutil/LinkedHashSet.java ! src/java.base/share/classes/java/anyutil/Queue.java + test/valhalla/java/anyutil/Collection/RefMOAT.java + test/valhalla/java/anyutil/Deque/RefChorusLine.java + test/valhalla/java/anyutil/Deque/ValChorusLine.java From brian.goetz at oracle.com Mon Dec 14 20:07:49 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 14 Dec 2015 15:07:49 -0500 Subject: Announcing: valhalla-spec-experts group Message-ID: <566F2195.2030100@oracle.com> It's time to take the next step in Valhalla's evolution. Accordingly, I've created the valhalla-spec-experts list, which will be the precursor to an eventual JCP Expert Group, and will operate under the JCP-friendly ToU. This list is publicly archived in the obvious place, and the initial membership includes both corporate participation (Oracle, IBM, Red Hat, Jetbrains, Google) and individuals from the OpenJDK community. As was the case when we formed the Lambda experts list, the role of valhalla-dev should now revert to its intended purpose, which is to discuss the *implementation* of Valhalla in OpenJDK. Examples of appropriate discussion topics for valhalla-dev include: - Bug reports - Code review comments - Test cases - Build or porting problems - Migration experiences for your own code What we are trying to avoid is: having design discussions play out concurrently in multiple places with differing sets of players. So please, we ask that you exercise restraint in commenting on active EG discussions -- as design decisions start to converge, we'll announce them here, and of course the primary place for implementation will remain the valhalla repositories. Please remember that EG discussions often happen at a different pace than in other open-source projects. Days can go by without a response, but this doesn't mean that the EG is not thinking about it, doesn't care, or is "stumped". Often it simply means the EG members are working privately (or with their teams) to figure out what their position is, and will respond when they've thought about it more. So, please let those discussions play out. From ali.ebrahimi1781 at gmail.com Mon Dec 14 22:24:41 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Tue, 15 Dec 2015 01:54:41 +0330 Subject: Announcing: valhalla-spec-experts group In-Reply-To: <566F2195.2030100@oracle.com> References: <566F2195.2030100@oracle.com> Message-ID: Hi Brian, Wouldn't we have valhalla-spec-(observers/comments)? -- Best Regards, Ali Ebrahimi On Mon, Dec 14, 2015 at 11:37 PM, Brian Goetz wrote: > It's time to take the next step in Valhalla's evolution. Accordingly, > I've created the valhalla-spec-experts list, which will be the precursor to > an eventual JCP Expert Group, and will operate under the JCP-friendly ToU. > This list is publicly archived in the obvious place, and the initial > membership includes both corporate participation (Oracle, IBM, Red Hat, > Jetbrains, Google) and individuals from the OpenJDK community. > > As was the case when we formed the Lambda experts list, the role of > valhalla-dev should now revert to its intended purpose, which is to discuss > the *implementation* of Valhalla in OpenJDK. Examples of appropriate > discussion topics for valhalla-dev include: > > - Bug reports > - Code review comments > - Test cases > - Build or porting problems > - Migration experiences for your own code > > What we are trying to avoid is: having design discussions play out > concurrently in multiple places with differing sets of players. So please, > we ask that you exercise restraint in commenting on active EG discussions > -- as design decisions start to converge, we'll announce them here, and of > course the primary place for implementation will remain the valhalla > repositories. > > Please remember that EG discussions often happen at a different pace than > in other open-source projects. Days can go by without a response, but this > doesn't mean that the EG is not thinking about it, doesn't care, or is > "stumped". Often it simply means the EG members are working privately (or > with their teams) to figure out what their position is, and will respond > when they've thought about it more. So, please let those discussions play > out. > > > From brian.goetz at oracle.com Mon Dec 14 22:28:47 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 14 Dec 2015 17:28:47 -0500 Subject: Announcing: valhalla-spec-experts group In-Reply-To: References: <566F2195.2030100@oracle.com> Message-ID: <566F429F.6000505@oracle.com> Eventually, yes; there's no reason not to have these, except for the administrative overhead of setting them up. But I didn't want to delay getting the discussions going in a more transparent manner over administrivia, so these will come later, and of course I'll announce them here when they do. Cheers, -Brian On 12/14/2015 5:24 PM, Ali Ebrahimi wrote: > Hi Brian, > Wouldn't we have valhalla-spec-(observers/comments)? > > -- > > Best Regards, > Ali Ebrahimi > On Mon, Dec 14, 2015 at 11:37 PM, Brian Goetz > wrote: > > It's time to take the next step in Valhalla's evolution. > Accordingly, I've created the valhalla-spec-experts list, which > will be the precursor to an eventual JCP Expert Group, and will > operate under the JCP-friendly ToU. This list is publicly > archived in the obvious place, and the initial membership includes > both corporate participation (Oracle, IBM, Red Hat, Jetbrains, > Google) and individuals from the OpenJDK community. > > As was the case when we formed the Lambda experts list, the role > of valhalla-dev should now revert to its intended purpose, which > is to discuss the *implementation* of Valhalla in OpenJDK. > Examples of appropriate discussion topics for valhalla-dev include: > > - Bug reports > - Code review comments > - Test cases > - Build or porting problems > - Migration experiences for your own code > > What we are trying to avoid is: having design discussions play out > concurrently in multiple places with differing sets of players. > So please, we ask that you exercise restraint in commenting on > active EG discussions -- as design decisions start to converge, > we'll announce them here, and of course the primary place for > implementation will remain the valhalla repositories. > > Please remember that EG discussions often happen at a different > pace than in other open-source projects. Days can go by without a > response, but this doesn't mean that the EG is not thinking about > it, doesn't care, or is "stumped". Often it simply means the EG > members are working privately (or with their teams) to figure out > what their position is, and will respond when they've thought > about it more. So, please let those discussions play out. > > > From brian.goetz at oracle.com Tue Dec 15 15:08:24 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 15 Dec 2015 10:08:24 -0500 Subject: Announcing: valhalla-spec-experts group In-Reply-To: <566F2195.2030100@oracle.com> References: <566F2195.2030100@oracle.com> Message-ID: <56702CE8.4070400@oracle.com> Several people tried to subscribe to valhalla-spec-experts, and were surprised to get rejected. So let me clarify what's going on. The valhalla-spec-experts list is publicly READABLE, but not publicly SUBSCRIBABLE -- the membership is limited. Now, lots of folks are interested in subscribing not as members of the EG, but to get updates on a timely basis without polling the archives, which is totally reasonable. For lambda, we had a lambda-spec-observers list which received copies of the traffic to lambda-spec-experts. We plan to do the same for valhalla, we just haven't yet gone through the administrative work to do so (and I'd rather make some technical progress this week than futz with mailing list provisioning.) So it will take some time to get the -observers and -comments lists up and running. So please be patient! On 12/14/2015 3:07 PM, Brian Goetz wrote: > It's time to take the next step in Valhalla's evolution. Accordingly, > I've created the valhalla-spec-experts list, which will be the > precursor to an eventual JCP Expert Group, and will operate under the > JCP-friendly ToU. This list is publicly archived in the obvious > place, and the initial membership includes both corporate > participation (Oracle, IBM, Red Hat, Jetbrains, Google) and > individuals from the OpenJDK community. > > As was the case when we formed the Lambda experts list, the role of > valhalla-dev should now revert to its intended purpose, which is to > discuss the *implementation* of Valhalla in OpenJDK. Examples of > appropriate discussion topics for valhalla-dev include: > > - Bug reports > - Code review comments > - Test cases > - Build or porting problems > - Migration experiences for your own code > > What we are trying to avoid is: having design discussions play out > concurrently in multiple places with differing sets of players. So > please, we ask that you exercise restraint in commenting on active EG > discussions -- as design decisions start to converge, we'll announce > them here, and of course the primary place for implementation will > remain the valhalla repositories. > > Please remember that EG discussions often happen at a different pace > than in other open-source projects. Days can go by without a > response, but this doesn't mean that the EG is not thinking about it, > doesn't care, or is "stumped". Often it simply means the EG members > are working privately (or with their teams) to figure out what their > position is, and will respond when they've thought about it more. So, > please let those discussions play out. > > From pbenedict at apache.org Tue Dec 15 15:11:41 2015 From: pbenedict at apache.org (Paul Benedict) Date: Tue, 15 Dec 2015 09:11:41 -0600 Subject: Announcing: valhalla-spec-experts group In-Reply-To: <56702CE8.4070400@oracle.com> References: <566F2195.2030100@oracle.com> <56702CE8.4070400@oracle.com> Message-ID: Brian, since you already know who is on the EG, why not grant write permissions to them only, and the rest allowed to be subscribale? Or does mailman not allow those types of controls? Cheers, Paul On Tue, Dec 15, 2015 at 9:08 AM, Brian Goetz wrote: > Several people tried to subscribe to valhalla-spec-experts, and were > surprised to get rejected. So let me clarify what's going on. > > The valhalla-spec-experts list is publicly READABLE, but not publicly > SUBSCRIBABLE -- the membership is limited. Now, lots of folks are > interested in subscribing not as members of the EG, but to get updates on a > timely basis without polling the archives, which is totally reasonable. > For lambda, we had a lambda-spec-observers list which received copies of > the traffic to lambda-spec-experts. We plan to do the same for valhalla, > we just haven't yet gone through the administrative work to do so (and I'd > rather make some technical progress this week than futz with mailing list > provisioning.) So it will take some time to get the -observers and > -comments lists up and running. So please be patient! > > > > On 12/14/2015 3:07 PM, Brian Goetz wrote: > >> It's time to take the next step in Valhalla's evolution. Accordingly, >> I've created the valhalla-spec-experts list, which will be the precursor to >> an eventual JCP Expert Group, and will operate under the JCP-friendly ToU. >> This list is publicly archived in the obvious place, and the initial >> membership includes both corporate participation (Oracle, IBM, Red Hat, >> Jetbrains, Google) and individuals from the OpenJDK community. >> >> As was the case when we formed the Lambda experts list, the role of >> valhalla-dev should now revert to its intended purpose, which is to discuss >> the *implementation* of Valhalla in OpenJDK. Examples of appropriate >> discussion topics for valhalla-dev include: >> >> - Bug reports >> - Code review comments >> - Test cases >> - Build or porting problems >> - Migration experiences for your own code >> >> What we are trying to avoid is: having design discussions play out >> concurrently in multiple places with differing sets of players. So please, >> we ask that you exercise restraint in commenting on active EG discussions >> -- as design decisions start to converge, we'll announce them here, and of >> course the primary place for implementation will remain the valhalla >> repositories. >> >> Please remember that EG discussions often happen at a different pace than >> in other open-source projects. Days can go by without a response, but this >> doesn't mean that the EG is not thinking about it, doesn't care, or is >> "stumped". Often it simply means the EG members are working privately (or >> with their teams) to figure out what their position is, and will respond >> when they've thought about it more. So, please let those discussions play >> out. >> >> >> > From kirk.pepperdine at gmail.com Tue Dec 15 15:21:16 2015 From: kirk.pepperdine at gmail.com (kirk.pepperdine at gmail.com) Date: Tue, 15 Dec 2015 16:21:16 +0100 Subject: Announcing: valhalla-spec-experts group In-Reply-To: References: <566F2195.2030100@oracle.com> <56702CE8.4070400@oracle.com> Message-ID: <33C13AC4-B5C0-44D2-AA22-5E77E72DCF05@gmail.com> I think that once you are subscribed you can email the list. I?m very much interested in observing but since I don?t have anything useful to add, I?d stay quiet.. but not everyone would. It?s a fine line to balance between stifling the discussion and not being distracted by endless discussions. Lets give Brian the benefit of his position on this one. Regards, Kirk > On Dec 15, 2015, at 4:11 PM, Paul Benedict wrote: > > Brian, since you already know who is on the EG, why not grant write > permissions to them only, and the rest allowed to be subscribale? Or does > mailman not allow those types of controls? > > Cheers, > Paul > > On Tue, Dec 15, 2015 at 9:08 AM, Brian Goetz wrote: > >> Several people tried to subscribe to valhalla-spec-experts, and were >> surprised to get rejected. So let me clarify what's going on. >> >> The valhalla-spec-experts list is publicly READABLE, but not publicly >> SUBSCRIBABLE -- the membership is limited. Now, lots of folks are >> interested in subscribing not as members of the EG, but to get updates on a >> timely basis without polling the archives, which is totally reasonable. >> For lambda, we had a lambda-spec-observers list which received copies of >> the traffic to lambda-spec-experts. We plan to do the same for valhalla, >> we just haven't yet gone through the administrative work to do so (and I'd >> rather make some technical progress this week than futz with mailing list >> provisioning.) So it will take some time to get the -observers and >> -comments lists up and running. So please be patient! >> >> >> >> On 12/14/2015 3:07 PM, Brian Goetz wrote: >> >>> It's time to take the next step in Valhalla's evolution. Accordingly, >>> I've created the valhalla-spec-experts list, which will be the precursor to >>> an eventual JCP Expert Group, and will operate under the JCP-friendly ToU. >>> This list is publicly archived in the obvious place, and the initial >>> membership includes both corporate participation (Oracle, IBM, Red Hat, >>> Jetbrains, Google) and individuals from the OpenJDK community. >>> >>> As was the case when we formed the Lambda experts list, the role of >>> valhalla-dev should now revert to its intended purpose, which is to discuss >>> the *implementation* of Valhalla in OpenJDK. Examples of appropriate >>> discussion topics for valhalla-dev include: >>> >>> - Bug reports >>> - Code review comments >>> - Test cases >>> - Build or porting problems >>> - Migration experiences for your own code >>> >>> What we are trying to avoid is: having design discussions play out >>> concurrently in multiple places with differing sets of players. So please, >>> we ask that you exercise restraint in commenting on active EG discussions >>> -- as design decisions start to converge, we'll announce them here, and of >>> course the primary place for implementation will remain the valhalla >>> repositories. >>> >>> Please remember that EG discussions often happen at a different pace than >>> in other open-source projects. Days can go by without a response, but this >>> doesn't mean that the EG is not thinking about it, doesn't care, or is >>> "stumped". Often it simply means the EG members are working privately (or >>> with their teams) to figure out what their position is, and will respond >>> when they've thought about it more. So, please let those discussions play >>> out. >>> >>> >>> >> From brian.goetz at oracle.com Tue Dec 15 15:24:57 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 15 Dec 2015 10:24:57 -0500 Subject: Announcing: valhalla-spec-experts group In-Reply-To: <33C13AC4-B5C0-44D2-AA22-5E77E72DCF05@gmail.com> References: <566F2195.2030100@oracle.com> <56702CE8.4070400@oracle.com> <33C13AC4-B5C0-44D2-AA22-5E77E72DCF05@gmail.com> Message-ID: <567030C9.2080407@oracle.com> Exactly so; members can post, and if they can, they will. Our mailing list software doesn't let us subscribe people read-only. You can set a moderated bit per-user, but then I get a moderation notification every time someone tries to post -- which defeats the purpose. Plus, observers also want a place to have discussions among themselves. Which is why we've historically had the -experts and -observers split. On 12/15/2015 10:21 AM, kirk.pepperdine at gmail.com wrote: > I think that once you are subscribed you can email the list. I?m very much interested in observing but since I don?t have anything useful to add, I?d stay quiet.. but not everyone would. It?s a fine line to balance between stifling the discussion and not being distracted by endless discussions. Lets give Brian the benefit of his position on this one. > > Regards, > Kirk > >> On Dec 15, 2015, at 4:11 PM, Paul Benedict wrote: >> >> Brian, since you already know who is on the EG, why not grant write >> permissions to them only, and the rest allowed to be subscribale? Or does >> mailman not allow those types of controls? >> >> Cheers, >> Paul >> >> On Tue, Dec 15, 2015 at 9:08 AM, Brian Goetz wrote: >> >>> Several people tried to subscribe to valhalla-spec-experts, and were >>> surprised to get rejected. So let me clarify what's going on. >>> >>> The valhalla-spec-experts list is publicly READABLE, but not publicly >>> SUBSCRIBABLE -- the membership is limited. Now, lots of folks are >>> interested in subscribing not as members of the EG, but to get updates on a >>> timely basis without polling the archives, which is totally reasonable. >>> For lambda, we had a lambda-spec-observers list which received copies of >>> the traffic to lambda-spec-experts. We plan to do the same for valhalla, >>> we just haven't yet gone through the administrative work to do so (and I'd >>> rather make some technical progress this week than futz with mailing list >>> provisioning.) So it will take some time to get the -observers and >>> -comments lists up and running. So please be patient! >>> >>> >>> >>> On 12/14/2015 3:07 PM, Brian Goetz wrote: >>> >>>> It's time to take the next step in Valhalla's evolution. Accordingly, >>>> I've created the valhalla-spec-experts list, which will be the precursor to >>>> an eventual JCP Expert Group, and will operate under the JCP-friendly ToU. >>>> This list is publicly archived in the obvious place, and the initial >>>> membership includes both corporate participation (Oracle, IBM, Red Hat, >>>> Jetbrains, Google) and individuals from the OpenJDK community. >>>> >>>> As was the case when we formed the Lambda experts list, the role of >>>> valhalla-dev should now revert to its intended purpose, which is to discuss >>>> the *implementation* of Valhalla in OpenJDK. Examples of appropriate >>>> discussion topics for valhalla-dev include: >>>> >>>> - Bug reports >>>> - Code review comments >>>> - Test cases >>>> - Build or porting problems >>>> - Migration experiences for your own code >>>> >>>> What we are trying to avoid is: having design discussions play out >>>> concurrently in multiple places with differing sets of players. So please, >>>> we ask that you exercise restraint in commenting on active EG discussions >>>> -- as design decisions start to converge, we'll announce them here, and of >>>> course the primary place for implementation will remain the valhalla >>>> repositories. >>>> >>>> Please remember that EG discussions often happen at a different pace than >>>> in other open-source projects. Days can go by without a response, but this >>>> doesn't mean that the EG is not thinking about it, doesn't care, or is >>>> "stumped". Often it simply means the EG members are working privately (or >>>> with their teams) to figure out what their position is, and will respond >>>> when they've thought about it more. So, please let those discussions play >>>> out. >>>> >>>> >>>> From timo.kinnunen at gmail.com Tue Dec 15 16:55:38 2015 From: timo.kinnunen at gmail.com (Timo Kinnunen) Date: Tue, 15 Dec 2015 17:55:38 +0100 Subject: Thoughts on peeling and readability In-Reply-To: <566DC21A.8020004@oracle.com> References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> <566C576D.2020806@oracle.com> <566ca345.55b21c0a.b1d90.4516@mx.google.com> <39F28927-7C94-47B6-B8D4-EB69448E9917@oracle.com> <566db9d9.296dc20a.acd5f.fffff668@mx.google.com> <566DC21A.8020004@oracle.com> Message-ID: <56704611.4a5ec20a.af75e.ffffcce0@mx.google.com> Another use-case I had in mind is a more compact storage form when added into a collection. An ArrayList permitting all values of a value type of 32bits at the same time would use about 17 GBs of memory and require splitting the array, at least currently. A Set permitting the same would use about 540 MBs by using 1 bit per value to record set membership. Also, one final point I?d like to make about sentinels and generating them. While I can think of ways of using sentinels without making new ones myself, there?s a problem. I could decide to partition the values I am given into partition A and partition B. Then, when I need a new sentinel, I can use one value from partition B as a sentinel in partition A and vice versa. Neat! However, I may have just inadvertently used someone?s social security number plastering it all over the memory with no way of knowing there was PII contained in them. And if the program contains a data race, well, you can imagine the rest. What is needed is a way of generating values that are inert, structurally sound and semantically meaningless, possibly semi-random but otherwise plausible. Anything else just feels various degrees of wrong. Without that the fact that values are one level of indirection less far away than usual and that they are so neatly packaged sure is making sun.misc.Unsafe start to look really tempting! -- Have a nice day, Timo Sent from Mail for Windows 10 From: Brian Goetz Sent: Sunday, December 13, 2015 20:08 To: Timo Kinnunen Cc: Maurizio Cimadamore; Paul Benedict; valhalla-dev at openjdk.java.net Subject: Re: Thoughts on peeling and readability Primitives like int and long are "special" in that all bit patterns are valid and there are no integrity constraints that would prevent a client from requesting a specific bit pattern.? But this is the special case, not the general case.? A linguistic construct like T.default would have to work for *all* T, not just the special cases.? There's nothing to stop you from writing an implementation that takes advantage of knowledge of specific types like int; there's a range of options there.? What we're uninterested in doing is allowing clients to have unsafe bit-level access to the representation of all value types.? There's also nothing to stop you from writing value types that allow raw-bit operations; we're just not going to require that every value type support that (which is what asking for a bit-oriented T.default would be.)? T.default will almost certainly not go through a constructor; the VM will zero out the bits as it does with the existing built-in types (the eight primitive types plus references.)? This process is pretty obvious from both a specification and implementation perspective, but it does create some responsibility for writers of value types -- specifically the need to deal with the default bit pattern.? As our friends in the .NET community have discovered, trying to enforce that the no-arg ctor is always executed before a value is exposed is a game of whack-a-mole, so having T.default go through a constructor simply reduces the probability of the "implicit null" surprise, but doesn't banish it.? One can be as snarky as one likes about the tradeoffs ("reinvented null" vs "reinvented serialization"), but the reality here is that there are risks lurking around both corners.? Personally I like the tradeoff we're converging towards, but we are aware it is not perfect.? Stepping back, rather than arguing the merits or demerits of a particular solution (which at this point we've more than exhausted), it's far more helpful to talk about the problem instead of the solution.? So, let me ask -- what use cases are you concerned about, other then the manufacture of multiple sentinels for use inside data structure implementations?? (I suspect in the end you will find that you will be able to accomplish what you need with the tools available, but the "let me specify the bit pattern for an arbitrary value type" approach is not the way to get there.) On 12/13/2015 1:32 PM, Timo Kinnunen wrote: Well, it?s ints and longs, and all primitive types copyable around as ints and longs, and all objects serializable to and from arrays of ints and longs, and all arrays of such, and all values made of such, and all arrays of such values, and all values made of such values, aaand I?m probably missing a dimension or two somewhere. ? These values are just as valid regardless of which bit patterns, all-zero or not, were used to construct them. They are safe to be copied around and, if you implemented them yourself, hashCode, equals, toString and any component-wise operations could also be done safely. Such operations simply can?t call any foreign code of any of the value or reference types involved. We don?t expect that we can take an arbitrary Object, use reflection to zero out its fields and then be able to call its instance methods like nothing had happened either. So, for reference types these operations would have to done using reflection, for value types VarHandles might give better performance. ? Ultimately I guess it all depends on whether T.default invokes some constructor or not. If it doesn?t or if the constructor is specified to always succeed trivially then we have reinvented null. Our new nulls trade off a large number of NPEs for silent invariant violations. This could be a good tradeoff but without knowing about the consequences of the violations beforehand it?s gonna be hard to say for certain. The good news is that hey, null is back! ? If T.default executes a constructor that can refuse an all-zero bit pattern, then we have reinvented serialization for value types and are requiring all value types support it. Our serialization protocol only recognizes one input value and can only deserialize, so it?s a bit useless. But with the addition of the missing serialize-function we can then define transforms for long[] <-> val <-> long[] and will have a quite general and capable system already. ? Or are we gonna just include the worst parts from both? ? -- Have a nice day, Timo Sent from Mail for Windows 10 ? ? From: Brian Goetz Sent: Sunday, December 13, 2015 01:20 To: Timo Kinnunen Cc: Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net Subject: Re: Thoughts on peeling and readability ? ? No. ?In general, to/from raw bit operations are not safe except in a few corner cases (like int and long.) ?Values are not uncontrolled buckets of bits. ? ? On the other hand, T.default *is* safe, because every type has a default bit pattern which is the initialization of any otherwise uninitialized field or array element. ?(It so happens that this default bit pattern corresponds to all zero bits for all types, though this is mostly a convenience for VM implementors.) ?For a composite value, the default value is comprised of the default value for all fields. ?By *definition*, the all-zero bit pattern is a valid element of all value types. ?However, there is no guarantee that any other bit pattern is valid for any given value type. ? ? If a particular value type wants to expose to/from raw bit constructors, that?s fine ? but you?re asking for a language feature that applies to *all* values ? and there is no guarantee that this is a safe operation for all values. ? ? On Dec 12, 2015, at 5:44 PM, Timo Kinnunen wrote: Field layout and bit fiddling isn?t exactly what I was thinking. Rather I was thinking something like Float.floatToRawIntBits() and Double.doubleToRawLongBits(), but without having to know about the types Float and Double or how many bits are in their raw bits. So something like this syntax: ? ??????????????? static T nextUp (T value) { ??????????????? ??????????????? rawBits = T.toRawBits(value); ??????????????? ??????????????? T nextValue = T.fromRawBits(rawBits + 1); ??????????????????????????????? return nextValue; ??????????????? } ? This should fit in Valhalla reasonably well, as it is just a generalization of T.default with its complement operation included. And as it is, all of the problems you listed already apply to T.default. For example, a value type with one long field: If the long value in the field is a handle pointing to a memory-mapped buffer then any use of a default value of such a type could cause a crash. Which can include asking a properly constructed value if it is equal to any of the values in an array you have. --? Have a nice day,? Timo Sent from Mail for Windows 10 ? ? From:?Brian Goetz Sent:?Saturday, December 12, 2015 18:21 To:?Timo Kinnunen;Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net Subject:?Re: Thoughts on peeling and readability ? ? Precise layout and bit control of values are anti-goals of Valhalla, so we're not really exploring this direction at this time. ? The problem with approaches like the one you suggest is they fall apart as soon as you leave the realm of "primitives modeled as values."? What about values that have refs in them?? What about values whose representations are private?? Their implementation is supposed to be in sole control of their representation.? This runs contrary to the "codes like a class" dictum. ? ? ? ? ? ? On 12/12/2015 4:43 AM, Timo Kinnunen wrote: > Hi, >? > One thing that I don?t remember seeing is any syntax for constructing arbitrary values in generic code without having to know about the precise field layouts and what the meaning of such fields is. Something like T.default but for values other than 0. Perhaps T.default(12345) or some such? >? > Or maybe this is slated to go with bytecode type specialization? What sort of syntax is envisioned to be driving that anyways? >? >? >? >? >? ? ? ? From palo.marton at gmail.com Wed Dec 16 12:38:49 2015 From: palo.marton at gmail.com (Palo Marton) Date: Wed, 16 Dec 2015 13:38:49 +0100 Subject: Thoughts on peeling and readability In-Reply-To: <56704611.4a5ec20a.af75e.ffffcce0@mx.google.com> References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> <566C576D.2020806@oracle.com> <566ca345.55b21c0a.b1d90.4516@mx.google.com> <39F28927-7C94-47B6-B8D4-EB69448E9917@oracle.com> <566db9d9.296dc20a.acd5f.fffff668@mx.google.com> <566DC21A.8020004@oracle.com> <56704611.4a5ec20a.af75e.ffffcce0@mx.google.com> Message-ID: > > What is needed is a way of generating values that are inert, structurally > sound and semantically meaningless, possibly semi-random but otherwise > plausible. Most value types will probably expose some if its internal fields as public (also with "setter" method), so it should be possible to use (future VT) reflection to create your sentinel value. E.g. take T.default and fill first available public field with sentinel value for that field. > Anything else just feels various degrees of wrong. Without that the fact > that values are one level of indirection less far away than usual and that > they are so neatly packaged sure is making sun.misc.Unsafe start to look > really tempting! > > > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > From: Brian Goetz > Sent: Sunday, December 13, 2015 20:08 > To: Timo Kinnunen > Cc: Maurizio Cimadamore; Paul Benedict; valhalla-dev at openjdk.java.net > Subject: Re: Thoughts on peeling and readability > > Primitives like int and long are "special" in that all bit patterns are > valid and there are no integrity constraints that would prevent a client > from requesting a specific bit pattern. But this is the special case, not > the general case. A linguistic construct like T.default would have to work > for *all* T, not just the special cases. > > There's nothing to stop you from writing an implementation that takes > advantage of knowledge of specific types like int; there's a range of > options there. What we're uninterested in doing is allowing clients to > have unsafe bit-level access to the representation of all value types. > There's also nothing to stop you from writing value types that allow > raw-bit operations; we're just not going to require that every value type > support that (which is what asking for a bit-oriented T.default would be.) > > T.default will almost certainly not go through a constructor; the VM will > zero out the bits as it does with the existing built-in types (the eight > primitive types plus references.) This process is pretty obvious from both > a specification and implementation perspective, but it does create some > responsibility for writers of value types -- specifically the need to deal > with the default bit pattern. > > As our friends in the .NET community have discovered, trying to enforce > that the no-arg ctor is always executed before a value is exposed is a game > of whack-a-mole, so having T.default go through a constructor simply > reduces the probability of the "implicit null" surprise, but doesn't banish > it. One can be as snarky as one likes about the tradeoffs ("reinvented > null" vs "reinvented serialization"), but the reality here is that there > are risks lurking around both corners. Personally I like the tradeoff > we're converging towards, but we are aware it is not perfect. > > > Stepping back, rather than arguing the merits or demerits of a particular > solution (which at this point we've more than exhausted), it's far more > helpful to talk about the problem instead of the solution. So, let me ask > -- what use cases are you concerned about, other then the manufacture of > multiple sentinels for use inside data structure implementations? > > (I suspect in the end you will find that you will be able to accomplish > what you need with the tools available, but the "let me specify the bit > pattern for an arbitrary value type" approach is not the way to get there.) > > On 12/13/2015 1:32 PM, Timo Kinnunen wrote: > Well, it?s ints and longs, and all primitive types copyable around as ints > and longs, and all objects serializable to and from arrays of ints and > longs, and all arrays of such, and all values made of such, and all arrays > of such values, and all values made of such values, aaand I?m probably > missing a dimension or two somewhere. > > These values are just as valid regardless of which bit patterns, all-zero > or not, were used to construct them. They are safe to be copied around and, > if you implemented them yourself, hashCode, equals, toString and any > component-wise operations could also be done safely. Such operations simply > can?t call any foreign code of any of the value or reference types > involved. We don?t expect that we can take an arbitrary Object, use > reflection to zero out its fields and then be able to call its instance > methods like nothing had happened either. So, for reference types these > operations would have to done using reflection, for value types VarHandles > might give better performance. > > Ultimately I guess it all depends on whether T.default invokes some > constructor or not. If it doesn?t or if the constructor is specified to > always succeed trivially then we have reinvented null. Our new nulls trade > off a large number of NPEs for silent invariant violations. This could be a > good tradeoff but without knowing about the consequences of the violations > beforehand it?s gonna be hard to say for certain. The good news is that > hey, null is back! > > If T.default executes a constructor that can refuse an all-zero bit > pattern, then we have reinvented serialization for value types and are > requiring all value types support it. Our serialization protocol only > recognizes one input value and can only deserialize, so it?s a bit useless. > But with the addition of the missing serialize-function we can then define > transforms for long[] <-> val <-> long[] and will have a quite general and > capable system already. > > Or are we gonna just include the worst parts from both? > > > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > > From: Brian Goetz > Sent: Sunday, December 13, 2015 01:20 > To: Timo Kinnunen > Cc: Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net > Subject: Re: Thoughts on peeling and readability > > > No. In general, to/from raw bit operations are not safe except in a few > corner cases (like int and long.) Values are not uncontrolled buckets of > bits. > > On the other hand, T.default *is* safe, because every type has a default > bit pattern which is the initialization of any otherwise uninitialized > field or array element. (It so happens that this default bit pattern > corresponds to all zero bits for all types, though this is mostly a > convenience for VM implementors.) For a composite value, the default value > is comprised of the default value for all fields. By *definition*, the > all-zero bit pattern is a valid element of all value types. However, there > is no guarantee that any other bit pattern is valid for any given value > type. > > If a particular value type wants to expose to/from raw bit constructors, > that?s fine ? but you?re asking for a language feature that applies to > *all* values ? and there is no guarantee that this is a safe operation for > all values. > > On Dec 12, 2015, at 5:44 PM, Timo Kinnunen > wrote: > > > > Field layout and bit fiddling isn?t exactly what I was thinking. Rather I > was thinking something like Float.floatToRawIntBits() and > Double.doubleToRawLongBits(), but without having to know about the types > Float and Double or how many bits are in their raw bits. So something like > this syntax: > > static T nextUp (T value) { > rawBits = > T.toRawBits(value); > T nextValue = T.fromRawBits(rawBits + 1); > return nextValue; > } > > This should fit in Valhalla reasonably well, as it is just a > generalization of T.default with its complement operation included. And as > it is, all of the problems you listed already apply to T.default. For > example, a value type with one long field: If the long value in the field > is a handle pointing to a memory-mapped buffer then any use of a default > value of such a type could cause a crash. Which can include asking a > properly constructed value if it is equal to any of the values in an array > you have. > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > > From: Brian Goetz > Sent: Saturday, December 12, 2015 18:21 > To: Timo Kinnunen;Maurizio Cimadamore;Paul > Benedict;valhalla-dev at openjdk.java.net > Subject: Re: Thoughts on peeling and readability > > > Precise layout and bit control of values are anti-goals of Valhalla, so > we're not really exploring this direction at this time. > > The problem with approaches like the one you suggest is they fall apart > as soon as you leave the realm of "primitives modeled as values." What > about values that have refs in them? What about values whose > representations are private? Their implementation is supposed to be in > sole control of their representation. This runs contrary to the "codes > like a class" dictum. > > > > > > > On 12/12/2015 4:43 AM, Timo Kinnunen wrote: > > Hi, > > > > One thing that I don?t remember seeing is any syntax for constructing > arbitrary values in generic code without having to know about the precise > field layouts and what the meaning of such fields is. Something like > T.default but for values other than 0. Perhaps T.default(12345) or some > such? > > > > Or maybe this is slated to go with bytecode type specialization? What > sort of syntax is envisioned to be driving that anyways? > > > > > > > > > > > > > > > > From vitalyd at gmail.com Wed Dec 16 13:42:40 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 16 Dec 2015 08:42:40 -0500 Subject: Thoughts on peeling and readability In-Reply-To: <56704611.4a5ec20a.af75e.ffffcce0@mx.google.com> References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> <566C576D.2020806@oracle.com> <566ca345.55b21c0a.b1d90.4516@mx.google.com> <39F28927-7C94-47B6-B8D4-EB69448E9917@oracle.com> <566db9d9.296dc20a.acd5f.fffff668@mx.google.com> <566DC21A.8020004@oracle.com> <56704611.4a5ec20a.af75e.ffffcce0@mx.google.com> Message-ID: If you're rolling your own data structure that needs a sentinel, why not pass a factory in the constructor that can instantiate a T from some parameter (e.g. int) and use that to create the sentinels? This is assuming no new generic constraint capability that allows constraining the T to having a constructor of some shape. I don't think we want to go the reflection or Unsafe route here. On Tuesday, December 15, 2015, Timo Kinnunen wrote: > Another use-case I had in mind is a more compact storage form when added > into a collection. An ArrayList permitting all values of a value type of > 32bits at the same time would use about 17 GBs of memory and require > splitting the array, at least currently. A Set permitting the same would > use about 540 MBs by using 1 bit per value to record set membership. > > Also, one final point I?d like to make about sentinels and generating > them. While I can think of ways of using sentinels without making new ones > myself, there?s a problem. I could decide to partition the values I am > given into partition A and partition B. Then, when I need a new sentinel, I > can use one value from partition B as a sentinel in partition A and vice > versa. Neat! However, I may have just inadvertently used someone?s social > security number plastering it all over the memory with no way of knowing > there was PII contained in them. And if the program contains a data race, > well, you can imagine the rest. > > What is needed is a way of generating values that are inert, structurally > sound and semantically meaningless, possibly semi-random but otherwise > plausible. Anything else just feels various degrees of wrong. Without that > the fact that values are one level of indirection less far away than usual > and that they are so neatly packaged sure is making sun.misc.Unsafe start > to look really tempting! > > > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > From: Brian Goetz > Sent: Sunday, December 13, 2015 20:08 > To: Timo Kinnunen > Cc: Maurizio Cimadamore; Paul Benedict; valhalla-dev at openjdk.java.net > > Subject: Re: Thoughts on peeling and readability > > Primitives like int and long are "special" in that all bit patterns are > valid and there are no integrity constraints that would prevent a client > from requesting a specific bit pattern. But this is the special case, not > the general case. A linguistic construct like T.default would have to work > for *all* T, not just the special cases. > > There's nothing to stop you from writing an implementation that takes > advantage of knowledge of specific types like int; there's a range of > options there. What we're uninterested in doing is allowing clients to > have unsafe bit-level access to the representation of all value types. > There's also nothing to stop you from writing value types that allow > raw-bit operations; we're just not going to require that every value type > support that (which is what asking for a bit-oriented T.default would be.) > > T.default will almost certainly not go through a constructor; the VM will > zero out the bits as it does with the existing built-in types (the eight > primitive types plus references.) This process is pretty obvious from both > a specification and implementation perspective, but it does create some > responsibility for writers of value types -- specifically the need to deal > with the default bit pattern. > > As our friends in the .NET community have discovered, trying to enforce > that the no-arg ctor is always executed before a value is exposed is a game > of whack-a-mole, so having T.default go through a constructor simply > reduces the probability of the "implicit null" surprise, but doesn't banish > it. One can be as snarky as one likes about the tradeoffs ("reinvented > null" vs "reinvented serialization"), but the reality here is that there > are risks lurking around both corners. Personally I like the tradeoff > we're converging towards, but we are aware it is not perfect. > > > Stepping back, rather than arguing the merits or demerits of a particular > solution (which at this point we've more than exhausted), it's far more > helpful to talk about the problem instead of the solution. So, let me ask > -- what use cases are you concerned about, other then the manufacture of > multiple sentinels for use inside data structure implementations? > > (I suspect in the end you will find that you will be able to accomplish > what you need with the tools available, but the "let me specify the bit > pattern for an arbitrary value type" approach is not the way to get there.) > > On 12/13/2015 1:32 PM, Timo Kinnunen wrote: > Well, it?s ints and longs, and all primitive types copyable around as ints > and longs, and all objects serializable to and from arrays of ints and > longs, and all arrays of such, and all values made of such, and all arrays > of such values, and all values made of such values, aaand I?m probably > missing a dimension or two somewhere. > > These values are just as valid regardless of which bit patterns, all-zero > or not, were used to construct them. They are safe to be copied around and, > if you implemented them yourself, hashCode, equals, toString and any > component-wise operations could also be done safely. Such operations simply > can?t call any foreign code of any of the value or reference types > involved. We don?t expect that we can take an arbitrary Object, use > reflection to zero out its fields and then be able to call its instance > methods like nothing had happened either. So, for reference types these > operations would have to done using reflection, for value types VarHandles > might give better performance. > > Ultimately I guess it all depends on whether T.default invokes some > constructor or not. If it doesn?t or if the constructor is specified to > always succeed trivially then we have reinvented null. Our new nulls trade > off a large number of NPEs for silent invariant violations. This could be a > good tradeoff but without knowing about the consequences of the violations > beforehand it?s gonna be hard to say for certain. The good news is that > hey, null is back! > > If T.default executes a constructor that can refuse an all-zero bit > pattern, then we have reinvented serialization for value types and are > requiring all value types support it. Our serialization protocol only > recognizes one input value and can only deserialize, so it?s a bit useless. > But with the addition of the missing serialize-function we can then define > transforms for long[] <-> val <-> long[] and will have a quite general and > capable system already. > > Or are we gonna just include the worst parts from both? > > > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > > From: Brian Goetz > Sent: Sunday, December 13, 2015 01:20 > To: Timo Kinnunen > Cc: Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net > > Subject: Re: Thoughts on peeling and readability > > > No. In general, to/from raw bit operations are not safe except in a few > corner cases (like int and long.) Values are not uncontrolled buckets of > bits. > > On the other hand, T.default *is* safe, because every type has a default > bit pattern which is the initialization of any otherwise uninitialized > field or array element. (It so happens that this default bit pattern > corresponds to all zero bits for all types, though this is mostly a > convenience for VM implementors.) For a composite value, the default value > is comprised of the default value for all fields. By *definition*, the > all-zero bit pattern is a valid element of all value types. However, there > is no guarantee that any other bit pattern is valid for any given value > type. > > If a particular value type wants to expose to/from raw bit constructors, > that?s fine ? but you?re asking for a language feature that applies to > *all* values ? and there is no guarantee that this is a safe operation for > all values. > > On Dec 12, 2015, at 5:44 PM, Timo Kinnunen > wrote: > > > > Field layout and bit fiddling isn?t exactly what I was thinking. Rather I > was thinking something like Float.floatToRawIntBits() and > Double.doubleToRawLongBits(), but without having to know about the types > Float and Double or how many bits are in their raw bits. So something like > this syntax: > > static T nextUp (T value) { > rawBits = > T.toRawBits(value); > T nextValue = T.fromRawBits(rawBits + 1); > return nextValue; > } > > This should fit in Valhalla reasonably well, as it is just a > generalization of T.default with its complement operation included. And as > it is, all of the problems you listed already apply to T.default. For > example, a value type with one long field: If the long value in the field > is a handle pointing to a memory-mapped buffer then any use of a default > value of such a type could cause a crash. Which can include asking a > properly constructed value if it is equal to any of the values in an array > you have. > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > > From: Brian Goetz > Sent: Saturday, December 12, 2015 18:21 > To: Timo Kinnunen;Maurizio Cimadamore;Paul > Benedict;valhalla-dev at openjdk.java.net > Subject: Re: Thoughts on peeling and readability > > > Precise layout and bit control of values are anti-goals of Valhalla, so > we're not really exploring this direction at this time. > > The problem with approaches like the one you suggest is they fall apart > as soon as you leave the realm of "primitives modeled as values." What > about values that have refs in them? What about values whose > representations are private? Their implementation is supposed to be in > sole control of their representation. This runs contrary to the "codes > like a class" dictum. > > > > > > > On 12/12/2015 4:43 AM, Timo Kinnunen wrote: > > Hi, > > > > One thing that I don?t remember seeing is any syntax for constructing > arbitrary values in generic code without having to know about the precise > field layouts and what the meaning of such fields is. Something like > T.default but for values other than 0. Perhaps T.default(12345) or some > such? > > > > Or maybe this is slated to go with bytecode type specialization? What > sort of syntax is envisioned to be driving that anyways? > > > > > > > > > > > > > > > > -- Sent from my phone From timo.kinnunen at gmail.com Wed Dec 16 14:59:40 2015 From: timo.kinnunen at gmail.com (Timo Kinnunen) Date: Wed, 16 Dec 2015 15:59:40 +0100 Subject: Thoughts on peeling and readability In-Reply-To: References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> <566C576D.2020806@oracle.com> <566ca345.55b21c0a.b1d90.4516@mx.google.com> <39F28927-7C94-47B6-B8D4-EB69448E9917@oracle.com> <566db9d9.296dc20a.acd5f.fffff668@mx.google.com> <566DC21A.8020004@oracle.com> <56704611.4a5ec20a.af75e.ffffcce0@mx.google.com> Message-ID: <56717c5c.89dec20a.7b97c.4dcc@mx.google.com> One reason is that requiring an additional constructor ripples all over the API, making it more cumbersome to use. For example, imagine an Arrays.asList() implementation that required passing in a constructor in addition to passing in the elements. This adds a lot of weight to an otherwise lightweight API. Another reason is that then the code depends on an external party to generate good sentinels for its internal use. What is a ?good sentinel value? is an implementation detail that might have to be documented. As the client is only concerned with generating real values for its own uses and the implementation cares about sentinels, an API where these are mixed goes against the separation of concerns design principle. And I agree, we don?t want to go with Unsafe or reflection, but we may not be able to avoid it. -- Have a nice day, Timo Sent from Mail for Windows 10 From: Vitaly Davidovich Sent: Wednesday, December 16, 2015 14:42 To: Timo Kinnunen Cc: Brian Goetz; valhalla-dev at openjdk.java.net Subject: Re: Thoughts on peeling and readability If you're rolling your own data structure that needs a sentinel, why not pass?a factory in the constructor that can instantiate a T from some parameter (e.g. int) and use that to create the sentinels? This is assuming no new generic constraint capability that allows constraining the T to having a constructor of some shape. I don't think we want to go the reflection or Unsafe route here. On Tuesday, December 15, 2015, Timo Kinnunen wrote: Another use-case I had in mind is a more compact storage form when added into a collection. An ArrayList permitting all values of a value type of 32bits at the same time would use about 17 GBs of memory and require splitting the array, at least currently. A Set permitting the same would use about 540 MBs by using 1 bit per value to record set membership. Also, one final point I?d like to make about sentinels and generating them. While I can think of ways of using sentinels without making new ones myself, there?s a problem. I could decide to partition the values I am given into partition A and partition B. Then, when I need a new sentinel, I can use one value from partition B as a sentinel in partition A and vice versa. Neat! However, I may have just inadvertently used someone?s social security number plastering it all over the memory with no way of knowing there was PII contained in them. And if the program contains a data race, well, you can imagine the rest. What is needed is a way of generating values that are inert, structurally sound and semantically meaningless, possibly semi-random but otherwise plausible. Anything else just feels various degrees of wrong. Without that the fact that values are one level of indirection less far away than usual and that they are so neatly packaged sure is making sun.misc.Unsafe start to look really tempting! -- Have a nice day, Timo Sent from Mail for Windows 10 From: Brian Goetz Sent: Sunday, December 13, 2015 20:08 To: Timo Kinnunen Cc: Maurizio Cimadamore; Paul Benedict; valhalla-dev at openjdk.java.net Subject: Re: Thoughts on peeling and readability Primitives like int and long are "special" in that all bit patterns are valid and there are no integrity constraints that would prevent a client from requesting a specific bit pattern.? But this is the special case, not the general case.? A linguistic construct like T.default would have to work for *all* T, not just the special cases.? There's nothing to stop you from writing an implementation that takes advantage of knowledge of specific types like int; there's a range of options there.? What we're uninterested in doing is allowing clients to have unsafe bit-level access to the representation of all value types.? There's also nothing to stop you from writing value types that allow raw-bit operations; we're just not going to require that every value type support that (which is what asking for a bit-oriented T.default would be.)? T.default will almost certainly not go through a constructor; the VM will zero out the bits as it does with the existing built-in types (the eight primitive types plus references.)? This process is pretty obvious from both a specification and implementation perspective, but it does create some responsibility for writers of value types -- specifically the need to deal with the default bit pattern.? As our friends in the .NET community have discovered, trying to enforce that the no-arg ctor is always executed before a value is exposed is a game of whack-a-mole, so having T.default go through a constructor simply reduces the probability of the "implicit null" surprise, but doesn't banish it.? One can be as snarky as one likes about the tradeoffs ("reinvented null" vs "reinvented serialization"), but the reality here is that there are risks lurking around both corners.? Personally I like the tradeoff we're converging towards, but we are aware it is not perfect.? Stepping back, rather than arguing the merits or demerits of a particular solution (which at this point we've more than exhausted), it's far more helpful to talk about the problem instead of the solution.? So, let me ask -- what use cases are you concerned about, other then the manufacture of multiple sentinels for use inside data structure implementations?? (I suspect in the end you will find that you will be able to accomplish what you need with the tools available, but the "let me specify the bit pattern for an arbitrary value type" approach is not the way to get there.) On 12/13/2015 1:32 PM, Timo Kinnunen wrote: Well, it?s ints and longs, and all primitive types copyable around as ints and longs, and all objects serializable to and from arrays of ints and longs, and all arrays of such, and all values made of such, and all arrays of such values, and all values made of such values, aaand I?m probably missing a dimension or two somewhere. ? These values are just as valid regardless of which bit patterns, all-zero or not, were used to construct them. They are safe to be copied around and, if you implemented them yourself, hashCode, equals, toString and any component-wise operations could also be done safely. Such operations simply can?t call any foreign code of any of the value or reference types involved. We don?t expect that we can take an arbitrary Object, use reflection to zero out its fields and then be able to call its instance methods like nothing had happened either. So, for reference types these operations would have to done using reflection, for value types VarHandles might give better performance. ? Ultimately I guess it all depends on whether T.default invokes some constructor or not. If it doesn?t or if the constructor is specified to always succeed trivially then we have reinvented null. Our new nulls trade off a large number of NPEs for silent invariant violations. This could be a good tradeoff but without knowing about the consequences of the violations beforehand it?s gonna be hard to say for certain. The good news is that hey, null is back! ? If T.default executes a constructor that can refuse an all-zero bit pattern, then we have reinvented serialization for value types and are requiring all value types support it. Our serialization protocol only recognizes one input value and can only deserialize, so it?s a bit useless. But with the addition of the missing serialize-function we can then define transforms for long[] <-> val <-> long[] and will have a quite general and capable system already. ? Or are we gonna just include the worst parts from both? ? -- Have a nice day, Timo Sent from Mail for Windows 10 ? ? From: Brian Goetz Sent: Sunday, December 13, 2015 01:20 To: Timo Kinnunen Cc: Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net Subject: Re: Thoughts on peeling and readability ? ? No.? In general, to/from raw bit operations are not safe except in a few corner cases (like int and long.) ?Values are not uncontrolled buckets of bits. ? ? On the other hand, T.default *is* safe, because every type has a default bit pattern which is the initialization of any otherwise uninitialized field or array element. ?(It so happens that this default bit pattern corresponds to all zero bits for all types, though this is mostly a convenience for VM implementors.) ?For a composite value, the default value is comprised of the default value for all fields.? By *definition*, the all-zero bit pattern is a valid element of all value types.? However, there is no guarantee that any other bit pattern is valid for any given value type. ? ? If a particular value type wants to expose to/from raw bit constructors, that?s fine ? but you?re asking for a language feature that applies to *all* values ? and there is no guarantee that this is a safe operation for all values. ? ? On Dec 12, 2015, at 5:44 PM, Timo Kinnunen wrote: Field layout and bit fiddling isn?t exactly what I was thinking. Rather I was thinking something like Float.floatToRawIntBits() and Double.doubleToRawLongBits(), but without having to know about the types Float and Double or how many bits are in their raw bits. So something like this syntax: ? ??????????????? static T nextUp (T value) { ??????????????? ??????????????? rawBits = T.toRawBits(value); ??????????????? ??????????????? T nextValue = T.fromRawBits(rawBits + 1); ??????????????????????????????? return nextValue; ??????????????? } ? This should fit in Valhalla reasonably well, as it is just a generalization of T.default with its complement operation included. And as it is, all of the problems you listed already apply to T.default. For example, a value type with one long field: If the long value in the field is a handle pointing to a memory-mapped buffer then any use of a default value of such a type could cause a crash. Which can include asking a properly constructed value if it is equal to any of the values in an array you have. --? Have a nice day,? Timo Sent from Mail for Windows 10 ? ? From:?Brian Goetz Sent:?Saturday, December 12, 2015 18:21 To:?Timo Kinnunen;Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net Subject:?Re: Thoughts on peeling and readability ? ? Precise layout and bit control of values are anti-goals of Valhalla, so we're not really exploring this direction at this time. ? The problem with approaches like the one you suggest is they fall apart as soon as you leave the realm of "primitives modeled as values."? What about values that have refs in them?? What about values whose representations are private?? Their implementation is supposed to be in sole control of their representation.? This runs contrary to the "codes like a class" dictum. ? ? ? ? ? ? On 12/12/2015 4:43 AM, Timo Kinnunen wrote: > Hi, >? > One thing that I don?t remember seeing is any syntax for constructing arbitrary values in generic code without having to know about the precise field layouts and what the meaning of such fields is. Something like T.default but for values other than 0. Perhaps T.default(12345) or some such? >? > Or maybe this is slated to go with bytecode type specialization? What sort of syntax is envisioned to be driving that anyways? >? >? >? >? >? ? ? ? -- Sent from my phone From vitalyd at gmail.com Wed Dec 16 15:08:13 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 16 Dec 2015 10:08:13 -0500 Subject: Thoughts on peeling and readability In-Reply-To: <56717c5c.89dec20a.7b97c.4dcc@mx.google.com> References: <566AAAE3.7090906@oracle.com> <566bec46.aa0bc30a.364ab.fffffbb2@mx.google.com> <566C576D.2020806@oracle.com> <566ca345.55b21c0a.b1d90.4516@mx.google.com> <39F28927-7C94-47B6-B8D4-EB69448E9917@oracle.com> <566db9d9.296dc20a.acd5f.fffff668@mx.google.com> <566DC21A.8020004@oracle.com> <56704611.4a5ec20a.af75e.ffffcce0@mx.google.com> <56717c5c.89dec20a.7b97c.4dcc@mx.google.com> Message-ID: On Wednesday, December 16, 2015, Timo Kinnunen wrote: > One reason is that requiring an additional constructor ripples all over > the API, making it more cumbersome to use. For example, imagine an > Arrays.asList() implementation that required passing in a constructor in > addition to passing in the elements. This adds a lot of weight to an > otherwise lightweight API. > You'd only need such a constraint where you generic type requires constructing the objects. Specific to this example, only where sentinels are used (not too common). > > > Another reason is that then the code depends on an external party to > generate good sentinels for its internal use. What is a ?good sentinel > value? is an implementation detail that might have to be documented. As the > client is only concerned with generating real values for its own uses and > the implementation cares about sentinels, an API where these are mixed goes > against the separation of concerns design principle. > Yes it does "leak" into the client. But, the client/caller "owns" the concrete type and so if you want to somehow piggyback on it for sentinel use, getting the caller involved in figuring that out makes some sense. Scribbling some random bits over their type seems worse. > > > And I agree, we don?t want to go with Unsafe or reflection, but we may not > be able to avoid it. > > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > > > *From: *Vitaly Davidovich > > *Sent: *Wednesday, December 16, 2015 14:42 > *To: *Timo Kinnunen > > *Cc: *Brian Goetz ; > valhalla-dev at openjdk.java.net > > *Subject: *Re: Thoughts on peeling and readability > > > > If you're rolling your own data structure that needs a sentinel, why not > pass a factory in the constructor that can instantiate a T from some > parameter (e.g. int) and use that to create the sentinels? This is assuming > no new generic constraint capability that allows constraining the T to > having a constructor of some shape. > > > > I don't think we want to go the reflection or Unsafe route here. > > On Tuesday, December 15, 2015, Timo Kinnunen > wrote: > > Another use-case I had in mind is a more compact storage form when added > into a collection. An ArrayList permitting all values of a value type of > 32bits at the same time would use about 17 GBs of memory and require > splitting the array, at least currently. A Set permitting the same would > use about 540 MBs by using 1 bit per value to record set membership. > > Also, one final point I?d like to make about sentinels and generating > them. While I can think of ways of using sentinels without making new ones > myself, there?s a problem. I could decide to partition the values I am > given into partition A and partition B. Then, when I need a new sentinel, I > can use one value from partition B as a sentinel in partition A and vice > versa. Neat! However, I may have just inadvertently used someone?s social > security number plastering it all over the memory with no way of knowing > there was PII contained in them. And if the program contains a data race, > well, you can imagine the rest. > > What is needed is a way of generating values that are inert, structurally > sound and semantically meaningless, possibly semi-random but otherwise > plausible. Anything else just feels various degrees of wrong. Without that > the fact that values are one level of indirection less far away than usual > and that they are so neatly packaged sure is making sun.misc.Unsafe start > to look really tempting! > > > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > From: Brian Goetz > Sent: Sunday, December 13, 2015 20:08 > To: Timo Kinnunen > Cc: Maurizio Cimadamore; Paul Benedict; valhalla-dev at openjdk.java.net > Subject: Re: Thoughts on peeling and readability > > Primitives like int and long are "special" in that all bit patterns are > valid and there are no integrity constraints that would prevent a client > from requesting a specific bit pattern. But this is the special case, not > the general case. A linguistic construct like T.default would have to work > for *all* T, not just the special cases. > > There's nothing to stop you from writing an implementation that takes > advantage of knowledge of specific types like int; there's a range of > options there. What we're uninterested in doing is allowing clients to > have unsafe bit-level access to the representation of all value types. > There's also nothing to stop you from writing value types that allow > raw-bit operations; we're just not going to require that every value type > support that (which is what asking for a bit-oriented T.default would be.) > > T.default will almost certainly not go through a constructor; the VM will > zero out the bits as it does with the existing built-in types (the eight > primitive types plus references.) This process is pretty obvious from both > a specification and implementation perspective, but it does create some > responsibility for writers of value types -- specifically the need to deal > with the default bit pattern. > > As our friends in the .NET community have discovered, trying to enforce > that the no-arg ctor is always executed before a value is exposed is a game > of whack-a-mole, so having T.default go through a constructor simply > reduces the probability of the "implicit null" surprise, but doesn't banish > it. One can be as snarky as one likes about the tradeoffs ("reinvented > null" vs "reinvented serialization"), but the reality here is that there > are risks lurking around both corners. Personally I like the tradeoff > we're converging towards, but we are aware it is not perfect. > > > Stepping back, rather than arguing the merits or demerits of a particular > solution (which at this point we've more than exhausted), it's far more > helpful to talk about the problem instead of the solution. So, let me ask > -- what use cases are you concerned about, other then the manufacture of > multiple sentinels for use inside data structure implementations? > > (I suspect in the end you will find that you will be able to accomplish > what you need with the tools available, but the "let me specify the bit > pattern for an arbitrary value type" approach is not the way to get there.) > > On 12/13/2015 1:32 PM, Timo Kinnunen wrote: > Well, it?s ints and longs, and all primitive types copyable around as ints > and longs, and all objects serializable to and from arrays of ints and > longs, and all arrays of such, and all values made of such, and all arrays > of such values, and all values made of such values, aaand I?m probably > missing a dimension or two somewhere. > > These values are just as valid regardless of which bit patterns, all-zero > or not, were used to construct them. They are safe to be copied around and, > if you implemented them yourself, hashCode, equals, toString and any > component-wise operations could also be done safely. Such operations simply > can?t call any foreign code of any of the value or reference types > involved. We don?t expect that we can take an arbitrary Object, use > reflection to zero out its fields and then be able to call its instance > methods like nothing had happened either. So, for reference types these > operations would have to done using reflection, for value types VarHandles > might give better performance. > > Ultimately I guess it all depends on whether T.default invokes some > constructor or not. If it doesn?t or if the constructor is specified to > always succeed trivially then we have reinvented null. Our new nulls trade > off a large number of NPEs for silent invariant violations. This could be a > good tradeoff but without knowing about the consequences of the violations > beforehand it?s gonna be hard to say for certain. The good news is that > hey, null is back! > > If T.default executes a constructor that can refuse an all-zero bit > pattern, then we have reinvented serialization for value types and are > requiring all value types support it. Our serialization protocol only > recognizes one input value and can only deserialize, so it?s a bit useless. > But with the addition of the missing serialize-function we can then define > transforms for long[] <-> val <-> long[] and will have a quite general and > capable system already. > > Or are we gonna just include the worst parts from both? > > > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > > From: Brian Goetz > Sent: Sunday, December 13, 2015 01:20 > To: Timo Kinnunen > Cc: Maurizio Cimadamore;Paul Benedict;valhalla-dev at openjdk.java.net > Subject: Re: Thoughts on peeling and readability > > > No. In general, to/from raw bit operations are not safe except in a few > corner cases (like int and long.) Values are not uncontrolled buckets of > bits. > > On the other hand, T.default *is* safe, because every type has a default > bit pattern which is the initialization of any otherwise uninitialized > field or array element. (It so happens that this default bit pattern > corresponds to all zero bits for all types, though this is mostly a > convenience for VM implementors.) For a composite value, the default value > is comprised of the default value for all fields. By *definition*, the > all-zero bit pattern is a valid element of all value types. However, there > is no guarantee that any other bit pattern is valid for any given value > type. > > If a particular value type wants to expose to/from raw bit constructors, > that?s fine ? but you?re asking for a language feature that applies to > *all* values ? and there is no guarantee that this is a safe operation for > all values. > > On Dec 12, 2015, at 5:44 PM, Timo Kinnunen > wrote: > > > > Field layout and bit fiddling isn?t exactly what I was thinking. Rather I > was thinking something like Float.floatToRawIntBits() and > Double.doubleToRawLongBits(), but without having to know about the types > Float and Double or how many bits are in their raw bits. So something like > this syntax: > > static T nextUp (T value) { > rawBits = > T.toRawBits(value); > T nextValue = T.fromRawBits(rawBits + 1); > return nextValue; > } > > This should fit in Valhalla reasonably well, as it is just a > generalization of T.default with its complement operation included. And as > it is, all of the problems you listed already apply to T.default. For > example, a value type with one long field: If the long value in the field > is a handle pointing to a memory-mapped buffer then any use of a default > value of such a type could cause a crash. Which can include asking a > properly constructed value if it is equal to any of the values in an array > you have. > > > > > -- > Have a nice day, > Timo > > Sent from Mail for Windows 10 > > > > From: Brian Goetz > Sent: Saturday, December 12, 2015 18:21 > To: Timo Kinnunen;Maurizio Cimadamore;Paul > Benedict;valhalla-dev at openjdk.java.net > Subject: Re: Thoughts on peeling and readability > > > Precise layout and bit control of values are anti-goals of Valhalla, so > we're not really exploring this direction at this time. > > The problem with approaches like the one you suggest is they fall apart > as soon as you leave the realm of "primitives modeled as values." What > about values that have refs in them? What about values whose > representations are private? Their implementation is supposed to be in > sole control of their representation. This runs contrary to the "codes > like a class" dictum. > > > > > > > On 12/12/2015 4:43 AM, Timo Kinnunen wrote: > > Hi, > > > > One thing that I don?t remember seeing is any syntax for constructing > arbitrary values in generic code without having to know about the precise > field layouts and what the meaning of such fields is. Something like > T.default but for values other than 0. Perhaps T.default(12345) or some > such? > > > > Or maybe this is slated to go with bytecode type specialization? What > sort of syntax is envisioned to be driving that anyways? > > > > > > > > > > > > > > > > > -- > Sent from my phone > > > -- Sent from my phone From pbenedict at apache.org Wed Dec 16 15:38:24 2015 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 16 Dec 2015 09:38:24 -0600 Subject: Any::isNull check Message-ID: When a method is specialized for values, Any::isNull can only return "false", which means if(Any.isNull(o)) can only ever have one outcome -- the other code branch can be discarded by the compiler. I know it's too early to talk about optimizations :-) but will javac be smart enough to recognize this optimization? Is it on your list for the future? Cheers, Paul From vitalyd at gmail.com Wed Dec 16 15:42:01 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 16 Dec 2015 10:42:01 -0500 Subject: Any::isNull check In-Reply-To: References: Message-ID: What's the use case of using Any.isNull when you don't know you're working with reference types? I'd expect comparison would be x.equals(T.default) in the generic context. On Wednesday, December 16, 2015, Paul Benedict wrote: > When a method is specialized for values, Any::isNull can only return > "false", which means if(Any.isNull(o)) can only ever have one outcome -- > the other code branch can be discarded by the compiler. I know it's too > early to talk about optimizations :-) but will javac be smart enough to > recognize this optimization? Is it on your list for the future? > > Cheers, > Paul > -- Sent from my phone From brian.goetz at oracle.com Wed Dec 16 15:42:14 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 16 Dec 2015 10:42:14 -0500 Subject: Any::isNull check In-Reply-To: References: Message-ID: <56718656.2040409@oracle.com> HotSpot is *really* good at eliminating dead code based on type profiles. On 12/16/2015 10:38 AM, Paul Benedict wrote: > When a method is specialized for values, Any::isNull can only return > "false", which means if(Any.isNull(o)) can only ever have one outcome -- > the other code branch can be discarded by the compiler. I know it's too > early to talk about optimizations :-) but will javac be smart enough to > recognize this optimization? Is it on your list for the future? > > Cheers, > Paul From vitalyd at gmail.com Wed Dec 16 15:44:37 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 16 Dec 2015 10:44:37 -0500 Subject: Any::isNull check In-Reply-To: <56718656.2040409@oracle.com> References: <56718656.2040409@oracle.com> Message-ID: One caveat here; if you don't eliminate dead code before JIT sees the method, it may be too big for inlining and suffer performance loss. On Wednesday, December 16, 2015, Brian Goetz wrote: > HotSpot is *really* good at eliminating dead code based on type profiles. > > On 12/16/2015 10:38 AM, Paul Benedict wrote: > >> When a method is specialized for values, Any::isNull can only return >> "false", which means if(Any.isNull(o)) can only ever have one outcome -- >> the other code branch can be discarded by the compiler. I know it's too >> early to talk about optimizations :-) but will javac be smart enough to >> recognize this optimization? Is it on your list for the future? >> >> Cheers, >> Paul >> > > -- Sent from my phone From pbenedict at apache.org Wed Dec 16 15:45:25 2015 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 16 Dec 2015 09:45:25 -0600 Subject: Any::isNull check In-Reply-To: <56718656.2040409@oracle.com> References: <56718656.2040409@oracle.com> Message-ID: Yes, HotSpot is really good at that. I agree. However, my point is there's no reason in even carrying around the dead bytecode in the class file. With lots of these if() checks, the class file can surely shrink if the compiler gets some additional smarts. Cheers, Paul On Wed, Dec 16, 2015 at 9:42 AM, Brian Goetz wrote: > HotSpot is *really* good at eliminating dead code based on type profiles. > > > On 12/16/2015 10:38 AM, Paul Benedict wrote: > >> When a method is specialized for values, Any::isNull can only return >> "false", which means if(Any.isNull(o)) can only ever have one outcome -- >> the other code branch can be discarded by the compiler. I know it's too >> early to talk about optimizations :-) but will javac be smart enough to >> recognize this optimization? Is it on your list for the future? >> >> Cheers, >> Paul >> > > From brian.goetz at oracle.com Wed Dec 16 15:46:48 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 16 Dec 2015 10:46:48 -0500 Subject: Any::isNull check In-Reply-To: References: Message-ID: <56718768.7090403@oracle.com> If you're working with an any-T, you don't know whether its a reference or value. So you still have to worry about NPE. But don't get too worked up about the particular expression of Any.isNull(). This is just the short-term, expedient hack to get the libraries port moving, its not a serious suggestion of how things will eventually look. On 12/16/2015 10:42 AM, Vitaly Davidovich wrote: > What's the use case of using Any.isNull when you don't know you're working > with reference types? I'd expect comparison would be x.equals(T.default) in > the generic context. > > On Wednesday, December 16, 2015, Paul Benedict wrote: > >> When a method is specialized for values, Any::isNull can only return >> "false", which means if(Any.isNull(o)) can only ever have one outcome -- >> the other code branch can be discarded by the compiler. I know it's too >> early to talk about optimizations :-) but will javac be smart enough to >> recognize this optimization? Is it on your list for the future? >> >> Cheers, >> Paul >> > From vitalyd at gmail.com Wed Dec 16 15:52:34 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 16 Dec 2015 10:52:34 -0500 Subject: Any::isNull check In-Reply-To: <56718768.7090403@oracle.com> References: <56718768.7090403@oracle.com> Message-ID: > > If you're working with an any-T, you don't know whether its a reference or > value. So you still have to worry about NPE. Wouldn't the "right" thing to do, then, be specialize the method in question for ref vs value types differently? You'd then also ensure the dead code for value types wouldn't be in the bytecode as well. But don't get too worked up about the particular expression of > Any.isNull(). This is just the short-term, expedient hack to get the > libraries port moving, its not a serious suggestion of how things will > eventually look. Yeah, I know this is just a temp thing, not hung up on the API just wondering about the use case in general. On Wed, Dec 16, 2015 at 10:46 AM, Brian Goetz wrote: > If you're working with an any-T, you don't know whether its a reference or > value. So you still have to worry about NPE. > > But don't get too worked up about the particular expression of > Any.isNull(). This is just the short-term, expedient hack to get the > libraries port moving, its not a serious suggestion of how things will > eventually look. > > > On 12/16/2015 10:42 AM, Vitaly Davidovich wrote: > >> What's the use case of using Any.isNull when you don't know you're working >> with reference types? I'd expect comparison would be x.equals(T.default) >> in >> the generic context. >> >> On Wednesday, December 16, 2015, Paul Benedict >> wrote: >> >> When a method is specialized for values, Any::isNull can only return >>> "false", which means if(Any.isNull(o)) can only ever have one outcome -- >>> the other code branch can be discarded by the compiler. I know it's too >>> early to talk about optimizations :-) but will javac be smart enough to >>> recognize this optimization? Is it on your list for the future? >>> >>> Cheers, >>> Paul >>> >>> >> > From brian.goetz at oracle.com Wed Dec 16 16:04:52 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 16 Dec 2015 11:04:52 -0500 Subject: Any::isNull check In-Reply-To: References: <56718768.7090403@oracle.com> Message-ID: <56718BA4.8040104@oracle.com> On 12/16/2015 10:52 AM, Vitaly Davidovich wrote: > > If you're working with an any-T, you don't know whether its a > reference or value. So you still have to worry about NPE. > > > Wouldn't the "right" thing to do, then, be specialize the method in > question for ref vs value types differently? You'd then also ensure > the dead code for value types wouldn't be in the bytecode as well. Depends what you mean by "right", and depends who is the implicit subject of the sentence. If you mean that the developers should write two versions of each method, one for nullable and one for not, that would be ridiculous. Duplicating N lines of code because one line of a method has to do a null check would be terrible. (Spend a half hour trying to port the Collections code with that approach, and ask yourself if you can honestly ask people to code that way.) If you mean that the static compiler should generate two separate versions based on a single source representation of the code, you are gaining the small optimization of early localized dead-code-elimination at the cost of more duplication. Maybe that's "right", maybe that's foolish. From vitalyd at gmail.com Wed Dec 16 16:31:02 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 16 Dec 2015 11:31:02 -0500 Subject: Any::isNull check In-Reply-To: <56718BA4.8040104@oracle.com> References: <56718768.7090403@oracle.com> <56718BA4.8040104@oracle.com> Message-ID: > > If you mean that the developers should write two versions of each method, > one for nullable and one for not, that would be ridiculous. Duplicating N > lines of code because one line of a method has to do a null check would be > terrible. (Spend a half hour trying to port the Collections code with that > approach, and ask yourself if you can honestly ask people to code that > way.) Not of each method. In Paul's case he made it sound like there's some substantial dead code to be eliminated in case of value types, making it sound like there's more than just one or two lines of difference. The other approach is to take the one or two lines in a bigger method that differ for value vs ref type and move just them into 2 specialized methods, one for ref and other for value types. I'd think that's clearer and cleaner than having one big method with large chunks of code applicable to just ref or value type. If you mean that the static compiler should generate two separate versions > based on a single source representation of the code, you are gaining the > small optimization of early localized dead-code-elimination at the cost of > more duplication. Maybe that's "right", maybe that's foolish. Given current limitations in inlining heuristics, removing dead bytecode outright before JIT sees it is a big win. Right now, lots of people end up splitting methods in awkward unnatural ways just to avoid hitting these issues. On Wed, Dec 16, 2015 at 11:04 AM, Brian Goetz wrote: > > > On 12/16/2015 10:52 AM, Vitaly Davidovich wrote: > > If you're working with an any-T, you don't know whether its a reference or >> value. So you still have to worry about NPE. > > > Wouldn't the "right" thing to do, then, be specialize the method in > question for ref vs value types differently? You'd then also ensure the > dead code for value types wouldn't be in the bytecode as well. > > > Depends what you mean by "right", and depends who is the implicit subject > of the sentence. > > If you mean that the developers should write two versions of each method, > one for nullable and one for not, that would be ridiculous. Duplicating N > lines of code because one line of a method has to do a null check would be > terrible. (Spend a half hour trying to port the Collections code with that > approach, and ask yourself if you can honestly ask people to code that > way.) > > If you mean that the static compiler should generate two separate versions > based on a single source representation of the code, you are gaining the > small optimization of early localized dead-code-elimination at the cost of > more duplication. Maybe that's "right", maybe that's foolish. > From forax at univ-mlv.fr Wed Dec 16 16:35:37 2015 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 16 Dec 2015 16:35:37 +0000 Subject: Any::isNull check In-Reply-To: References: <56718656.2040409@oracle.com> Message-ID: <0C34AAB7-04FE-4A97-BD0B-48F19989284A@univ-mlv.fr> The question is whenever you have several .class for one generic class or if there is only one fat .class and multiple classes (or something lighter) at runtime. If you have one .class, the compiler can not do something for you. regards, Remi Le 16 d?cembre 2015 16:45:25 CET, Paul Benedict a ?crit : >Yes, HotSpot is really good at that. I agree. However, my point is >there's >no reason in even carrying around the dead bytecode in the class file. >With >lots of these if() checks, the class file can surely shrink if the >compiler >gets some additional smarts. > >Cheers, >Paul > >On Wed, Dec 16, 2015 at 9:42 AM, Brian Goetz >wrote: > >> HotSpot is *really* good at eliminating dead code based on type >profiles. >> >> >> On 12/16/2015 10:38 AM, Paul Benedict wrote: >> >>> When a method is specialized for values, Any::isNull can only return >>> "false", which means if(Any.isNull(o)) can only ever have one >outcome -- >>> the other code branch can be discarded by the compiler. I know it's >too >>> early to talk about optimizations :-) but will javac be smart enough >to >>> recognize this optimization? Is it on your list for the future? >>> >>> Cheers, >>> Paul >>> >> >> From Mohammad.Rezaei at gs.com Wed Dec 16 16:49:03 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Wed, 16 Dec 2015 11:49:03 -0500 Subject: Any::isNull check In-Reply-To: <0C34AAB7-04FE-4A97-BD0B-48F19989284A@univ-mlv.fr> References: <56718656.2040409@oracle.com> <0C34AAB7-04FE-4A97-BD0B-48F19989284A@univ-mlv.fr> Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D91D@GSCMAMP06EX.firmwide.corp.gs.com> Doesn't this sort of optimization fit better in the (runtime) class specializer than either javac or C1/C2? Surely, the specializer has to deal with Any::isNull call sites regardless, so might as well not generate the dead code paths. Thanks Moh >-----Original Message----- >From: valhalla-dev [mailto:valhalla-dev-bounces at openjdk.java.net] On Behalf Of >R?mi Forax >Sent: Wednesday, December 16, 2015 11:36 AM >To: Paul Benedict; Brian Goetz >Cc: valhalla-dev >Subject: Re: Any::isNull check > >The question is whenever you have several .class for one generic class or if >there is only one fat .class and multiple classes (or something lighter) at >runtime. > >If you have one .class, the compiler can not do something for you. > >regards, >Remi > From pbenedict at apache.org Wed Dec 16 16:54:49 2015 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 16 Dec 2015 10:54:49 -0600 Subject: Any::isNull check In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D91D@GSCMAMP06EX.firmwide.corp.gs.com> References: <56718656.2040409@oracle.com> <0C34AAB7-04FE-4A97-BD0B-48F19989284A@univ-mlv.fr> <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D91D@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: When I asked my question, I wasn't aware Brian was using Any::isNull as an interim solution. He said things aren't going to stay this way. So my question turned out to be premature. Cheers, Paul On Wed, Dec 16, 2015 at 10:49 AM, Rezaei, Mohammad A. < Mohammad.Rezaei at gs.com> wrote: > Doesn't this sort of optimization fit better in the (runtime) class > specializer than either javac or C1/C2? Surely, the specializer has to deal > with Any::isNull call sites regardless, so might as well not generate the > dead code paths. > > Thanks > Moh > > >-----Original Message----- > >From: valhalla-dev [mailto:valhalla-dev-bounces at openjdk.java.net] On > Behalf Of > >R?mi Forax > >Sent: Wednesday, December 16, 2015 11:36 AM > >To: Paul Benedict; Brian Goetz > >Cc: valhalla-dev > >Subject: Re: Any::isNull check > > > >The question is whenever you have several .class for one generic class or > if > >there is only one fat .class and multiple classes (or something lighter) > at > >runtime. > > > >If you have one .class, the compiler can not do something for you. > > > >regards, > >Remi > > > > From brian.goetz at oracle.com Wed Dec 16 17:08:41 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 16 Dec 2015 12:08:41 -0500 Subject: Any::isNull check In-Reply-To: References: <56718768.7090403@oracle.com> <56718BA4.8040104@oracle.com> Message-ID: <56719A99.5000701@oracle.com> I think we're just spinning in "I could imagine ..." territory now. Try porting a real codebase (like Collections) and I think your perspective will change. On 12/16/2015 11:31 AM, Vitaly Davidovich wrote: > > If you mean that the developers should write two versions of each > method, one for nullable and one for not, that would be > ridiculous. Duplicating N lines of code because one line of a > method has to do a null check would be terrible. (Spend a half > hour trying to port the Collections code with that approach, and > ask yourself if you can honestly ask people to code that way.) > > > Not of each method. In Paul's case he made it sound like there's some > substantial dead code to be eliminated in case of value types, making > it sound like there's more than just one or two lines of difference. > The other approach is to take the one or two lines in a bigger method > that differ for value vs ref type and move just them into 2 > specialized methods, one for ref and other for value types. You mean, like the Any.isNull() method Paul was asking about? This solution sounds appealing until you try it. Asking people to split out tiny peeled methods sounds unintrusive. In the first 30 seconds, it even feels clever. Thereafter, it feels burdensome. > Given current limitations in inlining heuristics, removing dead > bytecode outright before JIT sees it is a big win. Right now, lots of > people end up splitting methods in awkward unnatural ways just to > avoid hitting these issues. When you say "people", be aware you really mean "the 0.001% of users for whom it makes a big enough difference to put up with it" (as this is the world you live in.) I care about those people too, but I care more about the code written by the other 99.999% of Java programmers. So, I'm focusing first on "what's the right expression of the code" and then "what's the right classfile representation". Going back to work now. From forax at univ-mlv.fr Wed Dec 16 17:28:37 2015 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 16 Dec 2015 18:28:37 +0100 (CET) Subject: Any::isNull check In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D91D@GSCMAMP06EX.firmwide.corp.gs.com> References: <56718656.2040409@oracle.com> <0C34AAB7-04FE-4A97-BD0B-48F19989284A@univ-mlv.fr> <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D91D@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: <801497216.1123906.1450286917894.JavaMail.zimbra@u-pem.fr> There are 7 'times' (for a lack of better word) were the specialization can appear, compile time, install time (a 'time' added by jigsaw when jlink is run to create the program image), load time, first invocation time, and jit runtime. I see no real reason to see the specialization as one monolithic phase, so specialization will certainly decomposed in different slices covering the last 3 'times'. regards, R?mi ----- Mail original ----- > De: "Mohammad A. Rezaei" > ?: "R?mi Forax" , "Paul Benedict" , "Brian Goetz" > Cc: "valhalla-dev" > Envoy?: Mercredi 16 D?cembre 2015 17:49:03 > Objet: RE: Any::isNull check > > Doesn't this sort of optimization fit better in the (runtime) class > specializer than either javac or C1/C2? Surely, the specializer has to deal > with Any::isNull call sites regardless, so might as well not generate the > dead code paths. > > Thanks > Moh > > >-----Original Message----- > >From: valhalla-dev [mailto:valhalla-dev-bounces at openjdk.java.net] On Behalf > >Of > >R?mi Forax > >Sent: Wednesday, December 16, 2015 11:36 AM > >To: Paul Benedict; Brian Goetz > >Cc: valhalla-dev > >Subject: Re: Any::isNull check > > > >The question is whenever you have several .class for one generic class or if > >there is only one fat .class and multiple classes (or something lighter) at > >runtime. > > > >If you have one .class, the compiler can not do something for you. > > > >regards, > >Remi > > > > From ali.ebrahimi1781 at gmail.com Wed Dec 16 17:37:26 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Wed, 16 Dec 2015 21:07:26 +0330 Subject: __WhereVal/__WhereRef on fields Message-ID: Hi, Is compiler supposed to handle restrictions on fields? public class Main { public static class Test{ public __WhereVal(T) int x; public __WhereRef(T) long x2; } public static void main(String[] args) { Test ti = new Test(); long p = ti.x2; //<======================= } } -- Best Regards, Ali Ebrahimi From brian.goetz at oracle.com Wed Dec 16 17:43:06 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 16 Dec 2015 12:43:06 -0500 Subject: __WhereVal/__WhereRef on fields In-Reply-To: References: Message-ID: <5671A2AA.1020105@oracle.com> It definitely does not currently do so. Not clear whether this will be part of the eventual model or not -- still figuring out the pros and cons of this. On 12/16/2015 12:37 PM, Ali Ebrahimi wrote: > Hi, > Is compiler supposed to handle restrictions on fields? > > public class Main { > > public static class Test{ > public __WhereVal(T) int x; > public __WhereRef(T) long x2; > } > public static void main(String[] args) { > Test ti = new Test(); > long p = ti.x2; //<======================= > } > } > > From brian.goetz at oracle.com Wed Dec 16 17:49:15 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 16 Dec 2015 12:49:15 -0500 Subject: __WhereVal/__WhereRef on fields In-Reply-To: <5671A2AA.1020105@oracle.com> References: <5671A2AA.1020105@oracle.com> Message-ID: <5671A41B.7090201@oracle.com> To put this in perspective, you can do anything with refs that you can do with values, and more (assign to null, synchronize on them, etc.) So rather than thinking in terms of "here's the value version, and here's the ref version", its more effective to think in terms of "here's the any version (that works for values and refs)" and "here's the optimized/specialized version for refs". (In other words, __WhereVal is not needed and is probably going away as part of the model.) At which point, being able to __WhereRef on a field is purely a space optimization -- you're free to declare the field and not use it. Hence, its a low priority to support that at this stage. On 12/16/2015 12:43 PM, Brian Goetz wrote: > It definitely does not currently do so. > > Not clear whether this will be part of the eventual model or not -- > still figuring out the pros and cons of this. > > On 12/16/2015 12:37 PM, Ali Ebrahimi wrote: >> Hi, >> Is compiler supposed to handle restrictions on fields? >> >> public class Main { >> >> public static class Test{ >> public __WhereVal(T) int x; >> public __WhereRef(T) long x2; >> } >> public static void main(String[] args) { >> Test ti = new Test(); >> long p = ti.x2; //<======================= >> } >> } >> >> > From vitalyd at gmail.com Wed Dec 16 17:52:07 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 16 Dec 2015 12:52:07 -0500 Subject: Any::isNull check In-Reply-To: <56719A99.5000701@oracle.com> References: <56718768.7090403@oracle.com> <56718BA4.8040104@oracle.com> <56719A99.5000701@oracle.com> Message-ID: On Wednesday, December 16, 2015, Brian Goetz wrote: > I think we're just spinning in "I could imagine ..." territory now. Try > porting a real codebase (like Collections) and I think your perspective > will change. > Specialization has existed in other languages for more than a decade. There's some imagination needed for the Java version but the general principle of "how do I specialize/organize code paths specific to a certain type or sets of types sharing common characteristics" is not new. > > On 12/16/2015 11:31 AM, Vitaly Davidovich wrote: > > If you mean that the developers should write two versions of each method, >> one for nullable and one for not, that would be ridiculous. Duplicating N >> lines of code because one line of a method has to do a null check would be >> terrible. (Spend a half hour trying to port the Collections code with that >> approach, and ask yourself if you can honestly ask people to code that >> way.) > > > Not of each method. In Paul's case he made it sound like there's some > substantial dead code to be eliminated in case of value types, making it > sound like there's more than just one or two lines of difference. The > other approach is to take the one or two lines in a bigger method that > differ for value vs ref type and move just them into 2 specialized methods, > one for ref and other for value types. > > You mean, like the Any.isNull() method Paul was asking about? > Paul's question wasn't about the method but rather the code it guards. > > > This solution sounds appealing until you try it. Asking people to split > out tiny peeled methods sounds unintrusive. In the first 30 seconds, it > even feels clever. Thereafter, it feels burdensome. > Is the concept burdensome or the syntax? I don't think the concept is burdensome as the code is dealing with fundamentally different types. The question is how do you write a generic method where some code paths are applicable for value and other for reference types; do you bulk it all up into one method with type guards or split it into separate distinct pieces entirely. Besides the byte code size implications I personally find splitting into distinct units more sound and easier to follow. I realize this is subjective though. > > > Given current limitations in inlining heuristics, removing dead bytecode > outright before JIT sees it is a big win. Right now, lots of people end up > splitting methods in awkward unnatural ways just to avoid hitting these > issues. > > > When you say "people", be aware you really mean "the 0.001% of users for > whom it makes a big enough difference to put up with it" (as this is the > world you live in.) I care about those people too, but I care more about > the code written by the other 99.999% of Java programmers. So, I'm > focusing first on "what's the right expression of the code" and then > "what's the right classfile representation". > > Going back to work now. > > I really dislike this percentile breakdown that you've mentioned several times on this list. JDK core libs developers themselves have to deal with this, and that code is used by many people with some expectation of performance. The whole value type project is performance geared! Moreover, nobody is suggesting some sharp tool be added only for experts. I'm simply saying that given the choice of presenting more byte code vs less makes a difference. If there's a reasonable way to present less it's a win. Saying JIT is really good at eliminating dead code is, while true, not the full picture when the topic is performance. -- Sent from my phone From brian.goetz at oracle.com Wed Dec 16 18:05:25 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 16 Dec 2015 13:05:25 -0500 Subject: Any::isNull check In-Reply-To: References: <56718768.7090403@oracle.com> <56718BA4.8040104@oracle.com> <56719A99.5000701@oracle.com> Message-ID: <5671A7E5.6010000@oracle.com> > I really dislike this percentile breakdown that you've mentioned > several times on this list. JDK core libs developers themselves have > to deal with this, and that code is used by many people with some > expectation of performance. The whole value type project is > performance geared! Yes. JDK developers have to, and are willing to, code in ways we're not willing to ask other people to code. We're fully aware of the need to make these things *possible*. But we're also quite aware of what it is, and isn't, reasonable to ask of developers in terms of how regular developers write their code. And we are for sure not going to let micro-performance issues like this drive how the majority of users are expected to code. > Moreover, nobody is suggesting some sharp tool be added only for > experts. I'm simply saying that given the choice of presenting more > byte code vs less makes a difference. If there's a reasonable way to > present less it's a win. Saying JIT is really good at eliminating > dead code is, while true, not the full picture when the topic is > performance. Performance is not the full picture either. Quite honestly, at this point, micro-performance issues like this should not even be part of the discussion at all. We are well aware of the performance tradeoffs that await us in the event we actually get to a model that we think is good enough to foist on Java users. But its simply way too early to deep-dive on micro-performance issues like this -- while this will eventually be important, at this early stage, its a distraction. So, can we please go back to work now? From vitalyd at gmail.com Wed Dec 16 18:14:17 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 16 Dec 2015 13:14:17 -0500 Subject: Any::isNull check In-Reply-To: <5671A7E5.6010000@oracle.com> References: <56718768.7090403@oracle.com> <56718BA4.8040104@oracle.com> <56719A99.5000701@oracle.com> <5671A7E5.6010000@oracle.com> Message-ID: > > Yes. JDK developers have to, and are willing to, code in ways we're not > willing to ask other people to code. We're fully aware of the need to make > these things *possible*. But we're also quite aware of what it is, and > isn't, reasonable to ask of developers in terms of how regular developers > write their code. And we are for sure not going to let micro-performance > issues like this drive how the majority of users are expected to code. The conversation was really about how to organize chunks of code dealing with ref vs value types; I mentioned the bytecode size implications of big methods on inlining as a secondary effect. Taking performance out of the equation, some "regular" Java developers will like to split their value vs ref type code blocks into separate methods; others may like to have one big method with Any.isNull(or whatever syntax will be) guards. Didn't Paul have a thread about that a few days ago? So either way I suspect you won't please everyone anyway, at which point, if you're going to choose one approach vs the other you may as well take the one with some side performance benefit. I'd also argue that inlining is not a micro optimization for Java -- the entire performance model is predicated on this happening in the right places. Anything that somehow upsets that fragile beast or worsens the situation will not be well-received. Anyway, I'll punt on further performance discussion at this point. > So, can we please go back to work now? Yes :) On Wed, Dec 16, 2015 at 1:05 PM, Brian Goetz wrote: > I really dislike this percentile breakdown that you've mentioned several >> times on this list. JDK core libs developers themselves have to deal with >> this, and that code is used by many people with some expectation of >> performance. The whole value type project is performance geared! >> > > Yes. JDK developers have to, and are willing to, code in ways we're not > willing to ask other people to code. We're fully aware of the need to make > these things *possible*. But we're also quite aware of what it is, and > isn't, reasonable to ask of developers in terms of how regular developers > write their code. And we are for sure not going to let micro-performance > issues like this drive how the majority of users are expected to code. > > Moreover, nobody is suggesting some sharp tool be added only for experts. >> I'm simply saying that given the choice of presenting more byte code vs >> less makes a difference. If there's a reasonable way to present less it's >> a win. Saying JIT is really good at eliminating dead code is, while true, >> not the full picture when the topic is performance. >> > > Performance is not the full picture either. > > Quite honestly, at this point, micro-performance issues like this should > not even be part of the discussion at all. We are well aware of the > performance tradeoffs that await us in the event we actually get to a model > that we think is good enough to foist on Java users. But its simply way too > early to deep-dive on micro-performance issues like this -- while this will > eventually be important, at this early stage, its a distraction. > > So, can we please go back to work now? > > > From maurizio.cimadamore at oracle.com Wed Dec 16 19:02:04 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Dec 2015 19:02:04 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: add support for inner classes specialization Message-ID: <201512161902.tBGJ24Qb027850@aojmv0008.oracle.com> Changeset: 7d1d10fd767b Author: mcimadamore Date: 2015-12-16 19:01 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/7d1d10fd767b Enhancement: add support for inner classes specialization ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Lower.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java + test/tools/javac/valhalla/typespec/Inner02.java From ali.ebrahimi1781 at gmail.com Wed Dec 16 20:30:44 2015 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Thu, 17 Dec 2015 00:00:44 +0330 Subject: __WhereVal/__WhereRef on fields In-Reply-To: <5671A41B.7090201@oracle.com> References: <5671A2AA.1020105@oracle.com> <5671A41B.7090201@oracle.com> Message-ID: Hi, On Wed, Dec 16, 2015 at 9:19 PM, Brian Goetz wrote: > To put this in perspective, you can do anything with refs that you can do > with values, and more (assign to null, synchronize on them, etc.) So > rather than thinking in terms of "here's the value version, and here's the > ref version", its more effective to think in terms of "here's the any > version (that works for values and refs)" and "here's the > optimized/specialized version for refs". (In other words, __WhereVal is not > needed and is probably going away as part of the model.) > I was thinking about some form of C's union types. Think of this as: class ArrayList ...{ private __WhereVal(T){Type1}|__WhereRef(T){T[]} elementData; Or even more customized storage: private __Where(T is boolean){BitVector}|__ELSE(T){Type2} elementData; Or if would need only val version: private __WhereVal(T){Type1} elementData; This should not allocated for reffs. In other word, one field of same name but with different type. I think for supporting this level of customization we would need an additional Template class file for Val specific bytecode. Test.class Class Test{ } Test.vtemplate Class Test.vtemplate ... // val type members } -- Best Regards, Ali Ebrahimi From maurizio.cimadamore at oracle.com Wed Dec 16 21:59:40 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Dec 2015 21:59:40 +0000 Subject: hg: valhalla/valhalla/jdk: 3 new changesets Message-ID: <201512162159.tBGLxeXv004918@aojmv0008.oracle.com> Changeset: 018488e3cfe5 Author: mcimadamore Date: 2015-12-16 19:02 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/018488e3cfe5 Enhancement: add inner class support; overhaul of signature specialization code. ! src/java.base/share/classes/java/lang/invoke/GenericMethodSpecializer.java ! src/java.base/share/classes/jdk/internal/org/objectweb/asm/ClassReader.java ! src/java.base/share/classes/jdk/internal/org/objectweb/asm/signature/SignatureWriter.java ! src/java.base/share/classes/valhalla/specializer/DebuggingSignatureVisitor.java ! src/java.base/share/classes/valhalla/specializer/SignatureSpecializer.java ! src/java.base/share/classes/valhalla/specializer/Specializer.java ! test/valhalla/boottest/valhalla/specializer/SignatureSpecializerTest.java ! test/valhalla/test/valhalla/specializer/SignatureVisitorTest.java Changeset: f6ae026f295c Author: mcimadamore Date: 2015-12-16 19:03 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/f6ae026f295c merge Changeset: e6ae3f64c62f Author: mcimadamore Date: 2015-12-16 21:59 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/e6ae3f64c62f Fix: signature specializer should ignore thrown types when generating descriptors ! src/java.base/share/classes/valhalla/specializer/SignatureSpecializer.java From maurizio.cimadamore at oracle.com Thu Dec 17 16:28:24 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 17 Dec 2015 16:28:24 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: inner class support doesn't work when outer class type is unbound Message-ID: <201512171628.tBHGSOa3017458@aojmv0008.oracle.com> Changeset: 3f01e9c496fe Author: mcimadamore Date: 2015-12-17 16:28 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/3f01e9c496fe Fix: inner class support doesn't work when outer class type is unbound ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java ! test/tools/javac/valhalla/typespec/Inner02.java From maurizio.cimadamore at oracle.com Thu Dec 17 16:30:51 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 17 Dec 2015 16:30:51 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: revert desugared inner class in anyutil package Message-ID: <201512171630.tBHGUpnk018540@aojmv0008.oracle.com> Changeset: 550d7df8cfa1 Author: mcimadamore Date: 2015-12-17 16:30 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/550d7df8cfa1 Enhancement: revert desugared inner class in anyutil package Fix: X.default doesn't get specialized properly when X is erased Fix: class specializations are not set up correctly if signature attribute is missing ! src/java.base/share/classes/java/anyutil/AbstractCollection.java ! src/java.base/share/classes/java/anyutil/AbstractList.java ! src/java.base/share/classes/java/anyutil/ArrayDeque.java ! src/java.base/share/classes/java/anyutil/ArrayList.java ! src/java.base/share/classes/java/anyutil/HashMap.java ! src/java.base/share/classes/java/anyutil/LinkedHashMap.java ! src/java.base/share/classes/java/anyutil/LinkedList.java ! src/java.base/share/classes/java/anyutil/SortedSet.java ! src/java.base/share/classes/java/anyutil/stream/ReduceOps.java ! src/java.base/share/classes/java/anyutil/stream/SpinedBuffer.java ! src/java.base/share/classes/valhalla/specializer/Specializer.java ! test/valhalla/test/valhalla/anyutil/CollectionsTest.java From twhitmore.nz at gmail.com Fri Dec 18 09:46:34 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Fri, 18 Dec 2015 22:46:34 +1300 Subject: sentinels & who owns the value space Re: valhalla-dev Digest, Vol 17, Issue 10 Message-ID: I'm going to agree with Vitaly here. [requiring a Sentinel Factory passed into constructor ripples all over the API, making it more cumbersome to use] > Another reason is that then the code depends on an external party to > generate good sentinels for its internal use. What is a ?good sentinel > value? is an implementation detail that might have to be documented. As the > client is only concerned with generating real values for its own uses and The definition of a 'good sentinel value', is a value the client doesn't use. The idea of dynamic sentinels seems complicated & marginal to me. If you did, it would have to be with regard to the Client's value space. Strategy is an obvious & correct answer. > Scribbling some random bits over their type seems worse. Can't work reliably. What you're proposing would break information theory; client unaware of any restrictions on value space and collection picking random sentinels, is essentially hoping that "randomness" will somehow enable you to convey more information than a byte can hold. Reliable use of sentinels needs to contract an agreed 'allowable' value space, or use allocation (for reference types) to obtain a unique one. Regards, Thomas From Mohammad.Rezaei at gs.com Fri Dec 18 12:28:43 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Fri, 18 Dec 2015 07:28:43 -0500 Subject: sentinels & who owns the value space Re: valhalla-dev Digest, Vol 17, Issue 10 In-Reply-To: References: Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9B3@GSCMAMP06EX.firmwide.corp.gs.com> > >The definition of a 'good sentinel value', is a value the client doesn't >use. > A properly written data structure (see trove, gs collections and koloboke for examples) does not preclude the client from using the sentinel values in the collection or for other purposes. null is a sentinel in hashmap, yet null keys are fully supported. A properly written data structure hides the actual sentinel values from the client. >> Scribbling some random bits over their type seems worse. > >Can't work reliably. > It provably works in C and can work for pure (no ref field) value types in Java. (I'm not saying it's a great idea; we've explored alternatives in another thread on this list) >What you're proposing would break information theory; client unaware of any >restrictions on value space and collection picking random sentinels, is >essentially hoping that "randomness" will somehow enable you to convey more >information than a byte can hold. No information theory is broken, because the client does not need to be aware of the internal sentinels used in the data structure. Maybe you're confusing API contracts that use constants (like map.get) with real sentinels. The API level issues can always be solved in other ways, e.g. calling map.contains first, or using a different form, like map.getOrElse or returning Optional from map.get. Thanks Moh From boaznahum at gmail.com Fri Dec 18 13:16:50 2015 From: boaznahum at gmail.com (Boaz Nahum) Date: Fri, 18 Dec 2015 13:16:50 +0000 Subject: Build failure on windows 'freetypeScaler.c' Message-ID: Hi Lately, I have this error: f:/Dev/JDKBuild/valhalla/jdk/src/java.desktop/share/native/libfontmanager/freetypeScaler.c(106) : error C2220: warning treated as error - no 'object' file generated f:/Dev/JDKBuild/valhalla/jdk/src/java.desktop/share/native/libfontmanager/freetypeScaler.c(106) : warning C4101: 'stream' : unreferenced local variable I comment out: static void freeNativeResources(JNIEnv *env, FTScalerInfo* scalerInfo) { // void *stream; And now it works. Thx Boaz From maurizio.cimadamore at oracle.com Fri Dec 18 18:24:22 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 18 Dec 2015 18:24:22 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: simplify Items hierarchy Message-ID: <201512181824.tBIIOMs7018722@aojmv0008.oracle.com> Changeset: 71e8f37f46ae Author: mcimadamore Date: 2015-12-18 18:24 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/71e8f37f46ae Enhancement: simplify Items hierarchy ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Lower.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/util/ByteBuffer.java ! test/tools/javac/valhalla/typespec/items/tests/TestRespecialization.java From maurizio.cimadamore at oracle.com Fri Dec 18 18:26:17 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 18 Dec 2015 18:26:17 +0000 Subject: hg: valhalla/valhalla/jdk: 8139803: Fix for 8132985 breaks OpenJDK build on windows. Message-ID: <201512181826.tBIIQHa1019332@aojmv0008.oracle.com> Changeset: aabf7e5b3dc7 Author: mcimadamore Date: 2015-12-18 18:26 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/aabf7e5b3dc7 8139803: Fix for 8132985 breaks OpenJDK build on windows. Reviewed-by: serb, simonis ! src/java.desktop/share/native/libfontmanager/freetypeScaler.c From maurizio.cimadamore at oracle.com Fri Dec 18 18:26:44 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 18 Dec 2015 18:26:44 +0000 Subject: Build failure on windows 'freetypeScaler.c' In-Reply-To: References: Message-ID: <56744FE4.4030404@oracle.com> Fix pushed Maurizio On 18/12/15 13:16, Boaz Nahum wrote: > Hi > > Lately, I have this error: > > f:/Dev/JDKBuild/valhalla/jdk/src/java.desktop/share/native/libfontmanager/freetypeScaler.c(106) > : error C2220: warning treated as error - no 'object' file generated > f:/Dev/JDKBuild/valhalla/jdk/src/java.desktop/share/native/libfontmanager/freetypeScaler.c(106) > : warning C4101: 'stream' : unreferenced local variable > > I comment out: > > static void freeNativeResources(JNIEnv *env, FTScalerInfo* scalerInfo) { > // void *stream; > > > And now it works. > > Thx > Boaz From twhitmore.nz at gmail.com Fri Dec 18 23:52:22 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Sat, 19 Dec 2015 12:52:22 +1300 Subject: sentinels & who owns the value space Re: valhalla-dev Digest, Vol 17, Issue 10 In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9B3@GSCMAMP06EX.firmwide.corp.gs.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9B3@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: Hi Mohammad. > A properly written data structure (see trove, gs collections and koloboke for examples) does > not preclude the client from using the sentinel values in the collection or for other purposes. null is a sentinel in hashmap, yet > null keys are fully supported. A properly written data structure hides the actual sentinel values from the client. Thanks for your email. You may be referring to a sentinel for _entries_, which is a different issue from sentinels for _keys_ as _entries_ are not the client value space. I'm not sure it would even be correct terminology to describe null entries as "sentinels"; they're definitely not sentinel key values. If the collection requires a sentinel for internal meaning, an unwrapped null sentinel value cannot be used in a collection allowing null values from the client where the same storage element is required to store both. This is due to the simple fact that the meaning of that sentinel cannot not be distinguished from all valid client values. Don't forget, if you want to consider eg. a byte type -- the client has 256 valid values, plus you're saying the collection has 1 more (the sentinel) for a total of 257 values. Bytes do not allow 257 values. When I said information theory, I meant it. We are discussing use of sentinels in value space _directly_. I expect you are referring to alternative approaches using additional storage/ or indirection, which make it easy to solve the problem -- see below. However your (very vague) references presumably to those approaches don't seem to shed any light on the (different) topic which we were discussing. Sentinels directly in value space, with no extra storage, indirection or workarounds, is really what we were discussing. > No information theory is broken, because the client does not need to be aware of the internal sentinels used in the data > structure. Information theory then tells us that a collection allowing 256 valid client values, plus distinguishing a sentinel, must using more than 1 byte for storage. Indirection, reference-type sentinels or additional structure are all possible. The collection will not be using just the declared datatype for storage, since those 257 distinct meanings cannot be coded in a single byte. Once we accept a need for indirection/ extra storage, the 'empty cell' problem is already solved -- the collection has space for a flag/ or can easily mask null with a unique non-null reference, etc. Simplest solutions are the most direct; which would seem to leave no benefit (only negatives) for random-bit approaches. Regards, Thomas From twhitmore.nz at gmail.com Sat Dec 19 00:26:59 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Sat, 19 Dec 2015 13:26:59 +1300 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: Hi Brian, Vitaly, Thanks for both your thoughts. I guess you can tell my design preferences are clearly on the side of KISS.. but we'll put that aside for now, and consider the question. My observation would be that "ArrayList" has fairly minimal/ uncommon utility.. We're not really discussing language changes for a list of booleans as the requirement. So I would question whether this whole discussion, is actually a stalking horse for something else. - ArrayList> perhaps? to me, that seems like the only really worthwhile candidate. - are there any others of genuine value? Vitaly, when we discuss the "contract of ArrayList" today -- or with simple byte-addressed value type genericization, where the array multiplier -- we get a single clear statement of what it does, which I would consider reasonable to describe as a "contract". As a thought experiment, let's suppose we give "somebody" this feature of allowing type-specific overrides, and let them run with it for a while. - let's say they find the benefits of making ArrayList more compact, as nearly zero value; - but hey, large objects could be stored off-heap! that's more valuable than making dumb old boolean a tiny bit smaller. --- so they add special support for some perceived large type, to store it off-heap. it works great. - but hey, we can store more if we go to disk. --- so they add file storage as well. there's a few dependencies & interactions, but come on, it works great. - that works great. but hey, we're putting Comparables in the list. shouldn't it be able to sort these automatically? --- so they add that.. it works great. All of these features work great! What's the problem? Problem is, the design of the class is now rubbish. It doesn't do one thing well any more, it does five. Ask what the "contract" is and now it's a meaningless jumble of special-cases. OO Principle is to prefer composition to inheritance. "Inheritance" in the principle means the embodying of functionality in a single class; for us, this would be via type-specific templating rather than inheritance. I for one, find a great deal of value in having simple & well-defined classes which do their job. - If I want a different implementation, I instantiate a different class. - I presume that a very large proportion of the requirement for List are actually aware that they're using a 'boolean' -- and can instantiate a BooleanBitList specifically, if they actually need/ want to optimize storage of it. While it's easy to say that SomeCollection> has a great case to support efficiently, I think that -- if we consider type-specific templating as a general architectural direction -- the place that road leads to, if pursued more than a very little bit, are questionable. Which obviously raises uncertainty -- if the architectural approach (when used other than minimally) leads in a direction we don't want to go, as to whether it's the right direction. I think this is an exploratory phase so these are all great questions to consider. But I'd like to conclude by asking, what types are the real drivers for this? Regards, Thomas From vitalyd at gmail.com Sat Dec 19 00:53:59 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 18 Dec 2015 19:53:59 -0500 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: Hi Thomas, > Thanks for both your thoughts. I guess you can tell my design preferences > are clearly on the side of KISS.. but we'll put that aside for now, and > consider the question. I'm going to preface by saying that I see this design decision (i.e. specialize vs dedicated type) akin to interface vs abstract class; both can be used to model abstractions, but each can be more appropriate/"better" in some circumstances. My observation would be that "ArrayList" has fairly minimal/ > uncommon utility.. We're not really discussing language changes for a list > of booleans as the requirement. So I would question whether this whole > discussion, is actually a stalking horse for something else. > - ArrayList> perhaps? to me, that seems like the only really > worthwhile candidate. > - are there any others of genuine value? ArrayList is just an easy example with similar "prior art" in a different language (i.e. C++ std::vector); the other one mentioned before, and you're alluding to here, is Optional where T is a ref type --> should be same size as T. There will be some domain-specific places where this can be leveraged, but I agree this won't be widespread, so to speak. Having said that, if Optional is ported to being a value type (or some successor is introduced, I don't really care), there are quite a few places where using it in APIs would provide more null safety and self-documenting code (aside: I'd rather not get into the whole debate about using @NotNull annotations vs library solutions here). However, having Optional where T=ref take up more space than just T would be a shame. It's not *critical*, of course, but unless the amount of work to make this a reality is tremendous, it's a very good optimization and makes using this type more palatable. As a thought experiment, let's suppose we give "somebody" this feature of > allowing type-specific overrides, and let them run with it for a while. > - let's say they find the benefits of making ArrayList more > compact, as nearly zero value; > - but hey, large objects could be stored off-heap! that's more valuable > than making dumb old boolean a tiny bit smaller. > --- so they add special support for some perceived large type, to store it > off-heap. it works great. > - but hey, we can store more if we go to disk. > --- so they add file storage as well. there's a few dependencies & > interactions, but come on, it works great. > - that works great. but hey, we're putting Comparables in the list. > shouldn't it be able to sort these automatically? > --- so they add that.. it works great. All of these features work great! What's the problem? Problem is, the > design of the class is now rubbish. It doesn't do one thing well any more, > it does five. Ask what the "contract" is and now it's a meaningless jumble > of special-cases. In my opinion, this has nothing to do with specialization vs dedicated types -- you can design a monster with either approach. In general, no language prevents "terrible" design, however one wants to define that (nevermind it's subjective, but let's no get hung up on that). The biggest advantage to doing this via specialization is that if you do come up with a better way to represent a generic type with a given type arg, the user of such instantiations doesn't need to change their code. As a bonus, you don't need to provide a name for this thing (we all like naming bikesheds right? :)). Your alternative approach doesn't stop someone from creating a named specialized type, providing some means for getting access to it (or even hiding it behind some interface/abstraction), and letting someone use it. So the only difference that I can see is that by explicitly making user change code to take advantage of the new type they don't unknowingly use a terribly-implemented specialization? On Fri, Dec 18, 2015 at 7:26 PM, Thomas W wrote: > Hi Brian, Vitaly, > > Thanks for both your thoughts. I guess you can tell my design preferences > are clearly on the side of KISS.. but we'll put that aside for now, and > consider the question. > > My observation would be that "ArrayList" has fairly minimal/ > uncommon utility.. We're not really discussing language changes for a list > of booleans as the requirement. So I would question whether this whole > discussion, is actually a stalking horse for something else. > - ArrayList> perhaps? to me, that seems like the only really > worthwhile candidate. > - are there any others of genuine value? > > Vitaly, when we discuss the "contract of ArrayList" today -- or with > simple byte-addressed value type genericization, where the array > multiplier -- we get a single clear statement of what it does, which I > would consider reasonable to describe as a "contract". > > As a thought experiment, let's suppose we give "somebody" this feature of > allowing type-specific overrides, and let them run with it for a while. > - let's say they find the benefits of making ArrayList more > compact, as nearly zero value; > - but hey, large objects could be stored off-heap! that's more valuable > than making dumb old boolean a tiny bit smaller. > --- so they add special support for some perceived large type, to store it > off-heap. it works great. > - but hey, we can store more if we go to disk. > --- so they add file storage as well. there's a few dependencies & > interactions, but come on, it works great. > - that works great. but hey, we're putting Comparables in the list. > shouldn't it be able to sort these automatically? > --- so they add that.. it works great. > > All of these features work great! What's the problem? Problem is, the > design of the class is now rubbish. It doesn't do one thing well any more, > it does five. Ask what the "contract" is and now it's a meaningless jumble > of special-cases. > > OO Principle is to prefer composition to inheritance. "Inheritance" in > the principle means the embodying of functionality in a single class; for > us, this would be via type-specific templating rather than inheritance. > > I for one, find a great deal of value in having simple & well-defined > classes which do their job. > - If I want a different implementation, I instantiate a different class. > - I presume that a very large proportion of the requirement for > List are actually aware that they're using a 'boolean' -- and can > instantiate a BooleanBitList specifically, if they actually need/ want to > optimize storage of it. > > While it's easy to say that SomeCollection> has a great case > to support efficiently, I think that -- if we consider type-specific > templating as a general architectural direction -- the place that road > leads to, if pursued more than a very little bit, are questionable. > > Which obviously raises uncertainty -- if the architectural approach (when > used other than minimally) leads in a direction we don't want to go, as to > whether it's the right direction. > > I think this is an exploratory phase so these are all great questions to > consider. But I'd like to conclude by asking, what types are the real > drivers for this? > > > Regards, > Thomas > From Mohammad.Rezaei at gs.com Sat Dec 19 04:57:20 2015 From: Mohammad.Rezaei at gs.com (Rezaei, Mohammad A.) Date: Fri, 18 Dec 2015 23:57:20 -0500 Subject: sentinels & who owns the value space Re: valhalla-dev Digest, Vol 17, Issue 10 In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9B3@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9C9@GSCMAMP06EX.firmwide.corp.gs.com> Let?s take a simple example: IntHashSet in GS Collections. As the name suggests, it stores ints. All ints may be stored in the set. The set is implemented as an open hash set with a single int array, 4 int fields and no reference objects: private int[] table; private int occupiedWithData; private int occupiedWithSentinels; // The 32 bits of this integer indicate whether the items 0 to 31 are present in the set. private int zeroToThirtyOne; private int zeroToThirtyOneOccupied; No wrapper objects of any kind (Integer, or Entry) are created when set operations are performed. Internally, there are two special values: EMPTY (0) and REMOVED (1). If a position in the array is EMPTY, there is no client value there. If a position in the array is REMOVED, there was once a client value there, but not anymore. We call these two special values sentinels. Any other value in the array means that value is in the set. The following code is perfectly legal: IntHashSet set = new IntHashSet(); set.contains(0); // returns false, even though the internal array is full of 0 set.add(0); set.add(1); set.contains(0); // returns true set.contains(1); // returns true this is accomplished by remembering the addition of 0 or 1 by flipping a bit in zeroToThirtyOne. All methods (add, contains, remove, etc) are guarded by if-statements that handle 0 and 1 specially. We don?t need 2^32 + 2 values, even though two values have special meaning in the structure, and they are not excluded from the client?s use, nor does the main storage structure need more than 32 bits per element. The cost of the extra storage (the int zeroToThiryOne) is constant and does not affect the main storage; it?s overhead goes to zero as the set size increases. All of this is possible because the set can designate two client space values as special and use a uniform single array for storage. The same approach is impossible with value types as proposed in Valhalla. Thanks Moh From: Thomas W [mailto:twhitmore.nz at gmail.com] Sent: Friday, December 18, 2015 6:52 PM To: Rezaei, Mohammad A. [Tech] Cc: valhalla-dev at openjdk.java.net; timo.kinnunen at gmail.com; vitalyd at gmail.com Subject: Re: sentinels & who owns the value space Re: valhalla-dev Digest, Vol 17, Issue 10 Hi Mohammad. > A properly written data structure (see trove, gs collections and koloboke for examples) does > not preclude the client from using the sentinel values in the collection or for other purposes. null is a sentinel in hashmap, yet > null keys are fully supported. A properly written data structure hides the actual sentinel values from the client. Thanks for your email. You may be referring to a sentinel for _entries_, which is a different issue from sentinels for _keys_ as _entries_ are not the client value space. I'm not sure it would even be correct terminology to describe null entries as "sentinels"; they're definitely not sentinel key values. If the collection requires a sentinel for internal meaning, an unwrapped null sentinel value cannot be used in a collection allowing null values from the client where the same storage element is required to store both. This is due to the simple fact that the meaning of that sentinel cannot not be distinguished from all valid client values. Don't forget, if you want to consider eg. a byte type -- the client has 256 valid values, plus you're saying the collection has 1 more (the sentinel) for a total of 257 values. Bytes do not allow 257 values. When I said information theory, I meant it. We are discussing use of sentinels in value space _directly_. I expect you are referring to alternative approaches using additional storage/ or indirection, which make it easy to solve the problem -- see below. However your (very vague) references presumably to those approaches don't seem to shed any light on the (different) topic which we were discussing. Sentinels directly in value space, with no extra storage, indirection or workarounds, is really what we were discussing. > No information theory is broken, because the client does not need to be aware of the internal sentinels used in the data > structure. Information theory then tells us that a collection allowing 256 valid client values, plus distinguishing a sentinel, must using more than 1 byte for storage. Indirection, reference-type sentinels or additional structure are all possible. The collection will not be using just the declared datatype for storage, since those 257 distinct meanings cannot be coded in a single byte. Once we accept a need for indirection/ extra storage, the 'empty cell' problem is already solved -- the collection has space for a flag/ or can easily mask null with a unique non-null reference, etc. Simplest solutions are the most direct; which would seem to leave no benefit (only negatives) for random-bit approaches. Regards, Thomas From twhitmore.nz at gmail.com Sat Dec 19 05:45:41 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Sat, 19 Dec 2015 18:45:41 +1300 Subject: sentinels & who owns the value space Re: valhalla-dev Digest, Vol 17, Issue 10 In-Reply-To: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9C9@GSCMAMP06EX.firmwide.corp.gs.com> References: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9B3@GSCMAMP06EX.firmwide.corp.gs.com> <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9C9@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: Great example. Thanks for being specific, Mohammad. Your example illustrates the use of additional storage (the *zeroToThirtyOne* field, etc) as an auxilliary to provide sufficient representation to distinguish known sentinels from valid client values. I would agree that this design might be harder to apply generically to arbitrary value types, but I'd presume it works for any integral types having >2 values (it doesn't exactly fail for boolean, representation become entirely auxilliary though). My thinking on Valhalla has emphasized "abstracting" the difference between types (reftypes, primitives & value types) and allowing existing idiom & usage to be expressed in an abstract way; my feeling being that building capable abstractions over 'any' type, would give a stronger foundation that needing to lean on programmatic specialization. I previously proposed having the language offer both "T.default" and "T.sentinel" as distinct values -- to represent the two distinct uses of null better in eg. integer space. Null can be a default and 0 is a good default; null can also be a sentinel, but Integer.MIN_VALUE is a much better sentinel than 0 :) I guess what I'm working to is, that having the language provide two abstracted values -- 'default' and 'sentinel' -- which are distinct, could be used as the two cell sentinels for your hashset & special-cased in the add/ contains check etc. This would be extensible to value-types by the component fields being set to default/ or sentinel appropriately. The only problem is ref-types, which in my proposal would have both RefType.default == null and RefType.sentinel == null; and would thus not be amenable to providing your two separate cell markers. Thanks for your thoughts. Regards, Thomas From twhitmore.nz at gmail.com Sat Dec 19 06:04:53 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Sat, 19 Dec 2015 19:04:53 +1300 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: Thanks Vitaly. If Optional is the goal, perhaps the question might be to inline/ specialize the storage of monadic/ "wrapper types" generally? There are probably a range of monadic/ wrapper types having one "major field" and decorated it with simple behavior. However relying on handcoding magic specializations in ArrayList, will restrict it to applying to Optional only. An alternative way of looking at this is to to designate Optional as "inline value" which then would enable a _generalized system_ to arbitrarily reorganize its storage and store/materialize it as necessary. While stored it would not be in standard representation and inlined accessors would be adapted to the inline storage representation. > The biggest advantage to doing this via specialization is that if you do come up with a better way to represent a generic type > with a given type arg, the user of such instantiations doesn't need to change their code. I was going to call you on this before -- claiming "flexibility" by having a very limited number of types specially coded for JDK Collections -- with the requirement for M * N custom code sections, M being the # of collection implementations and N being the number of types you specially support -- is not really very flexible. You're trading off the need for use-site awareness, for the need for collection-implementation awareness :) Regards, Thomas From vitalyd at gmail.com Sat Dec 19 06:16:44 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Sat, 19 Dec 2015 01:16:44 -0500 Subject: sentinels & who owns the value space Re: valhalla-dev Digest, Vol 17, Issue 10 In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9B3@GSCMAMP06EX.firmwide.corp.gs.com> <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9C9@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: I also fail to see how Moh's example generalizes over arbitrary user types. That technique works well for integrals and in particular for sets. If anything, this is an example of where specialization would be applicable. How would T.sentinel work? I'm not quite getting it. What's the difference between it and T.default? How do I create multiple sentinels? Can you sketch it out a bit more? On Saturday, December 19, 2015, Thomas W wrote: > Great example. Thanks for being specific, Mohammad. > > Your example illustrates the use of additional storage (the > *zeroToThirtyOne* field, etc) as an auxilliary to provide sufficient > representation to distinguish known sentinels from valid client values. > > I would agree that this design might be harder to apply generically to > arbitrary value types, but I'd presume it works for any integral types > having >2 values (it doesn't exactly fail for boolean, representation > become entirely auxilliary though). > > My thinking on Valhalla has emphasized "abstracting" the difference > between types (reftypes, primitives & value types) and allowing existing > idiom & usage to be expressed in an abstract way; my feeling being that > building capable abstractions over 'any' type, would give a stronger > foundation that needing to lean on programmatic specialization. > > I previously proposed having the language offer both "T.default" and > "T.sentinel" as distinct values -- to represent the two distinct uses of > null better in eg. integer space. Null can be a default and 0 is a good > default; null can also be a sentinel, but Integer.MIN_VALUE is a much > better sentinel than 0 :) > > I guess what I'm working to is, that having the language provide two > abstracted values -- 'default' and 'sentinel' -- which are distinct, could > be used as the two cell sentinels for your hashset & special-cased in the > add/ contains check etc. > > This would be extensible to value-types by the component fields being set > to default/ or sentinel appropriately. The only problem is ref-types, which > in my proposal would have both RefType.default == null and RefType.sentinel > == null; and would thus not be amenable to providing your two separate cell > markers. > > Thanks for your thoughts. > > Regards, > Thomas > -- Sent from my phone From vitalyd at gmail.com Sat Dec 19 06:23:21 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Sat, 19 Dec 2015 01:23:21 -0500 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: On Saturday, December 19, 2015, Thomas W wrote: > Thanks Vitaly. > > If Optional is the goal, perhaps the question might be to inline/ > specialize the storage of monadic/ "wrapper types" generally? > I don't see why types/wrappers over one other type are special. Optional is special in that when it abstracts over a ref type the semantic of the type can be fulfilled naturally by virtue of ref types already having an absence value: null. The main point is you can exploit type info when it's provided. That's kind of the point of specialization. > There are probably a range of monadic/ wrapper types having one "major > field" and decorated it with simple behavior. However relying on handcoding > magic specializations in ArrayList, will restrict it to applying to > Optional only. > > An alternative way of looking at this is to to designate Optional as > "inline value" which then would enable a _generalized system_ to > arbitrarily reorganize its storage and store/materialize it as necessary. > While stored it would not be in standard representation and inlined > accessors would be adapted to the inline storage representation. > > > The biggest advantage to doing this via specialization is that if you do > come up with a better way to represent a generic type > > with a given type arg, the user of such instantiations doesn't need to > change their code. > > I was going to call you on this before -- claiming "flexibility" by having > a very limited number of types specially coded for JDK Collections -- with > the requirement for M * N custom code sections, M being the # of collection > implementations and N being the number of types you specially support -- is > not really very flexible. > > You're trading off the need for use-site awareness, for the need for > collection-implementation awareness :) > > How is this different from creating named types for all those specializations? It's always preferable to put more burden on lib author than its users, when given the choice. > Regards, > Thomas > > -- Sent from my phone From boaznahum at gmail.com Sat Dec 19 21:04:58 2015 From: boaznahum at gmail.com (Boaz Nahum) Date: Sat, 19 Dec 2015 21:04:58 +0000 Subject: Build failure on windows 'freetypeScaler.c' In-Reply-To: <56744FE4.4030404@oracle.com> References: <56744FE4.4030404@oracle.com> Message-ID: I pull the last update, since then, when running running 'make images' all calls to '.../build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe' end with 'Segmentation fault' Thx Boaz /usr/bin/bash: line 1: 9888 Segmentation fault /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -c /cygdrive/f/Dev/JDKs/8/jdk1.8.0_20/bin/java -Xms32M -Xmx512M -cp /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/buildtools/jdk_tools_classes build.tools.spp.Spp -Kint -KintegralType -K -Dtype=int -DType=Int -Dfulltype=integer -DFulltype=Integer -Dstreamtype= -DStreamtype= -Dx=i -Dmemtype=int -DMemtype=Int -DSwaptype=Int -DfromBits= -DtoBits= -DLG_BYTES_PER_VALUE=2 -DBYTES_PER_VALUE="(1 << 2)" -DBO=B -Dswap= -DRW=R -Kro -Da=an -DA=An -KboB < /cygdrive/f/Dev/JDKBuild/valhalla/jdk/src/java.base/share/classes/java/nio/ByteBufferAs-X-Buffer.java.template > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsIntBufferRB.java.tmpGensrcBuffer.gmk:354: recipe for target '/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsIntBufferRB.java' failed make[3]: *** [/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsIntBufferRB.java] Error 139 make[3]: *** Waiting for unfinished jobs.... /usr/bin/bash: line 1: 2996 Segmentation fault /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -c /cygdrive/f/Dev/JDKs/8/jdk1.8.0_20/bin/java -Xms32M -Xmx512M -cp /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/buildtools/jdk_tools_classes build.tools.spp.Spp -Klong -KintegralType -K -Dtype=long -DType=Long -Dfulltype=long -DFulltype=Long -Dstreamtype= -DStreamtype= -Dx=l -Dmemtype=long -DMemtype=Long -DSwaptype=Long -DfromBits= -DtoBits= -DLG_BYTES_PER_VALUE=3 -DBYTES_PER_VALUE="(1 << 3)" -DBO=B -Dswap= -DRW= -Krw -Da=a -DA=A -KboB < /cygdrive/f/Dev/JDKBuild/valhalla/jdk/src/java.base/share/classes/java/nio/ByteBufferAs-X-Buffer.java.template > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferB.java.tmp GensrcBuffer.gmk:355: recipe for target '/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferB.java' failedmake[3]: *** [/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferB.java] Error 139 GensrcBuffer.gmk:356: recipe for target '/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferRB.java' failed /usr/bin/bash: line 1: 10376 Segmentation fault /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe -c /cygdrive/f/Dev/JDKs/8/jdk1.8.0_20/bin/java -Xms32M -Xmx512M -cp /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/buildtools/jdk_tools_classes build.tools.spp.Spp -Klong -KintegralType -K -Dtype=long -DType=Long -Dfulltype=long -DFulltype=Long -Dstreamtype= -DStreamtype= -Dx=l -Dmemtype=long -DMemtype=Long -DSwaptype=Long -DfromBits= -DtoBits= -DLG_BYTES_PER_VALUE=3 -DBYTES_PER_VALUE="(1 << 3)" -DBO=B -Dswap= -DRW=R -Kro -Da=a -DA=A -KboB < /cygdrive/f/Dev/JDKBuild/valhalla/jdk/src/java.base/share/classes/java/nio/ByteBufferAs-X-Buffer.java.template > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferRB.java.tmp make[3]: *** [/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferRB.java] Error 139 make/Main.gmk:103: recipe for target 'java.base-gensrc-jdk' failedmake[2]: *** [java.base-gensrc-jdk] Error 1 On Fri, Dec 18, 2015 at 8:26 PM Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > Fix pushed > > Maurizio > > On 18/12/15 13:16, Boaz Nahum wrote: > > Hi > > > > Lately, I have this error: > > > > > f:/Dev/JDKBuild/valhalla/jdk/src/java.desktop/share/native/libfontmanager/freetypeScaler.c(106) > > : error C2220: warning treated as error - no 'object' file generated > > > f:/Dev/JDKBuild/valhalla/jdk/src/java.desktop/share/native/libfontmanager/freetypeScaler.c(106) > > : warning C4101: 'stream' : unreferenced local variable > > > > I comment out: > > > > static void freeNativeResources(JNIEnv *env, FTScalerInfo* scalerInfo) { > > // void *stream; > > > > > > And now it works. > > > > Thx > > Boaz > > From maurizio.cimadamore at oracle.com Tue Dec 22 14:59:07 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 22 Dec 2015 14:59:07 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: add initial support for alternate generic method translation scheme Message-ID: <201512221459.tBMEx7oL019917@aojmv0008.oracle.com> Changeset: a1457d0bfa9e Author: mcimadamore Date: 2015-12-22 14:58 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/a1457d0bfa9e Enhancement: add initial support for alternate generic method translation scheme + src/java.base/share/classes/java/lang/invoke/DispatchContext.java + src/java.base/share/classes/java/lang/invoke/GenericInstanceDispatch.java + src/java.base/share/classes/java/lang/invoke/GenericStaticDispatch.java ! src/java.base/share/classes/valhalla/specializer/Specializer.java From maurizio.cimadamore at oracle.com Tue Dec 22 14:59:54 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 22 Dec 2015 14:59:54 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: add initial support for alternate generic method translation scheme Message-ID: <201512221459.tBMExsvV020187@aojmv0008.oracle.com> Changeset: 2bb582ce72c4 Author: mcimadamore Date: 2015-12-22 14:59 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/2bb582ce72c4 Enhancement: add initial support for alternate generic method translation scheme ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symtab.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java + test/tools/javac/valhalla/typespec/GenericMethod01.java From maurizio.cimadamore at oracle.com Tue Dec 22 19:12:20 2015 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 22 Dec 2015 19:12:20 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: do not generate desugared generic method class for abstract methods Message-ID: <201512221912.tBMJCLlo003778@aojmv0008.oracle.com> Changeset: dddf65a2e118 Author: mcimadamore Date: 2015-12-22 19:11 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/dddf65a2e118 Fix: do not generate desugared generic method class for abstract methods Fix: deal with 'this'/'super' references from desugared class ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java From twhitmore.nz at gmail.com Thu Dec 24 23:35:32 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Fri, 25 Dec 2015 12:35:32 +1300 Subject: sentinels & who owns the value space Re: valhalla-dev Digest, Vol 17, Issue 10 In-Reply-To: References: <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9B3@GSCMAMP06EX.firmwide.corp.gs.com> <6882C9A35DFB9B4FA2779F7BF5B9757D2087E8D9C9@GSCMAMP06EX.firmwide.corp.gs.com> Message-ID: Hi Vitaly, I've tended to approach "type" as something generic specialization should abstract us above. Mentally I believe it would be best to get us much mileage as possible out of abstraction -- being able to code today's idioms, but abstracted above the level of null/ reftypes/ primitives to generic understandings of what these are being used for. T.default provides an abstracted generic "default value"; - zero or null, etc. - for Value Types, 'default' answers a value with all fields set to FieldType.default. T.sentinel similarly would provide an abstracted generic "sentinel value'; - as distinct as reasonably possible, from normally-used values of that type. - for integrals, it would eg. Integer.MIN_VALUE, Byte.MIN_VALUE; - for floats, perhaps NaN or minus NaN; - for boolean, there is no ideal sentinel; but if we use 'true', algorithms such as Moh's will still operate correctly. - for Value Types, 'sentinel' answers a value with all fields set to FieldType.sentinel. Thus far we have defined T.default and T.sentinel as distinct values, for all primitive types & value types composed of primitives, in such a way that "open addressed" hash structures (Moh's example" are able to represent both "empty" and "removed" cells. At this point the approach has not answered how to provide a distinct sentinel for ref-types (other than 'null'). However being able to code Moh's algorithm in the abstract -- no need for specialized code -- such that it works over all primitive types & all value types composed of primitive types, is immensely useful. This enables us to cover vast amounts of the problem space/ type universe -- With just a single class & two 10-character syntaxes, providing abstract value representation! Sentinels are something that has real mileage at the concrete level, and thus would be an omission not to be "abstract over" in code at the generic level. The obvious further question, would be whether it is possible to provide a distinct sentinel for ref-types other than 'null'. The expression would have to be efficient & usable for all possible reftypes. There are perhaps a couple of approaches: 1) similar to today's "weakly typed" reftype generics, use a single Object SENTINEL global constant & defer type-checking; 2) factor efficient 'sentinel' and 'default' type-support into some TypeFactory, referenced by the collection. 3) (probably theoretical only) define a "bottom type" with one concrete value. Special-case would be needed in the JVM to handle it. See: https://en.wikipedia.org/wiki/Bottom_type Regards, Thomas From twhitmore.nz at gmail.com Fri Dec 25 00:31:06 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Fri, 25 Dec 2015 13:31:06 +1300 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: Hi Vitaly, > I don't see why types/wrappers over one other type are special. Optional is special in that when it abstracts over a ref type the > semantic of the type can be fulfilled naturally by virtue of ref types already having an absence value: null. The main point is you > can exploit type info when it's provided. That's kind of the point of specialization. Well, when you're talking about specializing ArrayList I keep feeling that ArrayList> is much more useful -- in terms of anybody using it -- than ArrayList. Am I wrong in this? Monads/ wrappers such as Optional are probably an "obvious" case to optimize, since they're at the same cardinality & logical level as the underlying data; but currently add significant instance, indirection, wrapping & storage cost. If we want to go to lengths & consider tackling storage size & efficiency, why not tackle it generically at the storage level -- with a system for "unpacking" value types into arrays -- rather than by specializing at the Collection implementation (eg. ArrayList) level? If we want to address storage, array[] is where storage of pluralities mostly happens. Why not target that directly? I could propose a "packedarray" type, which would implement (either at specialization or at VM level) as an array for each component field. Store & retrieve would be automatically packed & unpacked into it. We would then have: public class ArrayList { packedarray T[] elements; int size; } Store & retrieve from 'elements' would inline the packing & unpacking of elements. Functionality (at least some) needed for operations within the collection would also be inlined -- eg. inlined versions of equals() and hashCode() method would be available, to operate directly onto the packed layout. There'd be some heuristic that the value-type was small & simple enough to benefit from this -- I'd suggest this would apply for value-types with no more than 4 fields for now. Larger value-types would just get stored flat in a single array. If we are trying to target storage, I think general approaches are worth considering vs. code-level specialization. I'd far prefer to consider boolean -> single-bit storage in a generic "packed array" that's implemented once, which everyone -- including all user types implemented subsequently to the library -- get to use automatically, than need to code that stuff M*N times (M per-Collection, N per Element Type) in the library. > How is this different from creating named types for all those specializations? It's always preferable to put more burden on lib > author than its users, when given the choice. Nice intent, but placing support in the library means it's not extensible & it's far less useful. The type instantiation site is outside the library -- if somebody needs a special collection for their type, that can be instantiated out there too. If we require specialized sections of code in the library, it means only Java core types can get that special support -- and probably not many of them. Essentially other people's small types & wrappers become second-class citizens -- ArrayList will offer bets support only for JDK value types, etc. Many many great libraries such as Guava, Apache, Spring, Hibernate etc are leaders in innovation. They implemented features & types the Java platform needed, long before Java were aware they needed it :) They have also led with great OO design and innovation, often when the core platform sorely lacked it. Numerous times Java has followed the third-party & gained from this innovation. The Java platform & ecosystem need to support third-party libraries -- including their types -- as first-class citizens. M*N implementation costs and having efficient support only for "core Java" stuff makes it all seem a bit limited, in my view. Regards, Thomas From boaznahum at gmail.com Fri Dec 25 12:26:14 2015 From: boaznahum at gmail.com (Boaz Nahum) Date: Fri, 25 Dec 2015 12:26:14 +0000 Subject: Unable to run _ByValue example Message-ID: (last pull valhalla today) Trying this: public final __ByValue class Point { final int x; final int y; Point(int x, int y) { this.x = x; this.y = y; } public static void main(String[] args) { Point p = __Make Point(5, 6); } } (Compile file directly with bin/javac.exe not using IntelliJ) When trying to run: F:\Dev\JDKBuild\valhalla\build\windows-x86_64-normal-server-release\images\jdk\bin\java -cp F:\Dev\IntelliJProjectsOut\LearnJDK10\production\LearnJDK10 by_val.Point java.lang.ClassFormatError: Method "" in class by_val/Point has illegal signature "(II)Qby_val/Point;" at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:759) at java.lang.ClassLoader.defineClass(ClassLoader.java:835) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:184) at java.net.URLClassLoader.defineClass(URLClassLoader.java:547) at java.net.URLClassLoader.access$100(URLClassLoader.java:87) at java.net.URLClassLoader$1.run(URLClassLoader.java:413) at java.net.URLClassLoader$1.run(URLClassLoader.java:397) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:396) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:262) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:503) Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" Process finished with exit code 1 Thx Boaz From srikanth.adayapalam at oracle.com Mon Dec 28 05:00:56 2015 From: srikanth.adayapalam at oracle.com (Srikanth) Date: Mon, 28 Dec 2015 10:30:56 +0530 Subject: Unable to run _ByValue example In-Reply-To: References: Message-ID: <5680C208.8080208@oracle.com> Hello Boaz, This is a known issue - ATM, the VM does not yet handle the changes to signatures that embed 'Q' in them to denote value types and it may be some time before the VM support for this is fully there. We are exploring some interim options to allow interested early access testers to experiment with the usability aspects of the value type model (knowing that attendant performance improvements are yet to come) Stay tuned. Thanks! Srikanth On Friday 25 December 2015 05:56 PM, Boaz Nahum wrote: > (last pull valhalla today) > Trying this: > > public final __ByValue class Point { > final int x; > final int y; > Point(int x, int y) { > this.x = x; > this.y = y; > } > > public static void main(String[] args) { > Point p = __Make Point(5, 6); > } > } > > (Compile file directly with bin/javac.exe not using IntelliJ) > > When trying to run: > > F:\Dev\JDKBuild\valhalla\build\windows-x86_64-normal-server-release\images\jdk\bin\java > -cp F:\Dev\IntelliJProjectsOut\LearnJDK10\production\LearnJDK10 > by_val.Point > java.lang.ClassFormatError: Method "" in class by_val/Point > has illegal signature "(II)Qby_val/Point;" > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:759) > at java.lang.ClassLoader.defineClass(ClassLoader.java:835) > at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:184) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:547) > at java.net.URLClassLoader.access$100(URLClassLoader.java:87) > at java.net.URLClassLoader$1.run(URLClassLoader.java:413) > at java.net.URLClassLoader$1.run(URLClassLoader.java:397) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:396) > at java.lang.ClassLoader.loadClass(ClassLoader.java:423) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:262) > at java.lang.ClassLoader.loadClass(ClassLoader.java:356) > at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:503) > Error: A JNI error has occurred, please check your installation and try again > Exception in thread "main" > Process finished with exit code 1 > > Thx > Boaz From peter.levart at gmail.com Mon Dec 28 07:17:36 2015 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 28 Dec 2015 08:17:36 +0100 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: <5680E210.5090605@gmail.com> Hi, It seems this thread has not mentioned an existing tool used to abstract-away the selection of a particular implementation class - the good old static factory method. See EnumSet for an example. Valhalla has hooks to make the selection possible at runtime given the instantiated type parameters: public abstract class PackedList implements List { public static PackedList newInstance() { if (E.class == boolean.class) { return new BooleanBitPackedList(); } else { return new DefaultPackedList(); } } private static class BooleanBitPackedList extends PackedList { ... } private static class DefaultPackedList extends PackedList { ... } } Regards, Peter On 12/25/2015 01:31 AM, Thomas W wrote: > Hi Vitaly, > >> I don't see why types/wrappers over one other type are special. Optional > is special in that when it abstracts over a ref type the >> semantic of the type can be fulfilled naturally by virtue of ref types > already having an absence value: null. The main point is you >> can exploit type info when it's provided. That's kind of the point of > specialization. > > Well, when you're talking about specializing ArrayList I keep > feeling that ArrayList> is much more useful -- in terms of > anybody using it -- than ArrayList. Am I wrong in this? > > Monads/ wrappers such as Optional are probably an "obvious" case to > optimize, since they're at the same cardinality & logical level as the > underlying data; but currently add significant instance, indirection, > wrapping & storage cost. > > If we want to go to lengths & consider tackling storage size & efficiency, > why not tackle it generically at the storage level -- with a system for > "unpacking" value types into arrays -- rather than by specializing at the > Collection implementation (eg. ArrayList) level? > > If we want to address storage, array[] is where storage of pluralities > mostly happens. Why not target that directly? > > I could propose a "packedarray" type, which would implement (either at > specialization or at VM level) as an array for each component field. Store > & retrieve would be automatically packed & unpacked into it. > > We would then have: > > public class ArrayList { > packedarray T[] elements; > int size; > } > > Store & retrieve from 'elements' would inline the packing & unpacking of > elements. Functionality (at least some) needed for operations within the > collection would also be inlined -- eg. inlined versions of equals() and > hashCode() method would be available, to operate directly onto the packed > layout. > > There'd be some heuristic that the value-type was small & simple enough to > benefit from this -- I'd suggest this would apply for value-types with no > more than 4 fields for now. Larger value-types would just get stored flat > in a single array. > > If we are trying to target storage, I think general approaches are worth > considering vs. code-level specialization. > > I'd far prefer to consider boolean -> single-bit storage in a generic > "packed array" that's implemented once, which everyone -- including all > user types implemented subsequently to the library -- get to use > automatically, than need to code that stuff M*N times (M per-Collection, N > per Element Type) in the library. > >> How is this different from creating named types for all those > specializations? It's always preferable to put more burden on lib >> author than its users, when given the choice. > Nice intent, but placing support in the library means it's not extensible & > it's far less useful. The type instantiation site is outside the library -- > if somebody needs a special collection for their type, that can be > instantiated out there too. > > If we require specialized sections of code in the library, it means only > Java core types can get that special support -- and probably not many of > them. Essentially other people's small types & wrappers become second-class > citizens -- ArrayList will offer bets support only for JDK value types, etc. > > Many many great libraries such as Guava, Apache, Spring, Hibernate etc are > leaders in innovation. They implemented features & types the Java platform > needed, long before Java were aware they needed it :) They have also led > with great OO design and innovation, often when the core platform sorely > lacked it. Numerous times Java has followed the third-party & gained from > this innovation. > > The Java platform & ecosystem need to support third-party libraries -- > including their types -- as first-class citizens. M*N implementation costs > and having efficient support only for "core Java" stuff makes it all seem a > bit limited, in my view. > > > Regards, > Thomas From vitalyd at gmail.com Mon Dec 28 16:14:25 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 28 Dec 2015 11:14:25 -0500 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: <5680E210.5090605@gmail.com> References: <5680E210.5090605@gmail.com> Message-ID: On Monday, December 28, 2015, Peter Levart wrote: > Hi, > > It seems this thread has not mentioned an existing tool used to > abstract-away the selection of a particular implementation class - the good > old static factory method. See EnumSet for an example. I briefly mentioned factories earlier in this thread. They can work but have limitations: 1) you need to define them upfront if you want to avoid users changing their code later on. If you don't know of a specialization a priori, should you define a factory method just in case? Are constructors 2nd class citizens now? 2) your example below won't even compile as-is? You'd need to add a cast to the BooleanBitPackedList to return it as the generic type. And this is ugly as we now have generic code mixed with specialized code. > > Valhalla has hooks to make the selection possible at runtime given the > instantiated type parameters: > > public abstract class PackedList implements List { > > public static PackedList newInstance() { > if (E.class == boolean.class) { > return new BooleanBitPackedList(); > } else { > return new DefaultPackedList(); > } > } > > private static class BooleanBitPackedList extends PackedList > { ... } > > private static class DefaultPackedList extends PackedList { > ... } > } > > > Regards, Peter > > > On 12/25/2015 01:31 AM, Thomas W wrote: > >> Hi Vitaly, >> >> I don't see why types/wrappers over one other type are special. Optional >>> >> is special in that when it abstracts over a ref type the >> >>> semantic of the type can be fulfilled naturally by virtue of ref types >>> >> already having an absence value: null. The main point is you >> >>> can exploit type info when it's provided. That's kind of the point of >>> >> specialization. >> >> Well, when you're talking about specializing ArrayList I keep >> feeling that ArrayList> is much more useful -- in terms of >> anybody using it -- than ArrayList. Am I wrong in this? >> >> Monads/ wrappers such as Optional are probably an "obvious" case to >> optimize, since they're at the same cardinality & logical level as the >> underlying data; but currently add significant instance, indirection, >> wrapping & storage cost. >> >> If we want to go to lengths & consider tackling storage size & efficiency, >> why not tackle it generically at the storage level -- with a system for >> "unpacking" value types into arrays -- rather than by specializing at the >> Collection implementation (eg. ArrayList) level? >> >> If we want to address storage, array[] is where storage of pluralities >> mostly happens. Why not target that directly? >> >> I could propose a "packedarray" type, which would implement (either at >> specialization or at VM level) as an array for each component field. Store >> & retrieve would be automatically packed & unpacked into it. >> >> We would then have: >> >> public class ArrayList { >> packedarray T[] elements; >> int size; >> } >> >> Store & retrieve from 'elements' would inline the packing & unpacking of >> elements. Functionality (at least some) needed for operations within the >> collection would also be inlined -- eg. inlined versions of equals() and >> hashCode() method would be available, to operate directly onto the packed >> layout. >> >> There'd be some heuristic that the value-type was small & simple enough to >> benefit from this -- I'd suggest this would apply for value-types with no >> more than 4 fields for now. Larger value-types would just get stored flat >> in a single array. >> >> If we are trying to target storage, I think general approaches are worth >> considering vs. code-level specialization. >> >> I'd far prefer to consider boolean -> single-bit storage in a generic >> "packed array" that's implemented once, which everyone -- including all >> user types implemented subsequently to the library -- get to use >> automatically, than need to code that stuff M*N times (M per-Collection, N >> per Element Type) in the library. >> >> How is this different from creating named types for all those >>> >> specializations? It's always preferable to put more burden on lib >> >>> author than its users, when given the choice. >>> >> Nice intent, but placing support in the library means it's not extensible >> & >> it's far less useful. The type instantiation site is outside the library >> -- >> if somebody needs a special collection for their type, that can be >> instantiated out there too. >> >> If we require specialized sections of code in the library, it means only >> Java core types can get that special support -- and probably not many of >> them. Essentially other people's small types & wrappers become >> second-class >> citizens -- ArrayList will offer bets support only for JDK value types, >> etc. >> >> Many many great libraries such as Guava, Apache, Spring, Hibernate etc are >> leaders in innovation. They implemented features & types the Java platform >> needed, long before Java were aware they needed it :) They have also led >> with great OO design and innovation, often when the core platform sorely >> lacked it. Numerous times Java has followed the third-party & gained from >> this innovation. >> >> The Java platform & ecosystem need to support third-party libraries -- >> including their types -- as first-class citizens. M*N implementation >> costs >> and having efficient support only for "core Java" stuff makes it all seem >> a >> bit limited, in my view. >> >> >> Regards, >> Thomas >> > > -- Sent from my phone From vitalyd at gmail.com Mon Dec 28 16:24:58 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 28 Dec 2015 11:24:58 -0500 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: On Thursday, December 24, 2015, Thomas W wrote: > Hi Vitaly, > > > I don't see why types/wrappers over one other type are special. > Optional is special in that when it abstracts over a ref type the > > semantic of the type can be fulfilled naturally by virtue of ref types > already having an absence value: null. The main point is you > > can exploit type info when it's provided. That's kind of the point of > specialization. > > Well, when you're talking about specializing ArrayList I keep > feeling that ArrayList> is much more useful -- in terms of > anybody using it -- than ArrayList. Am I wrong in this? > Not sure which is more useful as they're different concepts (contiguous list of booleans vs optional Ts). > Monads/ wrappers such as Optional are probably an "obvious" case to > optimize, since they're at the same cardinality & logical level as the > underlying data; but currently add significant instance, indirection, > wrapping & storage cost. > > If we want to go to lengths & consider tackling storage size & efficiency, > why not tackle it generically at the storage level -- with a system for > "unpacking" value types into arrays -- rather than by specializing at the > Collection implementation (eg. ArrayList) level? > > If we want to address storage, array[] is where storage of pluralities > mostly happens. Why not target that directly? > > I could propose a "packedarray" type, which would implement (either at > specialization or at VM level) as an array for each component field. Store > & retrieve would be automatically packed & unpacked into it. > > We would then have: > > public class ArrayList { > packedarray T[] elements; > int size; > } > > Store & retrieve from 'elements' would inline the packing & unpacking of > elements. Functionality (at least some) needed for operations within the > collection would also be inlined -- eg. inlined versions of equals() and > hashCode() method would be available, to operate directly onto the packed > layout. > > > There'd be some heuristic that the value-type was small & simple enough to > benefit from this -- I'd suggest this would apply for value-types with no > more than 4 fields for now. Larger value-types would just get stored flat > in a single array. > > By packing you mean storing the elements inline with the enclosing class? If so, yes I'd like to be able to do that as well. Inline/packed arrays would be great for other reasons, but I don't know if that's Valhalla or Panama or not even on the table entirely. Either way, I think it's orthogonal to supporting specialization. > If we are trying to target storage, I think general approaches are worth > considering vs. code-level specialization. > > I'd far prefer to consider boolean -> single-bit storage in a generic > "packed array" that's implemented once, which everyone -- including all > user types implemented subsequently to the library -- get to use > automatically, than need to code that stuff M*N times (M per-Collection, N > per Element Type) in the library. > > > How is this different from creating named types for all those > specializations? It's always preferable to put more burden on lib > > author than its users, when given the choice. > > Nice intent, but placing support in the library means it's not extensible > & it's far less useful. The type instantiation site is outside the library > -- if somebody needs a special collection for their type, that can be > instantiated out there too. > > If we require specialized sections of code in the library, it means only > Java core types can get that special support -- and probably not many of > them. Essentially other people's small types & wrappers become second-class > citizens -- ArrayList will offer bets support only for JDK value types, etc. > > Many many great libraries such as Guava, Apache, Spring, Hibernate etc are > leaders in innovation. They implemented features & types the Java platform > needed, long before Java were aware they needed it :) They have also led > with great OO design and innovation, often when the core platform sorely > lacked it. Numerous times Java has followed the third-party & gained from > this innovation. > > The Java platform & ecosystem need to support third-party libraries -- > including their types -- as first-class citizens. M*N implementation costs > and having efficient support only for "core Java" stuff makes it all seem a > bit limited, in my view. > Specialization would allow author of the class to provide transparent and natural (I.e. Use constructor and not factory) specialization for the types they know. If they don't know the type, they can't specialize. If you, as a user, know more types and how to specialize them then use existing specialization facilities. > Regards, > Thomas > > -- Sent from my phone From twhitmore.nz at gmail.com Thu Dec 31 02:47:25 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Thu, 31 Dec 2015 15:47:25 +1300 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: <5680E210.5090605@gmail.com> Message-ID: Hi Peter, Vitaly, people Thanks for raising "Factories" -- I'd been thinking about these too. I guess I am really questioning whether "making Classes into multiple possible things" is really preferable to "asking a Factory Method to create the appropriate thing". Vitaly -- your objection to making constructors into 2nd-class citizens -- by introducing all kinds of variance within the class, we are running the risk of degrading what was single & clear (the concept of class) into its own 2nd-class citizen. If we haven't used factory methods eg. for List.newArrayList() before, perhaps we should start now? - Google Guava API use factory methods successfully; even for current Java, it seems a good design. - and for our purposes, it provides an ideal place to locate specialization; - *outside* of a single class (specific implementation) and able to despatch to type-specific alternate specialized implementations, as required. The main limitation with factory methods is multiple dimensions of type-specificity; eg. separate specialiations on Key and Value of a Map, for example. Template-style specialization (as Vitaly is currently working on) can handle that better, but I believe this to be a very much smaller proportion of the real world problem-space. Regards, Thomas From twhitmore.nz at gmail.com Thu Dec 31 03:19:25 2015 From: twhitmore.nz at gmail.com (Thomas W) Date: Thu, 31 Dec 2015 16:19:25 +1300 Subject: valhalla-dev Digest, Vol 17, Issue 3 In-Reply-To: References: Message-ID: Hi Vitaly, [ArrayList being perhaps a stalking horse for > ArrayList>] > > Not sure which is more useful as they're different concepts (contiguous list of booleans vs optional Ts). We should be measuring potential design decisions in cost vs benefit, and doing that requires some reasonable understanding of how application/ and framework programming will use them. We shouldn't talk about ArrayList as though it were something that should be done at all costs. So far, I've not hearing much/ or anything supporting its importance; therefore I don't agree that we can meaningfully use this as a "would like", "should" or "must have" requirement to drive design. I'm suggesting this discussion should either put some flesh on this strawman or (preferably) move on to some real usecases for storage/ or other specialization, so we can undertake meaningful cost/ benefit consideration. > Monads/ wrappers such as Optional are probably an "obvious" case to > optimize, since they're at the same cardinality & logical level as the > underlying data; but currently add significant instance, indirection, > wrapping & storage cost. > > If we want to go to lengths & consider tackling storage size & efficiency, > why not tackle it generically at the storage level -- with a system for > "unpacking" value types into arrays -- rather than by specializing at the > Collection implementation (eg. ArrayList) level? > > By packing you mean storing the elements inline with the enclosing class? If so, yes I'd like to be able to do that as well. > Inline/packed arrays would be great for other reasons, but I don't know if that's Valhalla or Panama or not even on the table entirely. > Either way, I think it's orthogonal to supporting specialization. Not a single array inline with the enclosing class -- that would lead to a variable-size object, which I don't think is wanted.What I'm proposing is multiple arrays (one per field) with the Value broken into its component fields and stored that way. eg. given class Vector {int x, y, z;} the Collection (say, ArrayList) would declared "packedarray T[] elements;" packedarray would be implemented in the JVM as: packedarray { int[] elements_x; int[] elements_y; } My point is that if we want to target *storage*, we can target it at the *Array level* -- below the Class. I question why we're trying to specialize storage at the Class level, at all? If we do it in the JVM array implementation -- perhaps by defining a new 'packedarray' or whatever -- this would obviate a lot of the need for template-style class specialization. > Specialization would allow author of the class to provide transparent and natural (I.e. Use constructor and not factory) specialization for > the types they know. I don't agree that different code & behavior within a single named Class can or should be assumed to be "natural". This changes the core Class concept, which has been singular throughout the history of Java. There are issues of quality & meaning, in regard to the whole cohesiveness and meaningfulness of the Class concept. I would want to see some really good cost/ benefit to justify this approach. > If they don't know the type, they can't specialize. If you, as a user, know more types and how to specialize them then use existing > specialization facilities Yes, that's my point -- specialization from such a location is limited, making it of less value. I would outline a reasonable conservative approach as follows: 1) "generic/ abstract" specialization from a single code source; 2) comparions/ null checks/ sentinels etc coded in the abstract and implemented appropriate to the type at specialization time. 3) Factory Methods and distinct implementation classes (eg. BooleanBitPackedList) for anything more. The question would be, what real-world benefits are we looking to gain, when considering the substantial cost & potential loss of conceptual integrity associated with template-style specialization? Regards, Thomas From magnus.ihse.bursie at oracle.com Fri Dec 18 13:28:44 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 18 Dec 2015 13:28:44 -0000 Subject: Build failure on windows 'freetypeScaler.c' In-Reply-To: References: Message-ID: <56740A06.1070803@oracle.com> On 2015-12-18 14:16, Boaz Nahum wrote: > Hi > > Lately, I have this error: > > f:/Dev/JDKBuild/valhalla/jdk/src/java.desktop/share/native/libfontmanager/freetypeScaler.c(106) > : error C2220: warning treated as error - no 'object' file generated > f:/Dev/JDKBuild/valhalla/jdk/src/java.desktop/share/native/libfontmanager/freetypeScaler.c(106) > : warning C4101: 'stream' : unreferenced local variable > > I comment out: > > static void freeNativeResources(JNIEnv *env, FTScalerInfo* scalerInfo) { This is a know problem which is fixed by JDK-8139803. /Magnus From magnus.ihse.bursie at oracle.com Sun Dec 20 00:02:07 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Sun, 20 Dec 2015 00:02:07 -0000 Subject: Build failure on windows 'freetypeScaler.c' In-Reply-To: References: <56744FE4.4030404@oracle.com> Message-ID: <5675EFFB.9060208@oracle.com> On 2015-12-19 22:04, Boaz Nahum wrote: > I pull the last update, since then, when running running 'make images' all > calls to > '.../build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe' > end with 'Segmentation fault' If you have build problems with no local changes that can have caused it, the sledgehammers to use (in increasing brutality) are: 1) make clean 2) make dist-clean This will remove your configuration. "make print-configuration" before will give you the command line you used for configure, if you need to be reminded of it. 3) remove the forest and re-clone it. My guess is that your problems can be solved by 1), but if not, try with a bigger hammer. /Magnus > Thx > Boaz > > /usr/bin/bash: line 1: 9888 Segmentation fault > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe > -c /cygdrive/f/Dev/JDKs/8/jdk1.8.0_20/bin/java -Xms32M -Xmx512M -cp > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/buildtools/jdk_tools_classes > build.tools.spp.Spp -Kint -KintegralType -K -Dtype=int -DType=Int > -Dfulltype=integer -DFulltype=Integer -Dstreamtype= -DStreamtype= -Dx=i > -Dmemtype=int -DMemtype=Int -DSwaptype=Int -DfromBits= -DtoBits= > -DLG_BYTES_PER_VALUE=2 -DBYTES_PER_VALUE="(1 << 2)" -DBO=B -Dswap= -DRW=R > -Kro -Da=an -DA=An -KboB < > /cygdrive/f/Dev/JDKBuild/valhalla/jdk/src/java.base/share/classes/java/nio/ByteBufferAs-X-Buffer.java.template > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsIntBufferRB.java.tmpGensrcBuffer.gmk:354: > recipe for target > '/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsIntBufferRB.java' > failed > > make[3]: *** > [/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsIntBufferRB.java] > Error 139 > make[3]: *** Waiting for unfinished jobs.... > /usr/bin/bash: line 1: 2996 Segmentation fault > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe > -c /cygdrive/f/Dev/JDKs/8/jdk1.8.0_20/bin/java -Xms32M -Xmx512M -cp > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/buildtools/jdk_tools_classes > build.tools.spp.Spp -Klong -KintegralType -K -Dtype=long -DType=Long > -Dfulltype=long -DFulltype=Long -Dstreamtype= -DStreamtype= -Dx=l > -Dmemtype=long -DMemtype=Long -DSwaptype=Long -DfromBits= -DtoBits= > -DLG_BYTES_PER_VALUE=3 -DBYTES_PER_VALUE="(1 << 3)" -DBO=B -Dswap= -DRW= > -Krw -Da=a -DA=A -KboB < > /cygdrive/f/Dev/JDKBuild/valhalla/jdk/src/java.base/share/classes/java/nio/ByteBufferAs-X-Buffer.java.template > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferB.java.tmp > GensrcBuffer.gmk:355: recipe for target > '/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferB.java' > failedmake[3]: *** > [/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferB.java] > Error 139 > > GensrcBuffer.gmk:356: recipe for target > '/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferRB.java' > failed > /usr/bin/bash: line 1: 10376 Segmentation fault > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe > -c /cygdrive/f/Dev/JDKs/8/jdk1.8.0_20/bin/java -Xms32M -Xmx512M -cp > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/buildtools/jdk_tools_classes > build.tools.spp.Spp -Klong -KintegralType -K -Dtype=long -DType=Long > -Dfulltype=long -DFulltype=Long -Dstreamtype= -DStreamtype= -Dx=l > -Dmemtype=long -DMemtype=Long -DSwaptype=Long -DfromBits= -DtoBits= > -DLG_BYTES_PER_VALUE=3 -DBYTES_PER_VALUE="(1 << 3)" -DBO=B -Dswap= -DRW=R > -Kro -Da=a -DA=A -KboB < > /cygdrive/f/Dev/JDKBuild/valhalla/jdk/src/java.base/share/classes/java/nio/ByteBufferAs-X-Buffer.java.template > /cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferRB.java.tmp > make[3]: *** > [/cygdrive/f/Dev/JDKBuild/valhalla/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBufferAsLongBufferRB.java] > Error 139 > make/Main.gmk:103: recipe for target 'java.base-gensrc-jdk' failedmake[2]: > *** [java.base-gensrc-jdk] Error 1 > > > On Fri, Dec 18, 2015 at 8:26 PM Maurizio Cimadamore < > maurizio.cimadamore at oracle.com> wrote: > >> Fix pushed >> >> Maurizio >> >> On 18/12/15 13:16, Boaz Nahum wrote: >>> Hi >>> >>> Lately, I have this error: >>> >>> >> f:/Dev/JDKBuild/valhalla/jdk/src/java.desktop/share/native/libfontmanager/freetypeScaler.c(106) >>> : error C2220: warning treated as error - no 'object' file generated >>> >> f:/Dev/JDKBuild/valhalla/jdk/src/java.desktop/share/native/libfontmanager/freetypeScaler.c(106) >>> : warning C4101: 'stream' : unreferenced local variable >>> >>> I comment out: >>> >>> static void freeNativeResources(JNIEnv *env, FTScalerInfo* scalerInfo) { >>> // void *stream; >>> >>> >>> And now it works. >>> >>> Thx >>> Boaz >>