From vicente.romero at oracle.com Thu Mar 1 04:48:16 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Thu, 01 Mar 2018 04:48:16 +0000 Subject: hg: amber/amber: update source and target options at BuildNashorn.gmk plus generating condy for source 11 only Message-ID: <201803010448.w214mG4C023110@aojmv0008.oracle.com> Changeset: e51a71b90343 Author: vromero Date: 2018-02-28 23:43 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/e51a71b90343 update source and target options at BuildNashorn.gmk plus generating condy for source 11 only ! make/BuildNashorn.gmk ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Source.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java From vicente.romero at oracle.com Thu Mar 1 14:01:39 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Thu, 1 Mar 2018 09:01:39 -0500 Subject: CFV: New Amber Committer: Jim Laskey In-Reply-To: <1db33dd8-a8b9-a0bd-b95a-f2e2cfe31a71@oracle.com> References: <1db33dd8-a8b9-a0bd-b95a-f2e2cfe31a71@oracle.com> Message-ID: <1f768af9-7369-941d-c9fb-77873382bf4a@oracle.com> Voting for Jim Laskey [1] is now closed. Yes: 8 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. Thanks, Vicente [1] http://mail.openjdk.java.net/pipermail/amber-dev/2018-February/002643.html On 02/15/2018 11:07 AM, Vicente Romero wrote: > I hereby nominate Jim Laskey (jlaskey) to Amber Committer. > > Jim is a member of the langtools team who has joined Project Amber to > lead the raw string literals project [3]. A list of his most recent > OpenJDK contributions is available at [4]. > > Votes are due by March 01, 2018. > > Only current Amber Committers [1] are eligible to vote on this > nomination. > > For Lazy Consensus voting instructions, see [2]. > > Thank you, > Vicente Romero > > [1]http://openjdk.java.net/census > [2]http://openjdk.java.net/projects/#committer-vote > [3]https://bugs.openjdk.java.net/browse/JDK-8196004 > [4]http://hg.openjdk.java.net/jdk/jdk/search/?rev=author(jlaskey) From james.laskey at oracle.com Thu Mar 1 16:13:58 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 1 Mar 2018 12:13:58 -0400 Subject: Survey on Developer Experience with String Literals In-Reply-To: References: <37277A8E-678D-4159-923D-FF0DA3113341@oracle.com> Message-ID: <50BA0FF9-27F0-4A18-887A-073E3610F518@oracle.com> Survey results: http://cr.openjdk.java.net/~jlaskey/surveys/Java%20String%20Literals%20Survey.pdf > On Feb 22, 2018, at 3:17 PM, Jim Laskey wrote: > > Please use https://www.surveymonkey.com/r/CJQM2VG . Oracle mail seems to be blocking the survey coming in from Survey Monkey. > > ? Jim > > >> On Feb 22, 2018, at 3:09 PM, Jim Laskey > wrote: >> >> >> Java String Literals >> >> >> >> We're conducting a survey and your input would be appreciated. Click the button below to start the survey. Thank you for your participation! >> >> We will publish the survey results when the survey period is complete. >> >> >> >> >> Begin Survey > >> >> >> Please do not forward this email as its survey link is unique to you. >> Unsubscribe > from this list >> >> >> >> Powered by >> > From cyrill.brunner at hispeed.ch Thu Mar 1 18:16:19 2018 From: cyrill.brunner at hispeed.ch (Cyrill Brunner) Date: Thu, 1 Mar 2018 19:16:19 +0100 Subject: multiline: Must we waste one of the final few 'free' symbols on this? In-Reply-To: <2006197156.999354.1519824278118.JavaMail.zimbra@u-pem.fr> References: <2006197156.999354.1519824278118.JavaMail.zimbra@u-pem.fr> Message-ID: <682345c9-9317-4eb9-e1dd-ad55de00e97e@hispeed.ch> The points raised in that discussion are valid reasons for not using fixed, possibly single-letter delimiters for raw strings, yes. But for the second suggestion made, would it not also be a possible to use 3+, symmetric double quotes? It would leave the use case of `while` etc. open whilst producing none of the detriments mentioned in the JEP directly. So instead of String text = ``This contains a backtick: `.`` you could similarly do String text = """"This string contains a triple quote: """."""" This would leave ` untouched for now, just as string"prefixes", whilst still allowing the arbitrary-but-symmetric number of double quotes. What would speak against this? - Cyrill Brunner Am 28.02.2018 um 14:24 schrieb Remi Forax: > see http://mail.openjdk.java.net/pipermail/amber-spec-experts/2018-February/000286.html > > R?mi > > ----- Mail original ----- >> De: "Reinier Zwitserloot" >> ?: "amber-dev" >> Envoy?: Mercredi 28 F?vrier 2018 07:06:20 >> Objet: multiline: Must we waste one of the final few 'free' symbols on this? >> Some feedback on multiline string literals. Where 'proposal' is referenced, >> it refers to: https://bugs.openjdk.java.net/browse/JDK-8196004 >> >> # Must we waste one of the final few 'free' symbols on this? # >> >> If you look at all easily accessible symbols on a keyboard, the only ones >> that don't yet have a syntactic meaning in java source files are the >> backtick and the hash. Everything else is either defined to be an >> identifierpart which makes using them as a symbol somewhat difficult >> (that'd be the underscore and the dollar, although the underscore has >> already backwards-incompatibly been torn out; presumably the dollar can be >> 'rescued' in the same fashion). Is THIS what we're going to spend one of >> our final 2 to 3 symbols on? >> >> One obvious alternate use for the backtick is for encoding identifiers; if >> you want to name a method "while", which the JVM spec does allow you to do, >> you could maybe one day use backticks. Some JVM-targeted languages already >> do this. I'm not saying this is a good idea, but I am saying that >> implementing the raw string literal proposal as written pretty much >> eliminates this notion from ever seeing the light of day, forever. Perhaps >> it's worth some debate before we just casually close that door in >> perpetuity. >> >> alternatives: >> >> Is: R"This is a raw string" an option? An advantage to the 'R' concept is >> that you can separate 'escapes arent processed' ('raw') from 'feel free to >> newline in these' ('multiline'): The R indicates raw, and hitting enter >> immediately after the quote indicates multiline, which would be backwards >> compatible as currently its always illegal java if you newline in the >> middle of string literals. Thus: >> >> String x = R"Escapes \t are not processed here; this contains raw >> backslash-t instead of a tab"; >> String multi = " >> This is >> multiline but \t DOES contain a tab"; >> String rawMulti = R" >> This is >> multi with \t backslash-t literally, not a tab"; >> >> Another option would be to investigate the use of triple quotes. In java9 >> syntax, having 3 quotes in immediate succession cannot possibly be valid in >> a source file unless in a comment. Therefore, it would seem possible to use >> triple quotes as a delimiter without creating the ambiguity mentioned in >> the 'Choice of delimiters' section. Example: >> >> String regex = """Hey now I don't have to \w+ escape my backslashes!"""; >> >> This syntax also has quite a lot of precedence (kotlin, swift, groovy, and >> python). Note that the 'other languages' section misconstrues how python >> works; triple quotes is for multiline strings. For raw strings, you use >> R"foo". Most python programmers seem to think the R stands for regex, as >> that's pretty much what they're always used for. Nevertheless, it stands >> for 'raw'. See: >> https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals >> >> In regards to investigating simply allowing java strings to contain >> newlines; the 'Choice of delimiters' section has this quote: >> >>> Enabling such a feature would affect tools and tests that assume >> multi-line traditional string literals as an error. >> >> This makes no sense. Any unupdated tool would consider use of a backtick >> also an error. Either way, tools not aware of the new feature would treat >> multiline string literals as a syntax error, whether you use backtick, >> quote, or triple-quote. Unlike the introduction of very fancy footwork to >> treat backslash-u escapes as raw inside these literals, addition of >> backtick (or triple quote, or single quote) as signifying raw and/or >> multiline strings won't be particularly difficult for existing java parsers >> to implement. It doesn't seem relevant as an argument for or against any >> particular delimiter. >> >> --Reinier Zwitserloot From brian.goetz at oracle.com Thu Mar 1 18:49:51 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 1 Mar 2018 13:49:51 -0500 Subject: multiline: Must we waste one of the final few 'free' symbols on this? In-Reply-To: <682345c9-9317-4eb9-e1dd-ad55de00e97e@hispeed.ch> References: <2006197156.999354.1519824278118.JavaMail.zimbra@u-pem.fr> <682345c9-9317-4eb9-e1dd-ad55de00e97e@hispeed.ch> Message-ID: It would surely be possible; the JEP refers to several existence proofs, and approaches like this were considered. I appreciate that you are concerned with the stewardship concern of managing our limited syntactic real estate.? We of course pay considerable attention to this concern as well.? Like any finite resource, you want to spend it carefully, but you also don't want to hoard it and never spend it.? Always a tough choice. As to who would speak against it: I think probably almost everyone, at least in one aspect: it's pretty heavy, often unnecessarily so.? For strings not containing single backticks (which already describes the vast majority of string literals), you don't need double or triple anything: ??? String s = `I'm a raw string`; Many users feel that the triple (or more) quotes are about as subtle as a hammer to the head.? Here, you only need to trot out the triple-quotes here if your embedded text has both single and double quotes in it already. Is it perfect?? Of course not.? Is the """ approach obviously better?? Also not.? Such is the nature of syntax selection; there's a lot of room for personal choice. On 3/1/2018 1:16 PM, Cyrill Brunner wrote: > The points raised in that discussion are valid reasons for not using > fixed, possibly single-letter delimiters for raw strings, yes. > But for the second suggestion made, would it not also be a possible to > use 3+, symmetric double quotes? It would leave the use case of > `while` etc. open whilst producing none of the detriments mentioned in > the JEP directly. > > So instead of > > String text = ``This contains a backtick: `.`` > > you could similarly do > > String text = """"This string contains a triple quote: """."""" > > This would leave ` untouched for now, just as string"prefixes", whilst > still allowing the arbitrary-but-symmetric number of double quotes. > > What would speak against this? > > - Cyrill Brunner > > > Am 28.02.2018 um 14:24 schrieb Remi Forax: >> see >> http://mail.openjdk.java.net/pipermail/amber-spec-experts/2018-February/000286.html >> >> R?mi >> >> ----- Mail original ----- >>> De: "Reinier Zwitserloot" >>> ?: "amber-dev" >>> Envoy?: Mercredi 28 F?vrier 2018 07:06:20 >>> Objet: multiline: Must we waste one of the final few 'free' symbols >>> on this? >>> Some feedback on multiline string literals. Where 'proposal' is >>> referenced, >>> it refers to: https://bugs.openjdk.java.net/browse/JDK-8196004 >>> >>> # Must we waste one of the final few 'free' symbols on this? # >>> >>> If you look at all easily accessible symbols on a keyboard, the only >>> ones >>> that don't yet have a syntactic meaning in java source files are the >>> backtick and the hash. Everything else is either defined to be an >>> identifierpart which makes using them as a symbol somewhat difficult >>> (that'd be the underscore and the dollar, although the underscore has >>> already backwards-incompatibly been torn out; presumably the dollar >>> can be >>> 'rescued' in the same fashion). Is THIS what we're going to spend >>> one of >>> our final 2 to 3 symbols on? >>> >>> One obvious alternate use for the backtick is for encoding >>> identifiers; if >>> you want to name a method "while", which the JVM spec does allow you >>> to do, >>> you could maybe one day use backticks. Some JVM-targeted languages >>> already >>> do this. I'm not saying this is a good idea, but I am saying that >>> implementing the raw string literal proposal as written pretty much >>> eliminates this notion from ever seeing the light of day, forever. >>> Perhaps >>> it's worth some debate before we just casually close that door in >>> perpetuity. >>> >>> alternatives: >>> >>> Is: R"This is a raw string" an option? An advantage to the 'R' >>> concept is >>> that you can separate 'escapes arent processed' ('raw') from 'feel >>> free to >>> newline in these' ('multiline'): The R indicates raw, and hitting enter >>> immediately after the quote indicates multiline, which would be >>> backwards >>> compatible as currently its always illegal java if you newline in the >>> middle of string literals. Thus: >>> >>> String x = R"Escapes \t are not processed here; this contains raw >>> backslash-t instead of a tab"; >>> String multi = " >>> ??? This is >>> ??? multiline but \t DOES contain a tab"; >>> String rawMulti = R" >>> ??? This is >>> ??? multi with \t backslash-t literally, not a tab"; >>> >>> Another option would be to investigate the use of triple quotes. In >>> java9 >>> syntax, having 3 quotes in immediate succession cannot possibly be >>> valid in >>> a source file unless in a comment. Therefore, it would seem possible >>> to use >>> triple quotes as a delimiter without creating the ambiguity >>> mentioned in >>> the 'Choice of delimiters' section. Example: >>> >>> String regex = """Hey now I don't have to \w+ escape my >>> backslashes!"""; >>> >>> This syntax also has quite a lot of precedence (kotlin, swift, >>> groovy, and >>> python). Note that the 'other languages' section misconstrues how >>> python >>> works; triple quotes is for multiline strings. For raw strings, you use >>> R"foo". Most python programmers seem to think the R stands for >>> regex, as >>> that's pretty much what they're always used for. Nevertheless, it >>> stands >>> for 'raw'. See: >>> https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals >>> >>> >>> In regards to investigating simply allowing java strings to contain >>> newlines; the 'Choice of delimiters' section has this quote: >>> >>>> Enabling such a feature would affect tools and tests that assume >>> multi-line traditional string literals as an error. >>> >>> This makes no sense. Any unupdated tool would consider use of a >>> backtick >>> also an error. Either way, tools not aware of the new feature would >>> treat >>> multiline string literals as a syntax error, whether you use backtick, >>> quote, or triple-quote. Unlike the introduction of very fancy >>> footwork to >>> treat backslash-u escapes as raw inside these literals, addition of >>> backtick (or triple quote, or single quote) as signifying raw and/or >>> multiline strings won't be particularly difficult for existing java >>> parsers >>> to implement. It doesn't seem relevant as an argument for or against >>> any >>> particular delimiter. >>> >>> ? --Reinier Zwitserloot > From brian.goetz at oracle.com Thu Mar 1 19:29:46 2018 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Thu, 01 Mar 2018 19:29:46 +0000 Subject: hg: amber/amber: Remove SymbolicRef, ConstantRef.{OfSelf,WithTypeDDescriptor}; make {Class,MethodType,MethodHandle}Ref into interfaces; create concrete classes for Constant{Class,MethodType,MethodHandle}Ref; add support for method handle combinators Message-ID: <201803011929.w21JTl86017425@aojmv0008.oracle.com> Changeset: 1903c35f59dc Author: briangoetz Date: 2018-03-01 14:25 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/1903c35f59dc Remove SymbolicRef, ConstantRef.{OfSelf,WithTypeDDescriptor}; make {Class,MethodType,MethodHandle}Ref into interfaces; create concrete classes for Constant{Class,MethodType,MethodHandle}Ref; add support for method handle combinators ! src/java.base/share/classes/java/lang/Class.java ! src/java.base/share/classes/java/lang/Double.java ! src/java.base/share/classes/java/lang/Enum.java ! src/java.base/share/classes/java/lang/Float.java ! src/java.base/share/classes/java/lang/Integer.java ! src/java.base/share/classes/java/lang/Long.java ! src/java.base/share/classes/java/lang/String.java ! src/java.base/share/classes/java/lang/invoke/MethodHandle.java ! src/java.base/share/classes/java/lang/invoke/MethodType.java ! src/java.base/share/classes/java/lang/invoke/VarHandle.java ! src/java.base/share/classes/java/lang/invoke/X-VarHandle.java.template + src/java.base/share/classes/java/lang/sym/AsTypeMethodHandleRef.java ! src/java.base/share/classes/java/lang/sym/ClassRef.java ! src/java.base/share/classes/java/lang/sym/Constable.java + src/java.base/share/classes/java/lang/sym/ConstantClassRef.java + src/java.base/share/classes/java/lang/sym/ConstantMethodHandleRef.java + src/java.base/share/classes/java/lang/sym/ConstantMethodTypeRef.java ! src/java.base/share/classes/java/lang/sym/ConstantRef.java + src/java.base/share/classes/java/lang/sym/ConstantRefs.java ! src/java.base/share/classes/java/lang/sym/DynamicCallSiteRef.java ! src/java.base/share/classes/java/lang/sym/DynamicConstantRef.java ! src/java.base/share/classes/java/lang/sym/EnumRef.java ! src/java.base/share/classes/java/lang/sym/MethodHandleRef.java ! src/java.base/share/classes/java/lang/sym/MethodTypeRef.java + src/java.base/share/classes/java/lang/sym/PrimitiveClassRef.java - src/java.base/share/classes/java/lang/sym/SymbolicRef.java - src/java.base/share/classes/java/lang/sym/SymbolicRefs.java ! src/java.base/share/classes/java/lang/sym/VarHandleRef.java ! src/java.base/share/classes/java/lang/sym/package-info.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/util/Constables.java ! test/jdk/java/lang/invoke/ConstantRefBootstrapsTest.java ! test/jdk/java/lang/sym/ClassRefTest.java ! test/jdk/java/lang/sym/CondyRefTest.java ! test/jdk/java/lang/sym/IntrinsifiedRefTest.java ! test/jdk/java/lang/sym/MethodHandleRefTest.java ! test/jdk/java/lang/sym/MethodTypeRefTest.java ! test/jdk/java/lang/sym/SymbolicRefTest.java ! test/langtools/tools/javac/condy/CheckForCondyDuplicatesTest.java ! test/langtools/tools/javac/specialConstantFolding/CheckForCorrectMRefTest.java ! test/langtools/tools/javac/specialConstantFolding/CondyCodeGenerationTest.java ! test/langtools/tools/javac/specialConstantFolding/DontCompileIfSymbolCantBeFoundTest.java ! test/langtools/tools/javac/specialConstantFolding/EffectivelyFinalTestNeg.java ! test/langtools/tools/javac/specialConstantFolding/IndyCodeGenerationTest.java ! test/langtools/tools/javac/specialConstantFolding/IndyCrashTest.java ! test/langtools/tools/javac/specialConstantFolding/IndyLinkageErrorTest.java ! test/langtools/tools/javac/specialConstantFolding/IndyNegativeTest01.java ! test/langtools/tools/javac/specialConstantFolding/IndyPositiveTest01.java ! test/langtools/tools/javac/specialConstantFolding/IntrinsicsTest.java ! test/langtools/tools/javac/specialConstantFolding/LDCNegativeTest.java ! test/langtools/tools/javac/specialConstantFolding/MultipleBSMEntriesTest.java ! test/langtools/tools/javac/specialConstantFolding/ReportIncorrectMHForIndyTest.java ! test/langtools/tools/javac/specialConstantFolding/TwoVisitsAreNeededCauseOfForwardRefTest.java ! test/langtools/tools/javac/specialConstantFolding/checkMethodTypeShape/MethodTypeNegTest.java ! test/langtools/tools/javac/specialConstantFolding/checkMethodTypeShape/MethodTypeNegTest.out ! test/langtools/tools/javac/specialConstantFolding/harness/tests/ConstantDefinitions.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/ConstantFoldingOfMethodTypeDiffContextsTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindConstructorTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindMethodWithGenericArgumentsTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindSetterTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindStaticGetterTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindStaticSetterTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindVirtualTest01.java ! test/langtools/tools/javac/specialConstantFolding/warningNotFoundOrIncorrect/WarningIfClassOrMemberNotFound3.java ! test/langtools/tools/javac/specialConstantFolding/warningNotFoundOrIncorrect/WarningIfMemberIncorrect.java From forax at univ-mlv.fr Thu Mar 1 19:29:48 2018 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Thu, 1 Mar 2018 20:29:48 +0100 (CET) Subject: multiline: Must we waste one of the final few 'free' symbols on this? In-Reply-To: <682345c9-9317-4eb9-e1dd-ad55de00e97e@hispeed.ch> References: <2006197156.999354.1519824278118.JavaMail.zimbra@u-pem.fr> <682345c9-9317-4eb9-e1dd-ad55de00e97e@hispeed.ch> Message-ID: <50592765.1548073.1519932588486.JavaMail.zimbra@u-pem.fr> Hi Cyrill, you can even use 2+ single quotes instead of 3+ double quotes. To answer to the question, "maybe one day exotic identifiers may want to use backticks" is not an argument for good reasons: - you can use the same symbol for both, it depends exactly where you allow exotic identifiers, by example if it's for exported symbols; name of methods, fields, classes that are visible outside; there is no problem because you can consider raw string as local identifier the same way the grammar considers 'module' or 'exports' as a keyword depending on the context. - we can use """ as you suggest (or '') to implement exotic identifiers. - as ma grand'ma was saying when i was starting my sentence with 'what if', the future paralyse only the fool :) regards, R?mi ----- Mail original ----- > De: "Cyrill Brunner" > ?: "Remi Forax" , "Reinier Zwitserloot" > Cc: "amber-dev" > Envoy?: Jeudi 1 Mars 2018 19:16:19 > Objet: Re: multiline: Must we waste one of the final few 'free' symbols on this? > The points raised in that discussion are valid reasons for not using fixed, > possibly single-letter delimiters for raw strings, yes. > But for the second suggestion made, would it not also be a possible to use 3+, > symmetric double quotes? It would leave the use case of `while` etc. open > whilst producing none of the detriments mentioned in the JEP directly. > > So instead of > > String text = ``This contains a backtick: `.`` > > you could similarly do > > String text = """"This string contains a triple quote: """."""" > > This would leave ` untouched for now, just as string"prefixes", whilst still > allowing the arbitrary-but-symmetric number of double quotes. > > What would speak against this? > > - Cyrill Brunner > > > Am 28.02.2018 um 14:24 schrieb Remi Forax: >> see >> http://mail.openjdk.java.net/pipermail/amber-spec-experts/2018-February/000286.html >> >> R?mi >> >> ----- Mail original ----- >>> De: "Reinier Zwitserloot" >>> ?: "amber-dev" >>> Envoy?: Mercredi 28 F?vrier 2018 07:06:20 >>> Objet: multiline: Must we waste one of the final few 'free' symbols on this? >>> Some feedback on multiline string literals. Where 'proposal' is referenced, >>> it refers to: https://bugs.openjdk.java.net/browse/JDK-8196004 >>> >>> # Must we waste one of the final few 'free' symbols on this? # >>> >>> If you look at all easily accessible symbols on a keyboard, the only ones >>> that don't yet have a syntactic meaning in java source files are the >>> backtick and the hash. Everything else is either defined to be an >>> identifierpart which makes using them as a symbol somewhat difficult >>> (that'd be the underscore and the dollar, although the underscore has >>> already backwards-incompatibly been torn out; presumably the dollar can be >>> 'rescued' in the same fashion). Is THIS what we're going to spend one of >>> our final 2 to 3 symbols on? >>> >>> One obvious alternate use for the backtick is for encoding identifiers; if >>> you want to name a method "while", which the JVM spec does allow you to do, >>> you could maybe one day use backticks. Some JVM-targeted languages already >>> do this. I'm not saying this is a good idea, but I am saying that >>> implementing the raw string literal proposal as written pretty much >>> eliminates this notion from ever seeing the light of day, forever. Perhaps >>> it's worth some debate before we just casually close that door in >>> perpetuity. >>> >>> alternatives: >>> >>> Is: R"This is a raw string" an option? An advantage to the 'R' concept is >>> that you can separate 'escapes arent processed' ('raw') from 'feel free to >>> newline in these' ('multiline'): The R indicates raw, and hitting enter >>> immediately after the quote indicates multiline, which would be backwards >>> compatible as currently its always illegal java if you newline in the >>> middle of string literals. Thus: >>> >>> String x = R"Escapes \t are not processed here; this contains raw >>> backslash-t instead of a tab"; >>> String multi = " >>> This is >>> multiline but \t DOES contain a tab"; >>> String rawMulti = R" >>> This is >>> multi with \t backslash-t literally, not a tab"; >>> >>> Another option would be to investigate the use of triple quotes. In java9 >>> syntax, having 3 quotes in immediate succession cannot possibly be valid in >>> a source file unless in a comment. Therefore, it would seem possible to use >>> triple quotes as a delimiter without creating the ambiguity mentioned in >>> the 'Choice of delimiters' section. Example: >>> >>> String regex = """Hey now I don't have to \w+ escape my backslashes!"""; >>> >>> This syntax also has quite a lot of precedence (kotlin, swift, groovy, and >>> python). Note that the 'other languages' section misconstrues how python >>> works; triple quotes is for multiline strings. For raw strings, you use >>> R"foo". Most python programmers seem to think the R stands for regex, as >>> that's pretty much what they're always used for. Nevertheless, it stands >>> for 'raw'. See: >>> https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals >>> >>> In regards to investigating simply allowing java strings to contain >>> newlines; the 'Choice of delimiters' section has this quote: >>> >>>> Enabling such a feature would affect tools and tests that assume >>> multi-line traditional string literals as an error. >>> >>> This makes no sense. Any unupdated tool would consider use of a backtick >>> also an error. Either way, tools not aware of the new feature would treat >>> multiline string literals as a syntax error, whether you use backtick, >>> quote, or triple-quote. Unlike the introduction of very fancy footwork to >>> treat backslash-u escapes as raw inside these literals, addition of >>> backtick (or triple quote, or single quote) as signifying raw and/or >>> multiline strings won't be particularly difficult for existing java parsers >>> to implement. It doesn't seem relevant as an argument for or against any >>> particular delimiter. >>> > >> --Reinier Zwitserloot From brian.goetz at oracle.com Thu Mar 1 19:33:54 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 1 Mar 2018 14:33:54 -0500 Subject: multiline: Must we waste one of the final few 'free' symbols on this? In-Reply-To: <50592765.1548073.1519932588486.JavaMail.zimbra@u-pem.fr> References: <2006197156.999354.1519824278118.JavaMail.zimbra@u-pem.fr> <682345c9-9317-4eb9-e1dd-ad55de00e97e@hispeed.ch> <50592765.1548073.1519932588486.JavaMail.zimbra@u-pem.fr> Message-ID: <1fd3344c-6475-c192-ae78-e1f5a970e4be@oracle.com> And, to close the syntactic-real-estate-stewardship loop: Using ` for exotic identifiers has a far worse return-on-syntax than using it for string literals.? The former will only be used by 0.1% of developers, 0.1% of the time; the latter are used by every developer, all of the time.? One of the reasons we didn't use ` for exotic identifiers is that we felt the return-on-syntax was too poor, so we saved it for something better to come along.? And this is that something better. On 3/1/2018 2:29 PM, forax at univ-mlv.fr wrote: > Hi Cyrill, > you can even use 2+ single quotes instead of 3+ double quotes. > > To answer to the question, "maybe one day exotic identifiers may want to use backticks" is not an argument for good reasons: > - you can use the same symbol for both, it depends exactly where you allow exotic identifiers, by example if it's for exported symbols; name of methods, fields, classes that are visible outside; there is no problem because you can consider raw string as local identifier the same way the grammar considers 'module' or 'exports' as a keyword depending on the context. > - we can use """ as you suggest (or '') to implement exotic identifiers. > - as ma grand'ma was saying when i was starting my sentence with 'what if', the future paralyse only the fool :) > > regards, > R?mi > > ----- Mail original ----- >> De: "Cyrill Brunner" >> ?: "Remi Forax" , "Reinier Zwitserloot" >> Cc: "amber-dev" >> Envoy?: Jeudi 1 Mars 2018 19:16:19 >> Objet: Re: multiline: Must we waste one of the final few 'free' symbols on this? >> The points raised in that discussion are valid reasons for not using fixed, >> possibly single-letter delimiters for raw strings, yes. >> But for the second suggestion made, would it not also be a possible to use 3+, >> symmetric double quotes? It would leave the use case of `while` etc. open >> whilst producing none of the detriments mentioned in the JEP directly. >> >> So instead of >> >> String text = ``This contains a backtick: `.`` >> >> you could similarly do >> >> String text = """"This string contains a triple quote: """."""" >> >> This would leave ` untouched for now, just as string"prefixes", whilst still >> allowing the arbitrary-but-symmetric number of double quotes. >> >> What would speak against this? >> >> - Cyrill Brunner >> >> >> Am 28.02.2018 um 14:24 schrieb Remi Forax: >>> see >>> http://mail.openjdk.java.net/pipermail/amber-spec-experts/2018-February/000286.html >>> >>> R?mi >>> >>> ----- Mail original ----- >>>> De: "Reinier Zwitserloot" >>>> ?: "amber-dev" >>>> Envoy?: Mercredi 28 F?vrier 2018 07:06:20 >>>> Objet: multiline: Must we waste one of the final few 'free' symbols on this? >>>> Some feedback on multiline string literals. Where 'proposal' is referenced, >>>> it refers to: https://bugs.openjdk.java.net/browse/JDK-8196004 >>>> >>>> # Must we waste one of the final few 'free' symbols on this? # >>>> >>>> If you look at all easily accessible symbols on a keyboard, the only ones >>>> that don't yet have a syntactic meaning in java source files are the >>>> backtick and the hash. Everything else is either defined to be an >>>> identifierpart which makes using them as a symbol somewhat difficult >>>> (that'd be the underscore and the dollar, although the underscore has >>>> already backwards-incompatibly been torn out; presumably the dollar can be >>>> 'rescued' in the same fashion). Is THIS what we're going to spend one of >>>> our final 2 to 3 symbols on? >>>> >>>> One obvious alternate use for the backtick is for encoding identifiers; if >>>> you want to name a method "while", which the JVM spec does allow you to do, >>>> you could maybe one day use backticks. Some JVM-targeted languages already >>>> do this. I'm not saying this is a good idea, but I am saying that >>>> implementing the raw string literal proposal as written pretty much >>>> eliminates this notion from ever seeing the light of day, forever. Perhaps >>>> it's worth some debate before we just casually close that door in >>>> perpetuity. >>>> >>>> alternatives: >>>> >>>> Is: R"This is a raw string" an option? An advantage to the 'R' concept is >>>> that you can separate 'escapes arent processed' ('raw') from 'feel free to >>>> newline in these' ('multiline'): The R indicates raw, and hitting enter >>>> immediately after the quote indicates multiline, which would be backwards >>>> compatible as currently its always illegal java if you newline in the >>>> middle of string literals. Thus: >>>> >>>> String x = R"Escapes \t are not processed here; this contains raw >>>> backslash-t instead of a tab"; >>>> String multi = " >>>> This is >>>> multiline but \t DOES contain a tab"; >>>> String rawMulti = R" >>>> This is >>>> multi with \t backslash-t literally, not a tab"; >>>> >>>> Another option would be to investigate the use of triple quotes. In java9 >>>> syntax, having 3 quotes in immediate succession cannot possibly be valid in >>>> a source file unless in a comment. Therefore, it would seem possible to use >>>> triple quotes as a delimiter without creating the ambiguity mentioned in >>>> the 'Choice of delimiters' section. Example: >>>> >>>> String regex = """Hey now I don't have to \w+ escape my backslashes!"""; >>>> >>>> This syntax also has quite a lot of precedence (kotlin, swift, groovy, and >>>> python). Note that the 'other languages' section misconstrues how python >>>> works; triple quotes is for multiline strings. For raw strings, you use >>>> R"foo". Most python programmers seem to think the R stands for regex, as >>>> that's pretty much what they're always used for. Nevertheless, it stands >>>> for 'raw'. See: >>>> https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals >>>> >>>> In regards to investigating simply allowing java strings to contain >>>> newlines; the 'Choice of delimiters' section has this quote: >>>> >>>>> Enabling such a feature would affect tools and tests that assume >>>> multi-line traditional string literals as an error. >>>> >>>> This makes no sense. Any unupdated tool would consider use of a backtick >>>> also an error. Either way, tools not aware of the new feature would treat >>>> multiline string literals as a syntax error, whether you use backtick, >>>> quote, or triple-quote. Unlike the introduction of very fancy footwork to >>>> treat backslash-u escapes as raw inside these literals, addition of >>>> backtick (or triple quote, or single quote) as signifying raw and/or >>>> multiline strings won't be particularly difficult for existing java parsers >>>> to implement. It doesn't seem relevant as an argument for or against any >>>> particular delimiter. >>>> >>>> --Reinier Zwitserloot From mark.reinhold at oracle.com Fri Mar 2 17:35:25 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 2 Mar 2018 09:35:25 -0800 (PST) Subject: JEP 326: Raw String Literals Message-ID: <20180302173525.ED2FB179D0A@eggemoggin.niobe.net> New JEP Candidate: http://openjdk.java.net/jeps/326 - Mark From vicente.romero at oracle.com Fri Mar 2 19:38:09 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Fri, 02 Mar 2018 19:38:09 +0000 Subject: hg: amber/amber: 0000000: Updating library support Message-ID: <201803021938.w22Jc99k028836@aojmv0008.oracle.com> Changeset: ad490b781dac Author: jlaskey Date: 2018-03-02 14:33 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/ad490b781dac 0000000: Updating library support ! src/java.base/share/classes/java/lang/String.java ! src/java.base/share/classes/java/lang/StringLatin1.java ! src/java.base/share/classes/java/lang/StringUTF16.java ! test/jdk/java/lang/String/RawStringLiteralLib.java From cushon at google.com Sat Mar 3 01:03:36 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 2 Mar 2018 17:03:36 -0800 Subject: deduplicating lambda methods Message-ID: Hello, I'm interested in adding support for deduplicating lambda methods to javac. The idea is that if a compilation unit contains two lambdas that are identical (including any captured state and the functional interface they implement) we could re-use the same implementation method for both. I understand there might have been some prior discussion about this. Is there interest in investigating the feature? What sort of technical considerations have been identified so far? I have been thinking about a couple of questions: 1) How to identifying duplicates: I have a prototype that runs during lambda desugaring and identifies duplicates by diffing ASTs. Is that the best place for deduplication, or it worth considering comparing generated code instead of ASTs? 2) Debug info: the optimization is safe if line numbers are not being emitted. If they are, is there a way to deduplicate the methods without breaking debug info? Thanks, Liam From brian.goetz at oracle.com Sat Mar 3 01:17:19 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 2 Mar 2018 20:17:19 -0500 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: <99ce48ba-8df1-d155-0638-2ec9a76582d7@oracle.com> I think this is a great idea.? And it synergizes well with some other work we've got in the pipe. It's a shame that if you use the same lambda twice in the same source file, we desugar two separate lambda$nnn methods, and spin two separate lambda proxy classes.? Deduplicating the lambda$nnn methods will address the former; a separate effort, where we are using the new "constantdynamic" instead of "invokedynamic" to evaluate method refs and non-capturing lambdas, will address the latter (once the former is addressed.)? So this will make Java programs more efficient overall.? (The condy translation will get us deduplication for free for method references, but not for lambdas.) I think AST comparison is likely to be easier and more effective. And it doesn't have to be perfect; if it gets fooled by occasional differences, that's OK, as long as it doesn't merge lambdas that are actually different.? And you don't have the generated code until later, when its likely harder to do the merging.? There's a whole pass in the compiler pipeline for lambda method desugaring (LambdaToMethod), so there's an obvious place to do this transformation. Another consideration is serializable lambdas; the scheme for serializable lambdas involves a parallel generation path for deserialization.? I suspect that its probably best to just avoid serializable lambdas entirely, at least at first. Is there any trickiness with capturing lambdas?? I don't think so -- I think we can merge these too, although I suspect the return on that effort is lower.? I'll bet the most common case is lambdas like e -> e, x -> System.out.println(x), etc. I know you have some good tools at Google for codebase statistics.? Maybe you could pull together data on how often lambdas are duplicated within a source file, and of the duplicated lambdas, what percentage are stateless and non-serializable? On 3/2/2018 8:03 PM, Liam Miller-Cushon wrote: > Hello, > > I'm interested in adding support for deduplicating lambda methods to > javac. The idea is that if a compilation unit contains two lambdas > that are identical (including any captured state and the functional > interface they implement) we could re-use the same implementation > method for both. > > I understand there might have been some prior discussion about this. > Is there interest in investigating the feature? What sort of technical > considerations have been identified so far? > > I have been thinking about a couple of questions: > > 1) How to identifying duplicates: I have a prototype that runs during > lambda desugaring and identifies duplicates by diffing ASTs. Is that > the best place for deduplication, or it worth considering comparing > generated code instead of ASTs? > > 2) Debug info: the optimization is safe if line numbers are not being > emitted. If they are, is there a way to deduplicate the methods > without breaking debug info? > > Thanks, > Liam From john.r.rose at oracle.com Sat Mar 3 01:57:10 2018 From: john.r.rose at oracle.com (John Rose) Date: Fri, 2 Mar 2018 17:57:10 -0800 Subject: deduplicating lambda methods In-Reply-To: <99ce48ba-8df1-d155-0638-2ec9a76582d7@oracle.com> References: <99ce48ba-8df1-d155-0638-2ec9a76582d7@oracle.com> Message-ID: On Mar 2, 2018, at 5:17 PM, Brian Goetz wrote: > > Is there any trickiness with capturing lambdas? I don't think so -- I think we can merge these too, although I suspect the return on that effort is lower. I'll bet the most common case is lambdas like e -> e, x -> System.out.println(x), etc. A good extension to Liam's proposal, that should arise naturally from his AST matching work, would be to recognize the top ten (or so) AST patterns, like the above and also e -> k and k::m (for constants k). The desugared methods for these a trivial, and as such should deduplicated *across compilation units*. They could be defined as part of this work as new members of the LMF class. Another good extension to consider is to evaluate the body of a capturing (but non-constant) lambda and detect whether the lambda capture point can be moved out of a loop. That's tricky because you don't want to trigger a capture for a zero-trip loop. But it's worthwhile because users frequently write loop-invariant lambdas without realizing it. (The latter problem is one of de-duplication for duplicates created by loop iterations. So it's not de-duplication in the same sense. But it does require an AST analysis of a lambda body, in context of a containing loop.) ? John From john.r.rose at oracle.com Sat Mar 3 02:12:48 2018 From: john.r.rose at oracle.com (John Rose) Date: Fri, 2 Mar 2018 18:12:48 -0800 Subject: deduplicating lambda methods In-Reply-To: References: <99ce48ba-8df1-d155-0638-2ec9a76582d7@oracle.com> Message-ID: <82129B27-DACD-40DE-875E-54B2B42B0C62@oracle.com> On Mar 2, 2018, at 5:57 PM, John Rose wrote: > > e -> k and k::m (for constants k) Do'oh. I meant to say "captured variables k". They are only constants in the sense of being final. But *not* compile time constants. Brian's condy stuff handles true compile-time constants like Integer.MAX_VALUE. And it might use the same off-the-shelf shared desugar methods I mentioned before. I suppose I should give examples of what cross-CU method sharing. Suppose the source file contains a stereotyped lambda; pick a constant-returning one. We can lower it several ways to either a method reference, or a partial application (curry) of a method reference: () -> 42 LOWER==> Foo::lamb$42 where private static int lamb$42() { return 42; } // in local CU OR==> curry LMF::intConstant on (42) public static int intConstant(int x) { return x; } // in LMF OR==> curry Integer::intValue on (42) // use a found method OR==> something outside the LMF, with MethodHandles.constant() and invokeExact Perhaps the LMF can be used directly to do the above curries. ? John From brian.goetz at oracle.com Sat Mar 3 02:14:43 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 2 Mar 2018 21:14:43 -0500 Subject: deduplicating lambda methods In-Reply-To: References: <99ce48ba-8df1-d155-0638-2ec9a76582d7@oracle.com> Message-ID: > A good extension to Liam's proposal, that should arise naturally > from his AST matching work, would be to recognize the top ten > (or so) AST patterns, like the above and also e -> k and k::m > (for constants k). ?The desugared methods for these a trivial, > and as such should deduplicated *across compilation units*. > They could be defined as part of this work as new members > of the LMF class. This seems more like a candidate for a jlink-time optimization. From john.r.rose at oracle.com Sat Mar 3 02:22:59 2018 From: john.r.rose at oracle.com (John Rose) Date: Fri, 2 Mar 2018 18:22:59 -0800 Subject: deduplicating lambda methods In-Reply-To: References: <99ce48ba-8df1-d155-0638-2ec9a76582d7@oracle.com> Message-ID: <270852C9-640F-480B-8CEB-0D8D70DBA4EA@oracle.com> On Mar 2, 2018, at 6:14 PM, Brian Goetz wrote: > > This seems more like a candidate for a jlink-time optimization. Yes, good, although the very first time it is used in any large body like java.base it will discover the same basic top five. Those might as well be done unconditionally. In which case the first batch can be done without de ending on jlink. Anyway, these are further steps. What Liam is starting with is great. From cushon at google.com Sat Mar 3 03:00:49 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 2 Mar 2018 19:00:49 -0800 Subject: deduplicating lambda methods In-Reply-To: <270852C9-640F-480B-8CEB-0D8D70DBA4EA@oracle.com> References: <99ce48ba-8df1-d155-0638-2ec9a76582d7@oracle.com> <270852C9-640F-480B-8CEB-0D8D70DBA4EA@oracle.com> Message-ID: On Fri, Mar 2, 2018 at 5:17 PM, Brian Goetz wrote: > I think AST comparison is likely to be easier and more effective. And it > doesn't have to be perfect; if it gets fooled by occasional differences, > that's OK, as long as it doesn't merge lambdas that are actually > different. And you don't have the generated code until later, when its > likely harder to do the merging. There's a whole pass in the compiler > pipeline for lambda method desugaring (LambdaToMethod), so there's an > obvious place to do this transformation. > That's where I started for the prototype. The appeal of bytecode is that it's easier to implement value equality, and it supports deduplicating different source that lowers to the same bytecode. However the AST comparison is easy enough to support (just a little more verbose), and the return on handling non-identical duplicates seems fairly small. > Is there any trickiness with capturing lambdas? I don't think so -- I > think we can merge these too, although I suspect the return on that effort > is lower. I'll bet the most common case is lambdas like e -> e, x -> > System.out.println(x), etc. > I don't think so either, and depending on where in lambda desugaring the deduplication happens we might get this for free. > Maybe you could pull together data on how often lambdas are duplicated > within a source file, and of the duplicated lambdas, what percentage are > stateless and non-serializable? Will do, thanks. On Fri, Mar 2, 2018 at 6:22 PM, John Rose wrote: > On Mar 2, 2018, at 6:14 PM, Brian Goetz wrote: > > > > This seems more like a candidate for a jlink-time optimization. > > Yes, good, although the very first time it is used in any large body like > java.base it will discover the same basic top five. Those might as well be > done unconditionally. In which case the first batch can be done without de > ending on jlink. I'll see if I can collect some data about what the top N lambdas look like. From vicente.romero at oracle.com Sat Mar 3 04:01:47 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 2 Mar 2018 23:01:47 -0500 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: very interesting! On 03/02/2018 08:03 PM, Liam Miller-Cushon wrote: > Hello, > > I'm interested in adding support for deduplicating lambda methods to > javac. The idea is that if a compilation unit contains two lambdas > that are identical (including any captured state and the functional > interface they implement) we could re-use the same implementation > method for both. > > I understand there might have been some prior discussion about this. > Is there interest in investigating the feature? What sort of technical > considerations have been identified so far? > > I have been thinking about a couple of questions: > > 1) How to identifying duplicates: I have a prototype that runs during > lambda desugaring and identifies duplicates by diffing ASTs. Is that > the best place for deduplication, or it worth considering comparing > generated code instead of ASTs? are you doing an exact diff? I assume that we want: s -> s to be equal to z -> z provided that the target is the same > > 2) Debug info: the optimization is safe if line numbers are not being > emitted. If they are, is there a way to deduplicate the methods > without breaking debug info? I haven't tried to debug a method with more than one LNT, the spec allows it but not sure how IDEs will operate on that > > Thanks, > Liam Vicente From james.laskey at oracle.com Sat Mar 3 13:05:12 2018 From: james.laskey at oracle.com (james.laskey at oracle.com) Date: Sat, 03 Mar 2018 13:05:12 +0000 Subject: hg: amber/amber: 0000000: Update RawStringLiteralLib tests Message-ID: <201803031305.w23D5C3B018685@aojmv0008.oracle.com> Changeset: 64f68718c2f7 Author: jlaskey Date: 2018-03-03 09:00 -0400 URL: http://hg.openjdk.java.net/amber/amber/rev/64f68718c2f7 0000000: Update RawStringLiteralLib tests ! test/jdk/java/lang/String/RawStringLiteralLib.java From vicente.romero at oracle.com Sat Mar 3 21:59:54 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Sat, 03 Mar 2018 21:59:54 +0000 Subject: hg: amber/amber: consistency refactoring: updating Pool.DynamicVariable in the lines of Pool.DynamicMethod, additional renaming Message-ID: <201803032159.w23LxtY0004272@aojmv0008.oracle.com> Changeset: 677168216389 Author: vromero Date: 2018-03-03 16:54 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/677168216389 consistency refactoring: updating Pool.DynamicVariable in the lines of Pool.DynamicMethod, additional renaming ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/ConstablesVisitor.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Code.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/util/Constables.java From vicente.romero at oracle.com Sat Mar 3 22:23:10 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Sat, 03 Mar 2018 22:23:10 +0000 Subject: hg: amber/amber: check that no duplicate condys are generated for the same method reference Message-ID: <201803032223.w23MNA7W014278@aojmv0008.oracle.com> Changeset: b9d0b29a9e10 Author: vromero Date: 2018-03-03 17:15 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/b9d0b29a9e10 check that no duplicate condys are generated for the same method reference ! test/langtools/tools/javac/condy/CheckForCondyDuplicatesTest.java From bsrbnd at gmail.com Sun Mar 4 12:39:01 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 4 Mar 2018 13:39:01 +0100 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: On 3 March 2018 at 05:01, Vicente Romero wrote: > very interesting! > > On 03/02/2018 08:03 PM, Liam Miller-Cushon wrote: >> >> Hello, >> >> I'm interested in adding support for deduplicating lambda methods to >> javac. The idea is that if a compilation unit contains two lambdas that are >> identical (including any captured state and the functional interface they >> implement) we could re-use the same implementation method for both. >> >> I understand there might have been some prior discussion about this. Is >> there interest in investigating the feature? What sort of technical >> considerations have been identified so far? >> >> I have been thinking about a couple of questions: >> >> 1) How to identifying duplicates: I have a prototype that runs during >> lambda desugaring and identifies duplicates by diffing ASTs. Is that the >> best place for deduplication, or it worth considering comparing generated >> code instead of ASTs? > > > are you doing an exact diff? I assume that we want: s -> s to be equal to z > -> z provided that the target is the same > >> >> 2) Debug info: the optimization is safe if line numbers are not being >> emitted. If they are, is there a way to deduplicate the methods without >> breaking debug info? > > > I haven't tried to debug a method with more than one LNT, the spec allows it > but not sure how IDEs will operate on that Interesting, but every dynamic call site would have to identify the right LNT and link it to the corresponding lambda instance via the bootstrap method so that a debugger could find it at runtime... is this already done? Note that, as recently discussed on compiler-dev, the indy line number is currently not emitted (causing some stack trace problems) but the lambda line numbers have to be emitted to step into the latter, is this currently done (I've not verified)? Bernard >> >> Thanks, >> Liam > > > Vicente From forax at univ-mlv.fr Sun Mar 4 13:11:28 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 4 Mar 2018 14:11:28 +0100 (CET) Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: <1933849696.2104307.1520169088671.JavaMail.zimbra@u-pem.fr> Hi Vincente, ----- Mail original ----- > De: "Vicente Romero" > ?: "Liam Miller-Cushon" , "amber-dev" , "Brian Goetz" > > Envoy?: Samedi 3 Mars 2018 05:01:47 > Objet: Re: deduplicating lambda methods [...] >> >> 2) Debug info: the optimization is safe if line numbers are not being >> emitted. If they are, is there a way to deduplicate the methods >> without breaking debug info? > > I haven't tried to debug a method with more than one LNT, the spec > allows it but not sure how IDEs will operate on that from JVMS 4.7.13, "The LocalVariableTable attribute is an optional variable-length attribute in the attributes table of a Code attribute (?4.7.3). It may be used by debuggers to determine the value of a given local variable during the execution of a method. If multiple LocalVariableTable attributes are present in the attributes table of a Code attribute, then they may appear in any order. There may be no more than one LocalVariableTable attribute per local variable in the attributes table of a Code attribute. " You can not have several LNTs that map different parts of a compilation unit to the same bytecode because if you have several attributes it's like you can merge them into one attribute, so deduplication if there are debug info is not possible as far as i know. > Vicente R?mi From bsrbnd at gmail.com Sun Mar 4 16:25:18 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 4 Mar 2018 17:25:18 +0100 Subject: deduplicating lambda methods In-Reply-To: <1933849696.2104307.1520169088671.JavaMail.zimbra@u-pem.fr> References: <1933849696.2104307.1520169088671.JavaMail.zimbra@u-pem.fr> Message-ID: On 4 March 2018 at 14:11, Remi Forax wrote: > Hi Vincente, > > ----- Mail original ----- >> De: "Vicente Romero" >> ?: "Liam Miller-Cushon" , "amber-dev" , "Brian Goetz" >> >> Envoy?: Samedi 3 Mars 2018 05:01:47 >> Objet: Re: deduplicating lambda methods > > [...] > >>> >>> 2) Debug info: the optimization is safe if line numbers are not being >>> emitted. If they are, is there a way to deduplicate the methods >>> without breaking debug info? >> >> I haven't tried to debug a method with more than one LNT, the spec >> allows it but not sure how IDEs will operate on that > > from JVMS 4.7.13, > "The LocalVariableTable attribute is an optional variable-length attribute in the attributes table of a Code attribute (?4.7.3). It may be used by debuggers to determine the value of a given local variable during the execution of a method. > If multiple LocalVariableTable attributes are present in the attributes table of a Code attribute, then they may appear in any order. > There may be no more than one LocalVariableTable attribute per local variable in the attributes table of a Code attribute. " > > You can not have several LNTs that map different parts of a compilation unit to the same bytecode because if you have several attributes it's like you can merge them into one attribute, > so deduplication if there are debug info is not possible as far as i know. The LVT is simply mapping local variable symbols ('name_index') to their memory offsets ('index'). It should be possible to merge them as the bytecode (modulo my previous comments to retrieve the right LNT) for identical lambdas, I think. Bernard >> Vicente > > R?mi From vicente.romero at oracle.com Sun Mar 4 16:38:03 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Sun, 4 Mar 2018 11:38:03 -0500 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: Hi Bernard, On 03/04/2018 07:39 AM, B. Blaser wrote: > On 3 March 2018 at 05:01, Vicente Romero wrote: >> very interesting! >> >> On 03/02/2018 08:03 PM, Liam Miller-Cushon wrote: >>> Hello, >>> >>> I'm interested in adding support for deduplicating lambda methods to >>> javac. The idea is that if a compilation unit contains two lambdas that are >>> identical (including any captured state and the functional interface they >>> implement) we could re-use the same implementation method for both. >>> >>> I understand there might have been some prior discussion about this. Is >>> there interest in investigating the feature? What sort of technical >>> considerations have been identified so far? >>> >>> I have been thinking about a couple of questions: >>> >>> 1) How to identifying duplicates: I have a prototype that runs during >>> lambda desugaring and identifies duplicates by diffing ASTs. Is that the >>> best place for deduplication, or it worth considering comparing generated >>> code instead of ASTs? >> >> are you doing an exact diff? I assume that we want: s -> s to be equal to z >> -> z provided that the target is the same >> >>> 2) Debug info: the optimization is safe if line numbers are not being >>> emitted. If they are, is there a way to deduplicate the methods without >>> breaking debug info? >> >> I haven't tried to debug a method with more than one LNT, the spec allows it >> but not sure how IDEs will operate on that > Interesting, but every dynamic call site would have to identify the > right LNT and link it to the corresponding lambda instance via the > bootstrap method so that a debugger could find it at runtime... is > this already done? > > Note that, as recently discussed on compiler-dev, the indy line number > is currently not emitted (causing some stack trace problems) but the > lambda line numbers have to be emitted to step into the latter, is > this currently done (I've not verified)? no we haven't done any change to this area yet > > Bernard Vicente > >>> Thanks, >>> Liam >> >> Vicente From bsrbnd at gmail.com Sun Mar 4 17:37:56 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 4 Mar 2018 18:37:56 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <1933849696.2104307.1520169088671.JavaMail.zimbra@u-pem.fr> Message-ID: On 4 March 2018 at 17:25, B. Blaser wrote: > On 4 March 2018 at 14:11, Remi Forax wrote: >> Hi Vincente, >> >> ----- Mail original ----- >>> De: "Vicente Romero" >>> ?: "Liam Miller-Cushon" , "amber-dev" , "Brian Goetz" >>> >>> Envoy?: Samedi 3 Mars 2018 05:01:47 >>> Objet: Re: deduplicating lambda methods >> >> [...] >> >>>> >>>> 2) Debug info: the optimization is safe if line numbers are not being >>>> emitted. If they are, is there a way to deduplicate the methods >>>> without breaking debug info? >>> >>> I haven't tried to debug a method with more than one LNT, the spec >>> allows it but not sure how IDEs will operate on that >> >> from JVMS 4.7.13, >> "The LocalVariableTable attribute is an optional variable-length attribute in the attributes table of a Code attribute (?4.7.3). It may be used by debuggers to determine the value of a given local variable during the execution of a method. >> If multiple LocalVariableTable attributes are present in the attributes table of a Code attribute, then they may appear in any order. >> There may be no more than one LocalVariableTable attribute per local variable in the attributes table of a Code attribute. " >> >> You can not have several LNTs that map different parts of a compilation unit to the same bytecode because if you have several attributes it's like you can merge them into one attribute, >> so deduplication if there are debug info is not possible as far as i know. > > The LVT is simply mapping local variable symbols ('name_index') to > their memory offsets ('index'). It should be possible to merge them as > the bytecode (modulo my previous comments to retrieve the right LNT) > for identical lambdas, I think. For example: $ cat R.java class R { void f() { Runnable r = () -> { int i = 0; }; r = () -> { int i = 0; }; } } $ javac -g R.java $ javap -l -c -p R.class Compiled from "R.java" class R { [...] private static void lambda$f$1(); Code: 0: iconst_0 1: istore_0 2: return LineNumberTable: line 4: 0 LocalVariableTable: Start Length Slot Name Signature 2 1 0 i I private static void lambda$f$0(); Code: 0: iconst_0 1: istore_0 2: return LineNumberTable: line 3: 0 LocalVariableTable: Start Length Slot Name Signature 2 1 0 i I } B. > Bernard > >>> Vicente >> >> R?mi From vicente.romero at oracle.com Mon Mar 5 02:29:51 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Mon, 05 Mar 2018 02:29:51 +0000 Subject: hg: amber/amber: temporary overloaded version of LMF.altMetafactory to experiment with condy Message-ID: <201803050229.w252TqSM002242@aojmv0008.oracle.com> Changeset: bce3abdce888 Author: vromero Date: 2018-03-04 19:55 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/bce3abdce888 temporary overloaded version of LMF.altMetafactory to experiment with condy ! src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java From vicente.romero at oracle.com Mon Mar 5 03:02:14 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Mon, 05 Mar 2018 03:02:14 +0000 Subject: hg: amber/amber: fixing regression tests failing in condy-folding repo Message-ID: <201803050302.w2532EtI015838@aojmv0008.oracle.com> Changeset: 2b25316b0e9b Author: vromero Date: 2018-03-04 21:29 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/2b25316b0e9b fixing regression tests failing in condy-folding repo ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! test/langtools/tools/javac/T8019486/WrongLNTForLambdaTest.java ! test/langtools/tools/javac/T8187978/FilterOutCandidatesForDiagnosticsTest.out From vicente.romero at oracle.com Mon Mar 5 03:54:37 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Mon, 05 Mar 2018 03:54:37 +0000 Subject: hg: amber/amber: using condy for serializable lambda and method references Message-ID: <201803050354.w253scBR007622@aojmv0008.oracle.com> Changeset: b1fefcb041a1 Author: vromero Date: 2018-03-04 22:05 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/b1fefcb041a1 using condy for serializable lambda and method references ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java From cushon at google.com Mon Mar 5 03:59:25 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Sun, 4 Mar 2018 19:59:25 -0800 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: On Fri, Mar 2, 2018 at 8:01 PM, Vicente Romero wrote: > 1) How to identifying duplicates: I have a prototype that runs during >> lambda desugaring and identifies duplicates by diffing ASTs. Is that the >> best place for deduplication, or it worth considering comparing generated >> code instead of ASTs? >> > > are you doing an exact diff? I assume that we want: s -> s to be equal to > z -> z provided that the target is the same Lambda parameters are currently handled as a special case. The diff tests that the syntax for the two trees is the same, and that the symbols associated with identifier and select nodes are equivalent. It accepts symbols that correspond to the same parameter in each of the associated lambda methods. E.g. when diffing the lambda bodies for `s -> s` and `x -> x` it sees that `s` and `x` are both the first parameter of their enclosing lambdas. From amaembo at gmail.com Mon Mar 5 07:51:40 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Mon, 5 Mar 2018 14:51:40 +0700 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: Hello! > E.g. when diffing the lambda bodies for `s -> s` and `x -> x` it sees that `s` and `x` are both the first parameter of their enclosing lambdas. By the way how types are matched? I assume that you won't merge `s -> s` and `x -> x` if `s` is String and `x` is an int, right? However merging two lambdas like `s -> true` and `sb -> true` where `s` is a String and `sb` is a StringBuilder seems possible (creating a synthetic method which receives an Object). Also two lambdas like `list -> list.size()` can be merged if the parameter types differ in type parameters (e.g. List and List), but have the same erasure. Tagir. On Mon, Mar 5, 2018 at 10:59 AM, Liam Miller-Cushon wrote: > On Fri, Mar 2, 2018 at 8:01 PM, Vicente Romero > wrote: > > > 1) How to identifying duplicates: I have a prototype that runs during > >> lambda desugaring and identifies duplicates by diffing ASTs. Is that the > >> best place for deduplication, or it worth considering comparing > generated > >> code instead of ASTs? > >> > > > > are you doing an exact diff? I assume that we want: s -> s to be equal to > > z -> z provided that the target is the same > > > Lambda parameters are currently handled as a special case. The diff tests > that the syntax for the two trees is the same, and that the symbols > associated with identifier and select nodes are equivalent. It accepts > symbols that correspond to the same parameter in each of the associated > lambda methods. > > E.g. when diffing the lambda bodies for `s -> s` and `x -> x` it sees that > `s` and `x` are both the first parameter of their enclosing lambdas. > From maurizio.cimadamore at oracle.com Mon Mar 5 11:30:17 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 5 Mar 2018 11:30:17 +0000 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: On 05/03/18 07:51, Tagir Valeev wrote: > Hello! > >> E.g. when diffing the lambda bodies for `s -> s` and `x -> x` it sees > that > `s` and `x` are both the first parameter of their enclosing lambdas. > > By the way how types are matched? I assume that you won't merge `s -> s` > and `x -> x` > if `s` is String and `x` is an int, right? However merging two lambdas like > `s -> true` and `sb -> true` > where `s` is a String and `sb` is a StringBuilder seems possible (creating > a synthetic method which > receives an Object). Yes, if parameters are unused, then I'd expect that an AST diff will just say that the body is the same for both. > Also two lambdas like `list -> list.size()` can be > merged if the parameter types > differ in type parameters (e.g. List and List), but have > the same erasure. Yes, the diff should work on erased types, which is what really matters in terms of bytecode generation. There are of course caveat - if you call List::get instead of List::size, then you can get extra synthetic casts, and, in that case, you need to avoid deduplication so that each lambda can have its own synthetic cast. Maurizio > > Tagir. > > On Mon, Mar 5, 2018 at 10:59 AM, Liam Miller-Cushon > wrote: > >> On Fri, Mar 2, 2018 at 8:01 PM, Vicente Romero >> wrote: >> >>> 1) How to identifying duplicates: I have a prototype that runs during >>>> lambda desugaring and identifies duplicates by diffing ASTs. Is that the >>>> best place for deduplication, or it worth considering comparing >> generated >>>> code instead of ASTs? >>>> >>> are you doing an exact diff? I assume that we want: s -> s to be equal to >>> z -> z provided that the target is the same >> >> Lambda parameters are currently handled as a special case. The diff tests >> that the syntax for the two trees is the same, and that the symbols >> associated with identifier and select nodes are equivalent. It accepts >> symbols that correspond to the same parameter in each of the associated >> lambda methods. >> >> E.g. when diffing the lambda bodies for `s -> s` and `x -> x` it sees that >> `s` and `x` are both the first parameter of their enclosing lambdas. >> From maurizio.cimadamore at oracle.com Mon Mar 5 11:37:46 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 5 Mar 2018 11:37:46 +0000 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: <4b7cb307-1af7-ccfe-02a0-3358dd75959c@oracle.com> The overall plan seems pretty solid; thanks for looking into this. As for (1), I think comparing bytecodes would be simpler from a technical perspective, as there's less fuzziness about it - you just compare the stream of bytecodes and, if they are the same, deduplicate. But it's more a problem for our compiler which happens to be very AST centric, so lots of passes are described in terms of AST manipulations, not bytecode ones - which I think means that this one should be AST-based too. As Brian mentioned, I would start simple, by ignoring edge cases such as serializable lambdas and/or deduplicating in the presence of debugging info. One question I have for the VM gurus: how does deduplication affect JIT optimizations? E.g. assume we have two lambdas used in two very different way in the same source file, which happen to share the same code - e.g. something like: Object o -> { if (o instanceof String) { System.err.println("Hello!"); } one use site always calls it with a string parameter, another always calls it with an int parameter. Now, with indy, I believe we can still optimize them fully, given that indy has some knowledge/state about the site making the call, so the two call won't be treated as 'identical' by the JIT and the profiling info won't be merged (I guess). But what if, someday, we were to replace indy with condy here? Would we lose performances? Cheers Maurizio On 03/03/18 01:03, Liam Miller-Cushon wrote: > Hello, > > I'm interested in adding support for deduplicating lambda methods to javac. > The idea is that if a compilation unit contains two lambdas that are > identical (including any captured state and the functional interface they > implement) we could re-use the same implementation method for both. > > I understand there might have been some prior discussion about this. Is > there interest in investigating the feature? What sort of technical > considerations have been identified so far? > > I have been thinking about a couple of questions: > > 1) How to identifying duplicates: I have a prototype that runs during > lambda desugaring and identifies duplicates by diffing ASTs. Is that the > best place for deduplication, or it worth considering comparing generated > code instead of ASTs? > > 2) Debug info: the optimization is safe if line numbers are not being > emitted. If they are, is there a way to deduplicate the methods without > breaking debug info? > > Thanks, > Liam From amaembo at gmail.com Mon Mar 5 11:49:09 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Mon, 5 Mar 2018 18:49:09 +0700 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: Hello! By the way how types are matched? I assume that you won't merge `s -> s` > and `x -> x` > if `s` is String and `x` is an int, right? However merging two lambdas like > `s -> true` and `sb -> true` > where `s` is a String and `sb` is a StringBuilder seems possible (creating > a synthetic method which > receives an Object). > Yes, if parameters are unused, then I'd expect that an AST diff will just say that the body is the same for both. What if unused parameter have different primitive type or primitive and object type? Will we tolerate unnecessary boxing? Well, it's possible just to drop an unused parameter from synthetic method declaration, though this might require changes in LMF. Also in this case we will not see the variable in debugger. This reminds me about (postponed?) lambda leftovers JEP. Probably if that JEP will be implemented, duplication should ignore underscore parameters only. In this case absence in debugger is not a problem. Tagir. Also two lambdas like `list -> list.size()` can be > merged if the parameter types > differ in type parameters (e.g. List and List), but have > the same erasure. > Yes, the diff should work on erased types, which is what really matters in terms of bytecode generation. There are of course caveat - if you call List::get instead of List::size, then you can get extra synthetic casts, and, in that case, you need to avoid deduplication so that each lambda can have its own synthetic cast. Maurizio > Tagir. > > On Mon, Mar 5, 2018 at 10:59 AM, Liam Miller-Cushon > wrote: > > On Fri, Mar 2, 2018 at 8:01 PM, Vicente Romero >> wrote: >> >> 1) How to identifying duplicates: I have a prototype that runs during >>> >>>> lambda desugaring and identifies duplicates by diffing ASTs. Is that the >>>> best place for deduplication, or it worth considering comparing >>>> >>> generated >> >>> code instead of ASTs? >>>> >>>> are you doing an exact diff? I assume that we want: s -> s to be equal >>> to >>> z -> z provided that the target is the same >>> >> >> Lambda parameters are currently handled as a special case. The diff tests >> that the syntax for the two trees is the same, and that the symbols >> associated with identifier and select nodes are equivalent. It accepts >> symbols that correspond to the same parameter in each of the associated >> lambda methods. >> >> E.g. when diffing the lambda bodies for `s -> s` and `x -> x` it sees that >> `s` and `x` are both the first parameter of their enclosing lambdas. >> >> From maurizio.cimadamore at oracle.com Mon Mar 5 13:40:51 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 5 Mar 2018 13:40:51 +0000 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: On 05/03/18 11:49, Tagir Valeev wrote: > Hello! > > By the way how types are matched? I assume that you won't > merge `s -> s` > and `x -> x` > if `s` is String and `x` is an int, right? However merging two > lambdas like > `s -> true` and `sb -> true` > where `s` is a String and `sb` is a StringBuilder seems > possible (creating > a synthetic method which > receives an Object). > > Yes, if parameters are unused, then I'd expect that an AST diff > will just say that the body is the same for both. > > > What if unused parameter have different primitive type or primitive > and object type? Will we tolerate unnecessary boxing? Well, it's > possible just to drop an unused parameter from synthetic method > declaration, though this might require changes in LMF. Also in this > case we will not see the variable in debugger. > > This reminds me about (postponed?) lambda leftovers JEP. Probably if > that JEP will be implemented, duplication should ignore underscore > parameters only. In this case absence in debugger is not a problem. Note that, from an implementation perspective, we could even drop the unused parameter from the synthetic lambda body entirely and use some kind of MH adaptation to inject a default one in there (based on the 'invokedType') - so, yes, this is a problem, and we might restrict the level of deduplication in case of primitive vs. reference clashes, but maybe there are technical solutions to address that one too. Of maybe we just want to wait to handle those for when we have better value support (see L-world effort in valhalla) :-) Maurizio > > Tagir. > > > > Also two lambdas like `list -> list.size()` can be > merged if the parameter types > differ in type parameters (e.g. List and > List), but have > the same erasure. > > Yes, the diff should work on erased types, which is what really > matters in terms of bytecode generation. There are of course > caveat - if you call List::get instead of List::size, then you can > get extra synthetic casts, and, in that case, you need to avoid > deduplication so that each lambda can have its own synthetic cast. > > Maurizio > > > Tagir. > > On Mon, Mar 5, 2018 at 10:59 AM, Liam Miller-Cushon > > > wrote: > > On Fri, Mar 2, 2018 at 8:01 PM, Vicente Romero > > > wrote: > > 1) How to identifying duplicates: I have a prototype > that runs during > > lambda desugaring and identifies duplicates by > diffing ASTs. Is that the > best place for deduplication, or it worth > considering comparing > > generated > > code instead of ASTs? > > are you doing an exact diff? I assume that we want: s > -> s to be equal to > z -> z provided that the target is the same > > > Lambda parameters are currently handled as a special case. > The diff tests > that the syntax for the two trees is the same, and that > the symbols > associated with identifier and select nodes are > equivalent. It accepts > symbols that correspond to the same parameter in each of > the associated > lambda methods. > > E.g. when diffing the lambda bodies for `s -> s` and `x -> > x` it sees that > `s` and `x` are both the first parameter of their > enclosing lambdas. > > > From lowasser at google.com Mon Mar 5 19:55:19 2018 From: lowasser at google.com (Louis Wasserman) Date: Mon, 05 Mar 2018 19:55:19 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <99ce48ba-8df1-d155-0638-2ec9a76582d7@oracle.com> Message-ID: So, here's a start on some of the data questions that have been asked in this thread. There's an attached PDF with a bunch of nice graphs and tables discussing all lambdas in Google's codebase. This is a pretty simplistic analysis, deduplicating lambdas syntactically modulo parameter name and target type including generics, but a more subtle analysis might merge some more things -- e.g. Function.identity(). Some other interesting data points not in the PDF: Among nongenerated files having any lambdas at all, - 16.5% have at least one syntactic duplicate in them. - The average number of lambdas with at least one syntactic duplicate is 0.24. - The average number of synthetic methods you'd eliminate by deduplicating within a file is 0.47. The six most common (target type including generics, syntactic method body modulo parameter naming) pairs across our entire codebase were: (Runnable) () -> {} // 674 (Predicate) str -> !str.isEmpty() // 640 (Function) str -> str // 492 (Callable) () -> null // 259 (Predicate) str -> !Strings.isNullOrEmpty(str) // 204 (Function) x -> x // 177 (Worth mentioning explicitly: x -> x + 1 was a ways down, with only 56 occurrences for UnaryOperator as the most common type.) Liam and I are still working on collecting information on method references and on statefulness and serializability. > ---------- Forwarded message ---------- >> From: Brian Goetz >> Date: Fri, Mar 2, 2018 at 5:17 PM >> Subject: Re: deduplicating lambda methods >> To: Liam Miller-Cushon , amber-dev at openjdk.java.net, >> Vicente-Arturo Romero-Zaldivar >> > > > I know you have some good tools at Google for codebase statistics. Maybe >> you could pull together data on how often lambdas are duplicated within a >> source file, and of the duplicated lambdas, what percentage are stateless >> and non-serializable? > > From cushon at google.com Mon Mar 5 20:22:18 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Mon, 5 Mar 2018 12:22:18 -0800 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: On Mon, Mar 5, 2018 at 3:30 AM, Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > > On 05/03/18 07:51, Tagir Valeev wrote: > >> Hello! >> >> E.g. when diffing the lambda bodies for `s -> s` and `x -> x` it sees >>> >> that >> `s` and `x` are both the first parameter of their enclosing lambdas. >> >> By the way how types are matched? I assume that you won't merge `s -> s` >> and `x -> x` >> if `s` is String and `x` is an int, right? However merging two lambdas >> like >> `s -> true` and `sb -> true` >> where `s` is a String and `sb` is a StringBuilder seems possible (creating >> a synthetic method which >> receives an Object). >> > Yes, if parameters are unused, then I'd expect that an AST diff will just > say that the body is the same for both. > > Also two lambdas like `list -> list.size()` can be >> merged if the parameter types >> differ in type parameters (e.g. List and List), but have >> the same erasure. >> > Yes, the diff should work on erased types, which is what really matters in > terms of bytecode generation. There are of course caveat - if you call > List::get instead of List::size, then you can get extra synthetic casts, > and, in that case, you need to avoid deduplication so that each lambda can > have its own synthetic cast. The initial proposal was to deduplicate lambdas that (1) implement the same functional interface, (2) capture the same state, and (3) have the same implementation. The diff is used specifically to determine (3), and (1) guarantees that the parameter types match. So deduplicating e.g. `(String s) -> true` and `(StringBuilder s) -> true` is something that could be investigated, but it isn't currently handled. From lowasser at google.com Mon Mar 5 20:24:51 2018 From: lowasser at google.com (Louis Wasserman) Date: Mon, 05 Mar 2018 20:24:51 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <99ce48ba-8df1-d155-0638-2ec9a76582d7@oracle.com> Message-ID: Apparently PDF attachments don't work. Here's a link: https://drive.google.com/file/d/1abAR_bueU0Zxy4e9XfLVzVe2nwy_POQm/view?usp=sharing On Mon, Mar 5, 2018 at 11:55 AM Louis Wasserman wrote: > So, here's a start on some of the data questions that have been asked in > this thread. There's an attached PDF with a bunch of nice graphs and > tables discussing all lambdas in Google's codebase. This is a pretty > simplistic analysis, deduplicating lambdas syntactically modulo parameter > name and target type including generics, but a more subtle analysis might > merge some more things -- e.g. Function.identity(). Some other interesting > data points not in the PDF: > > Among nongenerated files having any lambdas at all, > > - 16.5% have at least one syntactic duplicate in them. > - The average number of lambdas with at least one syntactic duplicate > is 0.24. > - The average number of synthetic methods you'd eliminate by > deduplicating within a file is 0.47. > > The six most common (target type including generics, syntactic method body > modulo parameter naming) pairs across our entire codebase were: > (Runnable) () -> {} // 674 > (Predicate) str -> !str.isEmpty() // 640 > (Function) str -> str // 492 > (Callable) () -> null // 259 > (Predicate) str -> !Strings.isNullOrEmpty(str) // 204 > (Function) x -> x // 177 > > (Worth mentioning explicitly: x -> x + 1 was a ways down, with only 56 > occurrences for UnaryOperator as the most common type.) > > Liam and I are still working on collecting information on method > references and on statefulness and serializability. > > >> ---------- Forwarded message ---------- >>> From: Brian Goetz >>> Date: Fri, Mar 2, 2018 at 5:17 PM >>> Subject: Re: deduplicating lambda methods >>> To: Liam Miller-Cushon , amber-dev at openjdk.java.net, >>> Vicente-Arturo Romero-Zaldivar >>> >> >> >> I know you have some good tools at Google for codebase statistics. Maybe >>> you could pull together data on how often lambdas are duplicated within a >>> source file, and of the duplicated lambdas, what percentage are stateless >>> and non-serializable? >> >> From brian.goetz at oracle.com Mon Mar 5 22:46:23 2018 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Mon, 05 Mar 2018 22:46:23 +0000 Subject: hg: amber/amber: Update specs as per Alex' comments Message-ID: <201803052246.w25MkNx6019384@aojmv0008.oracle.com> Changeset: 3f0706d6e402 Author: briangoetz Date: 2018-03-05 17:41 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/3f0706d6e402 Update specs as per Alex' comments ! src/java.base/share/classes/java/lang/sym/ClassRef.java ! src/java.base/share/classes/java/lang/sym/Constable.java ! src/java.base/share/classes/java/lang/sym/ConstantMethodHandleRef.java ! src/java.base/share/classes/java/lang/sym/ConstantRef.java ! src/java.base/share/classes/java/lang/sym/ConstantRefs.java ! src/java.base/share/classes/java/lang/sym/DynamicCallSiteRef.java ! src/java.base/share/classes/java/lang/sym/DynamicConstantRef.java ! src/java.base/share/classes/java/lang/sym/package-info.java From brian.goetz at oracle.com Mon Mar 5 23:10:13 2018 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Mon, 05 Mar 2018 23:10:13 +0000 Subject: hg: amber/amber: Better toString() and toConstantRef() implementations Message-ID: <201803052310.w25NAEGQ000143@aojmv0008.oracle.com> Changeset: c4453cddffe1 Author: briangoetz Date: 2018-03-05 18:02 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/c4453cddffe1 Better toString() and toConstantRef() implementations ! src/java.base/share/classes/java/lang/sym/ConstantClassRef.java ! src/java.base/share/classes/java/lang/sym/ConstantMethodHandleRef.java ! src/java.base/share/classes/java/lang/sym/ConstantMethodTypeRef.java ! src/java.base/share/classes/java/lang/sym/ConstantRefs.java ! src/java.base/share/classes/java/lang/sym/DynamicConstantRef.java ! src/java.base/share/classes/java/lang/sym/EnumRef.java ! src/java.base/share/classes/java/lang/sym/MethodHandleRef.java + src/java.base/share/classes/java/lang/sym/RefBootstraps.java ! src/java.base/share/classes/java/lang/sym/package-info.java From vicente.romero at oracle.com Mon Mar 5 23:38:56 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Mon, 05 Mar 2018 23:38:56 +0000 Subject: hg: amber/amber: javac was wrongly checking for instances of ConstantRef instead of Constable Message-ID: <201803052338.w25Ncuok013181@aojmv0008.oracle.com> Changeset: 36326b5ae7a1 Author: vromero Date: 2018-03-05 18:26 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/36326b5ae7a1 javac was wrongly checking for instances of ConstantRef instead of Constable ! src/jdk.compiler/share/classes/com/sun/tools/javac/util/Constables.java From vicente.romero at oracle.com Mon Mar 5 23:46:28 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Mon, 05 Mar 2018 23:46:28 +0000 Subject: hg: amber/amber: [mq]: making.constables.a.context.class Message-ID: <201803052346.w25NkSlx017049@aojmv0008.oracle.com> Changeset: 64684a632d15 Author: vromero Date: 2018-03-05 18:35 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/64684a632d15 [mq]: making.constables.a.context.class ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/ConstablesVisitor.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/util/Constables.java From john.r.rose at oracle.com Tue Mar 6 02:27:53 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 5 Mar 2018 18:27:53 -0800 Subject: deduplicating lambda methods In-Reply-To: <4b7cb307-1af7-ccfe-02a0-3358dd75959c@oracle.com> References: <4b7cb307-1af7-ccfe-02a0-3358dd75959c@oracle.com> Message-ID: On Mar 5, 2018, at 3:37 AM, Maurizio Cimadamore wrote: > > Now, with indy, I believe we can still optimize them fully, given that indy has some knowledge/state about the site making the call, so the two call won't be treated as 'identical' by the JIT and the profiling info won't be merged (I guess). But what if, someday, we were to replace indy with condy here? Would we lose performances? Deduplication of code always has the risk of lower performance, if the deduplicated code has a profile that the JIT might rely on. Duplicated code can sometimes collect crucial profile data which deduplicated code would miss. You are right about indy-generated lambdas being distinguished by the JIT. This is especially true when (as is current) each LMF invocation makes a fresh body of code. It's a clear invitation to the JIT to optimize that chunk of logic separately. Suppose the LMF were to de-duplicate dynamically. That would have a similar effect to a condy-based use of the LMF, in that the JIT would see fewer distinctions between lambdas from distinct capture sites. The most important distinctions are static types and dynamic type profiles. Monomorphic type profiles are especially favorable since they allow devirtualization and (usually) subsequent inlining. The worst performance cliff to fall down is when a monomorphic type profile point in a duplicated bytecode becomes polluted when it is merged with another duplicate that contributes incompatible type profile data. If there are virtual calls to the profiled type, they can fail to inline when a profile is polluted. This can happen when a generic algorithm is reused on two unrelated types, and it gets refactored into shared code. There's an important mitigating effect: If the generic code is inlined into a caller which has a monomorphic profile and/or verifiable strong static types, then the better types from the caller can override the polluted profile of the callee. In effect, a successful inlining decision makes new copies of the old code, which can sometimes reverse bad effects from deduplication or genericity. But no real profiling happens after the JIT splits out a copy of some bytecode for inlining. This means JIT inlining does not clean up polluted profiles. (That is a current limitation, not a fundamental problem, since HotSpot could collect split profiles from early tiers of compilation for use by later tiers. See JDK-8015416.) In the case of generic code, if a generic algorithm like ArrayList::indexOf is inlined into a caller which knows that the argument to indexOf is always a String, then the call to Object.equals inside the algorithm can be devirtualized to String.equals and inlined. The knowledge of the String type could come from the caller's static types or the caller's profile. Currently each lambda indy is a distinct lambda factory call, so that lambda gets its own adapter code and hence its own profile and static types. If the desugared body is simple and inlines, then the JVM has a good chance at optimizing it, regardless of the quality of the desugared body's profile. But if the body is complex, the applicability of the adapter's static types and profile will only apply "around the edges". So if two lambdas share a desugared body (whether locally or globally), the profile of that body might be polluted, but some (not all) of the pollution can be cleaned up if the JIT finds useful context in each indy site. The lambda bodies that are best suited for this are ones which mainly call methods on argument objects already seen (and maybe profiled) by the lambda's adapter object (FI implementation). A lambda body which creates a temporary value of a statically unpredictable type might fall off the cliff sooner, if it is shared and the sharing makes it harder to get a monomorphic profile on the unpredictable type. These rules of thumb are generally valid, but specific results are not robust, since JVM optimization techniques change over time. A new language construct like streams or lambdas or patterns always introduces new code shapes which the JIT has to respond to, and there is a teething period where the JIT can't optimize the construct as nicely as we want. JIT engineers are full of tricks, and they can often find a tweak that will make the code digest well. Sometimes that's hard, in which case it might be years before a new feature performs as well as hand-optimized code. The low-level stuff we are talking about here is probably in the easier category, of tweakable code. One way to preserve more contextuality from translation of lambdas is to retain indy in the translation strategy, but use condy to hold shared resources. Maybe the LMF can make a tunable, on-the-fly decision how to use the context to "hook in" a profile to the shared stuff in the condy. It would be very easy to wrap a trivial bit of bytecode around a method which does nothing except profile arguments and call the method on those arguments. In this way most class structure generated by LMFs could be reused across compilation units, while still collecting contextual profile information. And when I say "shared stuff in condy", it could be either locally deduplicated stuff, or dynamically and globally deduplicated. Perhaps a well balanced design would manage to create distinct profile points for every distinct spot in the source code, while sharing everything else as globally as possible. And telling the JIT about it, so that it never fails to inline the shared stuff into each distinctly profiled context. Our current system approximates this. ? John From vicente.romero at oracle.com Tue Mar 6 04:22:11 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Tue, 06 Mar 2018 04:22:11 +0000 Subject: hg: amber/amber: check that serializable lambdas work independently of the serialization approach Message-ID: <201803060422.w264MBIR018943@aojmv0008.oracle.com> Changeset: c5d728a6902c Author: vromero Date: 2018-03-05 23:16 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/c5d728a6902c check that serializable lambdas work independently of the serialization approach + test/langtools/tools/javac/condy/LambdaSerializationTest.java From vicente.romero at oracle.com Wed Mar 7 15:32:33 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Wed, 07 Mar 2018 15:32:33 +0000 Subject: hg: amber/amber: dont do dead code elimination for constables Message-ID: <201803071532.w27FWX88008150@aojmv0008.oracle.com> Changeset: b6b567d6db36 Author: vromero Date: 2018-03-07 10:25 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/b6b567d6db36 dont do dead code elimination for constables ! src/jdk.compiler/share/classes/com/sun/tools/javac/util/Constables.java From brian.goetz at oracle.com Wed Mar 7 15:39:21 2018 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Wed, 07 Mar 2018 15:39:21 +0000 Subject: hg: amber/amber: Document preconditions for switch bootstraps; assert preconditions Message-ID: <201803071539.w27FdLlX011285@aojmv0008.oracle.com> Changeset: f7e0fb7b2ce4 Author: briangoetz Date: 2018-03-07 10:34 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/f7e0fb7b2ce4 Document preconditions for switch bootstraps; assert preconditions ! src/java.base/share/classes/java/lang/runtime/SwitchBootstraps.java From bsrbnd at gmail.com Wed Mar 7 16:25:12 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 7 Mar 2018 17:25:12 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <4b7cb307-1af7-ccfe-02a0-3358dd75959c@oracle.com> Message-ID: On 6 March 2018 at 03:27, John Rose wrote: > On Mar 5, 2018, at 3:37 AM, Maurizio Cimadamore wrote: >> >> Now, with indy, I believe we can still optimize them fully, given that indy has some knowledge/state about the site making the call, so the two call won't be treated as 'identical' by the JIT and the profiling info won't be merged (I guess). But what if, someday, we were to replace indy with condy here? Would we lose performances? > > Deduplication of code always has the risk of lower performance, > if the deduplicated code has a profile that the JIT might rely on. > > Duplicated code can sometimes collect crucial profile data which > deduplicated code would miss. [...] > Currently each lambda indy is a distinct lambda factory call, > so that lambda gets its own adapter code and hence its own > profile and static types. If the desugared body is simple and > inlines, then the JVM has a good chance at optimizing it, > regardless of the quality of the desugared body's profile. > But if the body is complex, the applicability of the adapter's > static types and profile will only apply "around the edges". > > So if two lambdas share a desugared body (whether locally or > globally), the profile of that body might be polluted, but some > (not all) of the pollution can be cleaned up if the JIT finds > useful context in each indy site. The lambda bodies that > are best suited for this are ones which mainly call methods > on argument objects already seen (and maybe profiled) > by the lambda's adapter object (FI implementation). > A lambda body which creates a temporary value of > a statically unpredictable type might fall off the cliff sooner, > if it is shared and the sharing makes it harder to get > a monomorphic profile on the unpredictable type. > > These rules of thumb are generally valid, but specific > results are not robust, since JVM optimization techniques > change over time. [...] > One way to preserve more contextuality from translation > of lambdas is to retain indy in the translation strategy, > but use condy to hold shared resources. Maybe the > LMF can make a tunable, on-the-fly decision how to use > the context to "hook in" a profile to the shared stuff > in the condy. It would be very easy to wrap a trivial > bit of bytecode around a method which does nothing > except profile arguments and call the method on those > arguments. In this way most class structure generated > by LMFs could be reused across compilation units, > while still collecting contextual profile information. > > And when I say "shared stuff in condy", it could be > either locally deduplicated stuff, or dynamically and > globally deduplicated. > > Perhaps a well balanced design would manage to > create distinct profile points for every distinct spot in the > source code, while sharing everything else as globally > as possible. And telling the JIT about it, so that it > never fails to inline the shared stuff into each distinctly > profiled context. Our current system approximates this. Looking at the stats [1], we see that most of the cases are trivial and mostly related to primitive & boxing types. I'm wondering if JIT profiles would be affected if de-duplication was simply focusing on those trivial cases? If profiles were not polluted in such cases, It wouldn't be necessary to "create distinct profile points for every distinct spot in the source code". But I'm still wondering what to do with the LNT/LVT (as debugging information might be lost when merging lambdas)? Note that the only risk is to point to another lambda in the source file but with the same code... Bernard [1] http://mail.openjdk.java.net/pipermail/amber-dev/2018-March/002731.html > ? John From vicente.romero at oracle.com Wed Mar 7 16:51:32 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Wed, 07 Mar 2018 16:51:32 +0000 Subject: hg: amber/amber: adding test to check that javac is not doing DCE for subclasses of j.l.s.Constable Message-ID: <201803071651.w27GpWAA017609@aojmv0008.oracle.com> Changeset: bfe9f5ccf4a0 Author: vromero Date: 2018-03-07 11:46 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/bfe9f5ccf4a0 adding test to check that javac is not doing DCE for subclasses of j.l.s.Constable + test/langtools/tools/javac/specialConstantFolding/dead_code_elimination/DontDoDCEOfConstableTest.java From james.laskey at oracle.com Wed Mar 7 19:27:41 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 7 Mar 2018 15:27:41 -0400 Subject: RFR: JDK-8198986 - 3.10.7: Raw string literals Message-ID: JEP 326: Raw String Literals JDK-8196004 has been approved as a JEP Candidate. I would like to propose this JEP to target sometime soon. Of course, there are many steps to getting there, including the review of the Java? Language Specification. Please find a draft of the required Java? Language Specification changes at JDK-8198986 for your review. Cheers, ? Jim From john.r.rose at oracle.com Wed Mar 7 19:38:06 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 7 Mar 2018 11:38:06 -0800 Subject: deduplicating lambda methods In-Reply-To: References: <4b7cb307-1af7-ccfe-02a0-3358dd75959c@oracle.com> Message-ID: On Mar 7, 2018, at 8:25 AM, B. Blaser wrote: > > Looking at the stats [1], we see that most of the cases are trivial > and mostly related to primitive & boxing types. I'm wondering if JIT > profiles would be affected if de-duplication was simply focusing on > those trivial cases? Good questions. Type profiling is mostly on receivers of virtual calls, sometimes on other arguments or return types, and also on casts and instanceof. Branch profiling is on control flow branches, of course. If a trivial lambda has no such constructs, it's likely that its profile contributes little to the JIT. > If profiles were not polluted in such cases, It wouldn't be necessary > to "create distinct profile points for every distinct spot in the > source code". Correct. > But I'm still wondering what to do with the LNT/LVT (as debugging > information might be lost when merging lambdas)? There might be a performance cost here associated with debuggability. That's rare in Java, but still happens from time to time. The resolution might be to arrange things so that the optimization choice is pushed to runtime, so that the slow-down occurs only in JVMs which somehow know that they are debug mode. Doesn't save the case of attaching to a JVM already started in production mode. This feels like a follow-on project, because it may require very specific refactorings either inside the LMF or to the LMF API to accept contextual information from the capture site, and either discard it conditionally, or put it in some sort of side container specially engineered to be available to debuggers but not affect code identity. And that might require tweaks to debug APIs. > Note that the only risk is to point to another lambda in the source > file but with the same code? ?Or if we take the option to globalize some "top ten" list of shapes. Then you'd step into a lambda body in some java.base library. (Or maybe not step at all?) If your single-stepping takes you to a lambda with the same shape as the one you expect to single-step into, but the source location you actually reach looks unreached, it's confusing but arguably OK. After all lambda identity (as an object) is not guaranteed, and can be pre-allocated. To the debugging programmer, it looks like the compiler and/or JVM chose to preallocate the same lambda but in a different place. Exact source locations are debatable in a way expression semantics are not. Again, full debuggability sometimes conflicts with full performance. ? John From vicente.romero at oracle.com Wed Mar 7 19:43:27 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Wed, 07 Mar 2018 19:43:27 +0000 Subject: hg: amber/amber: move Foldable to jdk.internal.vm.annotation Message-ID: <201803071943.w27JhRxk015890@aojmv0008.oracle.com> Changeset: a22e16cecbc4 Author: vromero Date: 2018-03-07 14:34 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/a22e16cecbc4 move Foldable to jdk.internal.vm.annotation - src/java.base/share/classes/java/lang/annotation/Foldable.java ! src/java.base/share/classes/java/lang/sym/AsTypeMethodHandleRef.java ! src/java.base/share/classes/java/lang/sym/ClassRef.java ! src/java.base/share/classes/java/lang/sym/ConstantMethodHandleRef.java ! src/java.base/share/classes/java/lang/sym/ConstantMethodTypeRef.java ! src/java.base/share/classes/java/lang/sym/ConstantRefs.java ! src/java.base/share/classes/java/lang/sym/DynamicCallSiteRef.java ! src/java.base/share/classes/java/lang/sym/DynamicConstantRef.java ! src/java.base/share/classes/java/lang/sym/EnumRef.java ! src/java.base/share/classes/java/lang/sym/MethodHandleRef.java ! src/java.base/share/classes/java/lang/sym/MethodTypeRef.java ! src/java.base/share/classes/java/lang/sym/RefBootstraps.java ! src/java.base/share/classes/java/lang/sym/VarHandleRef.java + src/java.base/share/classes/jdk/internal/vm/annotation/Foldable.java ! src/java.base/share/classes/module-info.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symtab.java ! test/langtools/tools/javac/specialConstantFolding/dead_code_elimination/DontDoDCEOfConstableTest.java From vicente.romero at oracle.com Wed Mar 7 20:04:06 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Wed, 07 Mar 2018 20:04:06 +0000 Subject: hg: amber/amber: moving Foldable to new package jdk.internal.lang.annotation Message-ID: <201803072004.w27K46L1025226@aojmv0008.oracle.com> Changeset: b87fd4c97bd3 Author: vromero Date: 2018-03-07 14:59 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/b87fd4c97bd3 moving Foldable to new package jdk.internal.lang.annotation ! src/java.base/share/classes/java/lang/sym/AsTypeMethodHandleRef.java ! src/java.base/share/classes/java/lang/sym/ClassRef.java ! src/java.base/share/classes/java/lang/sym/ConstantMethodHandleRef.java ! src/java.base/share/classes/java/lang/sym/ConstantMethodTypeRef.java ! src/java.base/share/classes/java/lang/sym/ConstantRefs.java ! src/java.base/share/classes/java/lang/sym/DynamicCallSiteRef.java ! src/java.base/share/classes/java/lang/sym/DynamicConstantRef.java ! src/java.base/share/classes/java/lang/sym/EnumRef.java ! src/java.base/share/classes/java/lang/sym/MethodHandleRef.java ! src/java.base/share/classes/java/lang/sym/MethodTypeRef.java ! src/java.base/share/classes/java/lang/sym/RefBootstraps.java ! src/java.base/share/classes/java/lang/sym/VarHandleRef.java + src/java.base/share/classes/jdk/internal/lang/annotation/Foldable.java - src/java.base/share/classes/jdk/internal/vm/annotation/Foldable.java ! src/java.base/share/classes/module-info.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symtab.java ! test/langtools/tools/javac/specialConstantFolding/dead_code_elimination/DontDoDCEOfConstableTest.java From vicente.romero at oracle.com Wed Mar 7 21:38:55 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Wed, 07 Mar 2018 21:38:55 +0000 Subject: hg: amber/amber: moving classes under j.l.sym to j.l.i.constant, relocating tests accordingly Message-ID: <201803072138.w27LctXu012876@aojmv0008.oracle.com> Changeset: f6a1ff6755b3 Author: vromero Date: 2018-03-07 16:34 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/f6a1ff6755b3 moving classes under j.l.sym to j.l.i.constant, relocating tests accordingly ! src/java.base/share/classes/java/lang/Class.java ! src/java.base/share/classes/java/lang/Double.java ! src/java.base/share/classes/java/lang/Enum.java ! src/java.base/share/classes/java/lang/Float.java ! src/java.base/share/classes/java/lang/Integer.java ! src/java.base/share/classes/java/lang/Long.java ! src/java.base/share/classes/java/lang/String.java ! src/java.base/share/classes/java/lang/invoke/Intrinsics.java ! src/java.base/share/classes/java/lang/invoke/MethodHandle.java ! src/java.base/share/classes/java/lang/invoke/MethodType.java ! src/java.base/share/classes/java/lang/invoke/VarHandle.java ! src/java.base/share/classes/java/lang/invoke/X-VarHandle.java.template + src/java.base/share/classes/java/lang/invoke/constant/AsTypeMethodHandleRef.java + src/java.base/share/classes/java/lang/invoke/constant/ClassRef.java + src/java.base/share/classes/java/lang/invoke/constant/Constable.java + src/java.base/share/classes/java/lang/invoke/constant/ConstantClassRef.java + src/java.base/share/classes/java/lang/invoke/constant/ConstantMethodHandleRef.java + src/java.base/share/classes/java/lang/invoke/constant/ConstantMethodTypeRef.java + src/java.base/share/classes/java/lang/invoke/constant/ConstantRef.java + src/java.base/share/classes/java/lang/invoke/constant/ConstantRefs.java + src/java.base/share/classes/java/lang/invoke/constant/DynamicCallSiteRef.java + src/java.base/share/classes/java/lang/invoke/constant/DynamicConstantRef.java + src/java.base/share/classes/java/lang/invoke/constant/EnumRef.java + src/java.base/share/classes/java/lang/invoke/constant/MethodHandleRef.java + src/java.base/share/classes/java/lang/invoke/constant/MethodTypeRef.java + src/java.base/share/classes/java/lang/invoke/constant/PrimitiveClassRef.java + src/java.base/share/classes/java/lang/invoke/constant/RefBootstraps.java + src/java.base/share/classes/java/lang/invoke/constant/VarHandleRef.java + src/java.base/share/classes/java/lang/invoke/constant/package-info.java - src/java.base/share/classes/java/lang/sym/AsTypeMethodHandleRef.java - src/java.base/share/classes/java/lang/sym/ClassRef.java - src/java.base/share/classes/java/lang/sym/Constable.java - src/java.base/share/classes/java/lang/sym/ConstantClassRef.java - src/java.base/share/classes/java/lang/sym/ConstantMethodHandleRef.java - src/java.base/share/classes/java/lang/sym/ConstantMethodTypeRef.java - src/java.base/share/classes/java/lang/sym/ConstantRef.java - src/java.base/share/classes/java/lang/sym/ConstantRefs.java - src/java.base/share/classes/java/lang/sym/DynamicCallSiteRef.java - src/java.base/share/classes/java/lang/sym/DynamicConstantRef.java - src/java.base/share/classes/java/lang/sym/EnumRef.java - src/java.base/share/classes/java/lang/sym/MethodHandleRef.java - src/java.base/share/classes/java/lang/sym/MethodTypeRef.java - src/java.base/share/classes/java/lang/sym/PrimitiveClassRef.java - src/java.base/share/classes/java/lang/sym/RefBootstraps.java - src/java.base/share/classes/java/lang/sym/VarHandleRef.java - src/java.base/share/classes/java/lang/sym/package-info.java ! src/java.base/share/classes/jdk/internal/lang/annotation/Foldable.java ! src/java.base/share/classes/module-info.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symtab.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/util/Constables.java + test/jdk/java/lang/invoke/constant/ClassRefTest.java + test/jdk/java/lang/invoke/constant/CondyRefTest.java + test/jdk/java/lang/invoke/constant/IndyRefTest.java + test/jdk/java/lang/invoke/constant/IntrinsifiedRefTest.java + test/jdk/java/lang/invoke/constant/MethodHandleRefTest.java + test/jdk/java/lang/invoke/constant/MethodTypeRefTest.java + test/jdk/java/lang/invoke/constant/SymbolicRefTest.java - test/jdk/java/lang/sym/ClassRefTest.java - test/jdk/java/lang/sym/CondyRefTest.java - test/jdk/java/lang/sym/IndyRefTest.java - test/jdk/java/lang/sym/IntrinsifiedRefTest.java - test/jdk/java/lang/sym/MethodHandleRefTest.java - test/jdk/java/lang/sym/MethodTypeRefTest.java - test/jdk/java/lang/sym/SymbolicRefTest.java ! test/langtools/tools/javac/condy/CheckForCondyDuplicatesTest.java ! test/langtools/tools/javac/specialConstantFolding/CheckForCorrectMRefTest.java ! test/langtools/tools/javac/specialConstantFolding/CondyCodeGenerationTest.java ! test/langtools/tools/javac/specialConstantFolding/CrashWithPrimitiveArgumentsTest.java ! test/langtools/tools/javac/specialConstantFolding/DontCompileIfSymbolCantBeFoundTest.java ! test/langtools/tools/javac/specialConstantFolding/EffectivelyFinalTestNeg.java ! test/langtools/tools/javac/specialConstantFolding/IndyCodeGenerationTest.java ! test/langtools/tools/javac/specialConstantFolding/IndyCrashTest.java ! test/langtools/tools/javac/specialConstantFolding/IndyLinkageErrorTest.java ! test/langtools/tools/javac/specialConstantFolding/IndyNegativeTest01.java ! test/langtools/tools/javac/specialConstantFolding/IndyPositiveTest01.java ! test/langtools/tools/javac/specialConstantFolding/IntrinsicsTest.java ! test/langtools/tools/javac/specialConstantFolding/LDCNegativeTest.java ! test/langtools/tools/javac/specialConstantFolding/MultipleBSMEntriesTest.java ! test/langtools/tools/javac/specialConstantFolding/ReflectiveErrorTest.java ! test/langtools/tools/javac/specialConstantFolding/ReflectiveErrorTest.out ! test/langtools/tools/javac/specialConstantFolding/ReportIncorrectMHForIndyTest.java ! test/langtools/tools/javac/specialConstantFolding/TwoVisitsAreNeededCauseOfForwardRefTest.java ! test/langtools/tools/javac/specialConstantFolding/checkMethodTypeShape/MethodTypeNegTest.java ! test/langtools/tools/javac/specialConstantFolding/checkMethodTypeShape/MethodTypeNegTest.out ! test/langtools/tools/javac/specialConstantFolding/dead_code_elimination/DontDoDCEOfConstableTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/ConstantDefinitions.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/ConstantFoldingOfMethodTypeDiffContextsTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/ConstantPropagationTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/EffectivelyFinalTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindConstructorTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindGetterTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindMethodWithGenericArgumentsTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindSetterTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindStaticGetterTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindStaticSetterTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindStaticTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindVirtualTest01.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/FindVirtualTest02.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/InstanceTrackableMethodsTest.java ! test/langtools/tools/javac/specialConstantFolding/harness/tests/StringFoldingTest.java ! test/langtools/tools/javac/specialConstantFolding/warningNotFoundOrIncorrect/WarningIfClassOrMemberNotFound.java ! test/langtools/tools/javac/specialConstantFolding/warningNotFoundOrIncorrect/WarningIfClassOrMemberNotFound2.java ! test/langtools/tools/javac/specialConstantFolding/warningNotFoundOrIncorrect/WarningIfClassOrMemberNotFound3.java ! test/langtools/tools/javac/specialConstantFolding/warningNotFoundOrIncorrect/WarningIfMemberIncorrect.java From john.r.rose at oracle.com Wed Mar 7 21:41:45 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 7 Mar 2018 13:41:45 -0800 Subject: RFR: JDK-8198986 - 3.10.7: Raw string literals In-Reply-To: References: Message-ID: <3FF68A89-1FB4-4911-8DCD-715CCA38FB98@oracle.com> On Mar 7, 2018, at 11:27 AM, Jim Laskey wrote: > > JEP 326: Raw String Literals JDK-8196004 has been approved as a JEP Candidate. I would like to propose this JEP to target sometime soon. Of course, there are many steps to getting there, including the review of the Java? Language Specification. > > Please find a draft of the required Java? Language Specification changes at JDK-8198986 for your review. Looks good! Glad to see the clarifying examples about backtick count (Bob and Jim's adventures). One problem in the examples: ```````````` Hello, world ```````````` // the twelve characters H e l l o , SP w o r l d I think that should be ```````````` Hello, world ```````````` // the 14 characters NL H e l l o , SP w o r l d NL ? John P.S. When we get to library support, I will be looking for some way to strip *at most* one leading and trailing non-quote character, such as SP or pipe '|', in order to escape the restriction about backticks beginning or ending raw strings. The trim-margins stuff we talked about once will probably do, if it can be configured to trim the SP/pipe char from the end of the raw string as well as the beginning. I like your idea of trimming at most one blank line (NL) from the fore and aft, since that supports multi-line expressions like the Hello, world example above. And a single-line idiom is desirable too. From jan.lahoda at oracle.com Thu Mar 8 15:59:58 2018 From: jan.lahoda at oracle.com (jan.lahoda at oracle.com) Date: Thu, 08 Mar 2018 15:59:58 +0000 Subject: hg: amber/amber: Return and continue should not just outside of a switch expression. Message-ID: <201803081559.w28FxwTM015759@aojmv0008.oracle.com> Changeset: 5a006d1108a4 Author: jlahoda Date: 2018-03-08 16:06 +0100 URL: http://hg.openjdk.java.net/amber/amber/rev/5a006d1108a4 Return and continue should not just outside of a switch expression. ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties ! test/langtools/tools/javac/switchexpr/ExpressionSwitchBreaks1.java ! test/langtools/tools/javac/switchexpr/ExpressionSwitchBreaks2.java ! test/langtools/tools/javac/switchexpr/ExpressionSwitchBreaks2.out From james.laskey at oracle.com Fri Mar 9 19:03:51 2018 From: james.laskey at oracle.com (james.laskey at oracle.com) Date: Fri, 09 Mar 2018 19:03:51 +0000 Subject: hg: amber/amber: 0000000: Revise library support Message-ID: <201803091903.w29J3q5e009353@aojmv0008.oracle.com> Changeset: 5a2e574f43fb Author: jlaskey Date: 2018-03-09 14:58 -0400 URL: http://hg.openjdk.java.net/amber/amber/rev/5a2e574f43fb 0000000: Revise library support ! src/java.base/share/classes/java/lang/String.java ! src/java.base/share/classes/java/lang/StringLatin1.java ! src/java.base/share/classes/java/lang/StringUTF16.java ! test/jdk/java/lang/String/RawStringLiteralLib.java From vicente.romero at oracle.com Fri Mar 9 20:32:44 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Fri, 09 Mar 2018 20:32:44 +0000 Subject: hg: amber/amber: adding parameter-less constructors to records, complete the bodies with super plus field initialization Message-ID: <201803092032.w29KWiA0022930@aojmv0008.oracle.com> Changeset: e6b9e58a272e Author: vromero Date: 2018-03-09 15:27 -0500 URL: http://hg.openjdk.java.net/amber/amber/rev/e6b9e58a272e adding parameter-less constructors to records, complete the bodies with super plus field initialization ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TypeEnter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeInfo.java ! src/jdk.jshell/share/classes/jdk/jshell/ReplParser.java ! test/langtools/tools/javac/datum/NoAddFieldsCanBeDeclaredInDatumTest.out + test/langtools/tools/javac/datum/ParameterLessConstructorsTest.java From amaembo at gmail.com Mon Mar 12 04:48:53 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Mon, 12 Mar 2018 11:48:53 +0700 Subject: RFR: JDK-8198986 - 3.10.7: Raw string literals In-Reply-To: References: Message-ID: Hello! Proposed spec changes look good and unambiguous to me, thanks. We should not have any troubles implementing this in IntelliJ IDEA. After all we already have raw strings support for Kotlin, Groovy, Scala and other languages. I personally still not very happy with the proposed syntax, but seems that I failed to convince anybody in previous discussion [1], so probably I'm wrong (or the world is not yet ready for such changes). To me, as an IDE user, the problem of writing complex string literal is solved many years ago: in IntelliJ IDEA you can edit the embedded code snippet in separate editor view with all the bells and whistles like syntax highlighting, quick-fixes and so on (you can assign the language of your choice to the embedded code snippet), and IDEA will automatically add necessary escaping or split with "...\n" + LF "...". You don't even concentrate on the escaped string, you just switch to the separate editor and see it raw. Even less sophisticated editors support paste inside string literal which will perform necessary escaping. So I would instead concentrate on readability, as people read code in much more places than write (e-mails, code review tools, repository browsers, diff tools, StackOverflow). For me, ability to paste the snippet as is in dumb editor should not be the priority feature. But ok, I already expressed my thoughts and have nothing new to add for now. With best regards, Tagir Valeev. [1] http://mail.openjdk.java.net/pipermail/amber-spec-experts/2018-January/000251.html On Thu, Mar 8, 2018 at 2:27 AM, Jim Laskey wrote: > JEP 326: Raw String Literals JDK-8196004 net/browse/JDK-8196004> has been approved as a JEP Candidate. I would > like to propose this JEP to target sometime soon. Of course, there are many > steps to getting there, including the review of the Java? Language > Specification. > > Please find a draft of the required Java? Language Specification changes > at JDK-8198986 for > your review. > > Cheers, > > ? Jim > > From forax at univ-mlv.fr Mon Mar 12 07:07:18 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 12 Mar 2018 08:07:18 +0100 (CET) Subject: hg: amber/amber: 0000000: Revise library support In-Reply-To: <201803091903.w29J3q5e009353@aojmv0008.oracle.com> References: <201803091903.w29J3q5e009353@aojmv0008.oracle.com> Message-ID: <1316898124.26910.1520838438616.JavaMail.zimbra@u-pem.fr> Hi Jim, i'm not a big fan of using streams inside the String class, for several reasons, it may create VM bootstrap error if one of these methods is used early when bootstrapping the VM when method handles/lambdas are not yet fully initialized and in term of performance, String methods tend to be used at leaf of method graph calls so calling Stream methods in these methods may not be optimized well because there is not enough inlining budget left at that point. regards, R?mi ----- Mail original ----- > De: "Jim Laskey" > ?: "amber-dev" > Envoy?: Vendredi 9 Mars 2018 20:03:51 > Objet: hg: amber/amber: 0000000: Revise library support > Changeset: 5a2e574f43fb > Author: jlaskey > Date: 2018-03-09 14:58 -0400 > URL: http://hg.openjdk.java.net/amber/amber/rev/5a2e574f43fb > > 0000000: Revise library support > > ! src/java.base/share/classes/java/lang/String.java > ! src/java.base/share/classes/java/lang/StringLatin1.java > ! src/java.base/share/classes/java/lang/StringUTF16.java > ! test/jdk/java/lang/String/RawStringLiteralLib.java From james.laskey at oracle.com Mon Mar 12 10:50:54 2018 From: james.laskey at oracle.com (James Laskey) Date: Mon, 12 Mar 2018 07:50:54 -0300 Subject: hg: amber/amber: 0000000: Revise library support In-Reply-To: <1316898124.26910.1520838438616.JavaMail.zimbra@u-pem.fr> References: <201803091903.w29J3q5e009353@aojmv0008.oracle.com> <1316898124.26910.1520838438616.JavaMail.zimbra@u-pem.fr> Message-ID: <072829DE-AE60-4096-9527-EAE4E8476A7C@oracle.com> Yes I am aware of this. Always a issue when working on Jigsaw. Sent from my iPhone > On Mar 12, 2018, at 4:07 AM, Remi Forax wrote: > > Hi Jim, > i'm not a big fan of using streams inside the String class, for several reasons, > it may create VM bootstrap error if one of these methods is used early when bootstrapping the VM when method handles/lambdas are not yet fully initialized and in term of performance, String methods tend to be used at leaf of method graph calls so calling Stream methods in these methods may not be optimized well because there is not enough inlining budget left at that point. > > regards, > R?mi > > ----- Mail original ----- >> De: "Jim Laskey" >> ?: "amber-dev" >> Envoy?: Vendredi 9 Mars 2018 20:03:51 >> Objet: hg: amber/amber: 0000000: Revise library support > >> Changeset: 5a2e574f43fb >> Author: jlaskey >> Date: 2018-03-09 14:58 -0400 >> URL: http://hg.openjdk.java.net/amber/amber/rev/5a2e574f43fb >> >> 0000000: Revise library support >> >> ! src/java.base/share/classes/java/lang/String.java >> ! src/java.base/share/classes/java/lang/StringLatin1.java >> ! src/java.base/share/classes/java/lang/StringUTF16.java >> ! test/jdk/java/lang/String/RawStringLiteralLib.java From vicente.romero at oracle.com Mon Mar 12 15:45:21 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Mon, 12 Mar 2018 15:45:21 +0000 Subject: hg: amber/amber: do DCE for instances of DynamicCallSiteRef too Message-ID: <201803121545.w2CFjMsn029064@aojmv0008.oracle.com> Changeset: 47f797792311 Author: vromero Date: 2018-03-12 11:40 -0400 URL: http://hg.openjdk.java.net/amber/amber/rev/47f797792311 do DCE for instances of DynamicCallSiteRef too ! src/jdk.compiler/share/classes/com/sun/tools/javac/util/Constables.java From james.laskey at oracle.com Mon Mar 12 16:44:02 2018 From: james.laskey at oracle.com (james.laskey at oracle.com) Date: Mon, 12 Mar 2018 16:44:02 +0000 Subject: hg: amber/amber: 0000000: Clean up comments, trimMarkers and use isWhitespace Message-ID: <201803121644.w2CGi2VY001650@aojmv0008.oracle.com> Changeset: fc3f1ec7cd52 Author: jlaskey Date: 2018-03-12 13:39 -0300 URL: http://hg.openjdk.java.net/amber/amber/rev/fc3f1ec7cd52 0000000: Clean up comments, trimMarkers and use isWhitespace ! src/java.base/share/classes/java/lang/String.java ! src/java.base/share/classes/java/lang/StringUTF16.java ! test/jdk/java/lang/String/RawStringLiteralLib.java From jan.lahoda at oracle.com Mon Mar 12 16:56:15 2018 From: jan.lahoda at oracle.com (jan.lahoda at oracle.com) Date: Mon, 12 Mar 2018 16:56:15 +0000 Subject: hg: amber/amber: "case null:" no longer needs to be the first case in a switch; cleaning up non-indy desugaring. Message-ID: <201803121656.w2CGuFqi008602@aojmv0008.oracle.com> Changeset: 89e859ae9375 Author: jlahoda Date: 2018-03-12 17:49 +0100 URL: http://hg.openjdk.java.net/amber/amber/rev/89e859ae9375 "case null:" no longer needs to be the first case in a switch; cleaning up non-indy desugaring. ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Lower.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties ! test/langtools/tools/javac/switchnull/SwitchNull.java + test/langtools/tools/javac/switchnull/SwitchNull.out ! test/langtools/tools/javac/switchnull/SwitchNullNegative.java ! test/langtools/tools/javac/switchnull/SwitchNullNegative.out From james.laskey at oracle.com Tue Mar 13 13:49:26 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Tue, 13 Mar 2018 10:49:26 -0300 Subject: Raw String Literals Library Support Message-ID: <0F481056-67EE-435F-9D3A-E81F1AFC3E7A@oracle.com> Please refer to core-libs-dev at openjdk.java.net to follow the Raw String Literals Library Support discussion . http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-March/051997.html From scolebourne at joda.org Tue Mar 13 22:59:07 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 13 Mar 2018 22:59:07 +0000 Subject: RFR: JDK-8198986 - 3.10.7: Raw string literals In-Reply-To: <3FF68A89-1FB4-4911-8DCD-715CCA38FB98@oracle.com> References: <3FF68A89-1FB4-4911-8DCD-715CCA38FB98@oracle.com> Message-ID: On 7 March 2018 at 21:41, John Rose wrote: > One problem in the examples: > > ```````````` > Hello, world > ```````````` // the twelve characters H e l l o , SP w o r l d Using multiple backtick repetition really is a poor choice as an example like this shows. What if you have 12 backticks at the start but 13 at the end? While the compiler will catch it (at the expense of a weird rule that raw string literals can't start/end with a backtick), your eye cannot. Adding a language feature that your eye can easily get wrong is a mistake (and what a boring/hellish Java certification question). Java also has no other feature where unlimited repetitions are used to indicate something important. Somewhere along the line this feature has simply been over-thought. Raw strings need to be a *simple* form that copes with most use cases. It should use a single well-defined start/end that is not subject to unlimited repetition. It would be *much* simpler to have a single backtick at the start/end with two backticks internally acting as an escape. It also eliminates the weird no backtick at start/end rule. `` = "" (empty) ``` - compile error ```` = "`" (single backtick) ```AAA``` = "`AAA`" (AAA surrounded by backticks) Simple, straightforward, and far more in line with expectations (and much easier for pasting code in/out). Please can we stop the backtick repetition craziness now! Stephen From john.r.rose at oracle.com Tue Mar 13 23:10:56 2018 From: john.r.rose at oracle.com (John Rose) Date: Tue, 13 Mar 2018 16:10:56 -0700 Subject: RFR: JDK-8198986 - 3.10.7: Raw string literals In-Reply-To: References: <3FF68A89-1FB4-4911-8DCD-715CCA38FB98@oracle.com> Message-ID: <12135679-F69E-44EE-AE90-3ED60F5D7A21@oracle.com> On Mar 13, 2018, at 3:59 PM, Stephen Colebourne wrote: > > Simple, straightforward, and far more in line with expectations (and > much easier for pasting code in/out). > > Please can we stop the backtick repetition craziness now! We considered your proposal and decided against it. You are moving the craziness around, not removing it. The pigeonholing principle prevents means there has to be an escape or special syntax somewhere, and you are putting it into the body of the string, wherever a backtick occurs (you double it). We have decided to keep the special syntax completely outside of the pasted string body. In essence, the doubled backticks are forced into the boundary of the string body. While you say you would prefer to hunt through the body of the string looking for doubled backticks, I think most folks will be relieved that they can quickly spot the boundaries of the string, no matter how baroque, and then read the string body forgetting that it is inside some other container. You correctly observe that this sort of thing is a rich source of puzzlers. Again, that is true no matter what we do. I think in low-level syntax questions like this there will always be some of us (unfortunately you in this case, sorry!) who are repulsed by the decision. It's natural to make the mistake of fleeing to some other choice but failing to see the downsides that other choice. In this case, the trade-off is to let the backticks pile up on the edge of the string, in order to allow them to exist undisturbed in the interior. An edge effect is always smaller than a bulk effect, or large bulks. Hence our design. ? John From scolebourne at joda.org Tue Mar 13 23:15:20 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 13 Mar 2018 23:15:20 +0000 Subject: Switch expression fall through Message-ID: A recent email to amber-spec implies that expression switch will support fall-through. http://mail.openjdk.java.net/pipermail/amber-spec-experts/2018-March/000332.html Is this true? If so, on what basis is this justified? (Fall through in switch is the spawn of the devil. Allowing it in the brand new expression switch would be horrific IMO.) Stephen From scolebourne at joda.org Tue Mar 13 23:36:10 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 13 Mar 2018 23:36:10 +0000 Subject: RFR: JDK-8198986 - 3.10.7: Raw string literals In-Reply-To: <12135679-F69E-44EE-AE90-3ED60F5D7A21@oracle.com> References: <3FF68A89-1FB4-4911-8DCD-715CCA38FB98@oracle.com> <12135679-F69E-44EE-AE90-3ED60F5D7A21@oracle.com> Message-ID: On 13 March 2018 at 23:10, John Rose wrote: > While you say you would prefer to hunt through > the body of the string looking for doubled backticks, No, I expect my IDE to double the backticks when I paste. In the unlikely event that there are backticks. > You correctly observe that this sort of thing is a > rich source of puzzlers. Again, that is true no > matter what we do. The proposal as it stands has far more puzzlers. * getting the count of backticks mismatched at start/end * no ability to have an empty raw string * no ability to start/end the raw string with a backtick * adding backticks in the middle always requires checking to see if the boundary is long enough The much simpler single backtick proposal has none of these flaws. > In this case, the trade-off is to let the backticks > pile up on the edge of the string, in order to > allow them to exist undisturbed in the interior. > An edge effect is always smaller than a bulk > effect, or large bulks. Hence our design. I get why it appeals to a language designer. However it is my view that practising developers would disagree. Dealing with a simple escape mechanism is considerably preferable to remembering the laundry list of weird rules currently proposed. Stephen From brian.goetz at oracle.com Wed Mar 14 01:16:25 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 13 Mar 2018 21:16:25 -0400 Subject: Switch expression fall through In-Reply-To: References: Message-ID: Expression switches and statement switches are the same construct. There are some mild differences around the edges (i.e., one returns a value, the other does not), but they are one construct. I get that you hate fall through, but we can?t rid ourselves of fall through (we?ll still have switch statements), and having two subtly different switch constructs is even worse. Java?s switch construct, for better or worse, has fall through. (And, by the way, there?s nothing wrong with fall through. The mistake is _fallthrough by default_. Fall through itself is perfectly reasonable; its just that the default is wrong. Like so many others.) But, I?m repeating stuff you could just as well have read on amber-spec-experts, so I?ll stop here. > On Mar 13, 2018, at 7:15 PM, Stephen Colebourne wrote: > > A recent email to amber-spec implies that expression switch will > support fall-through. > http://mail.openjdk.java.net/pipermail/amber-spec-experts/2018-March/000332.html > > Is this true? If so, on what basis is this justified? > > (Fall through in switch is the spawn of the devil. Allowing it in the > brand new expression switch would be horrific IMO.) > > Stephen From ali.ebrahimi1781 at gmail.com Wed Mar 14 08:00:08 2018 From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi) Date: Wed, 14 Mar 2018 11:30:08 +0330 Subject: Switch expression fall through In-Reply-To: References: Message-ID: Hi, I fully support current proposal, since for java users that doesn't add breaking and surprising rules. We can't change java's old rules. On Wed, Mar 14, 2018 at 4:46 AM, Brian Goetz wrote: > Expression switches and statement switches are the same construct. There > are some mild differences around the edges (i.e., one returns a value, the > other does not), but they are one construct. I get that you hate fall > through, but we can?t rid ourselves of fall through (we?ll still have > switch statements), and having two subtly different switch constructs is > even worse. Java?s switch construct, for better or worse, has fall through. > > (And, by the way, there?s nothing wrong with fall through. The mistake is > _fallthrough by default_. Fall through itself is perfectly reasonable; its > just that the default is wrong. Like so many others.) > > But, I?m repeating stuff you could just as well have read on > amber-spec-experts, so I?ll stop here. > > > > On Mar 13, 2018, at 7:15 PM, Stephen Colebourne > wrote: > > > > A recent email to amber-spec implies that expression switch will > > support fall-through. > > http://mail.openjdk.java.net/pipermail/amber-spec-experts/ > 2018-March/000332.html > > > > Is this true? If so, on what basis is this justified? > > > > (Fall through in switch is the spawn of the devil. Allowing it in the > > brand new expression switch would be horrific IMO.) > > > > Stephen > > -- Best Regards, Ali Ebrahimi From gavin.bierman at oracle.com Wed Mar 14 16:30:03 2018 From: gavin.bierman at oracle.com (Gavin Bierman) Date: Wed, 14 Mar 2018 16:30:03 +0000 Subject: Switch expression fall through In-Reply-To: References: Message-ID: <4F9B81DE-C93F-44C7-B370-DABD4CD49034@oracle.com> > On 13 Mar 2018, at 23:15, Stephen Colebourne wrote: > > A recent email to amber-spec implies that expression switch will > support fall-through. > http://mail.openjdk.java.net/pipermail/amber-spec-experts/2018-March/000332.html > > Is this true? If so, on what basis is this justified? > > (Fall through in switch is the spawn of the devil. Allowing it in the > brand new expression switch would be horrific IMO.) If you write all your expression switches using the -> form exclusively, then you will be able to forget that fall-through even exists. Gavin From scolebourne at joda.org Wed Mar 14 17:57:09 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 14 Mar 2018 17:57:09 +0000 Subject: Switch expression fall through In-Reply-To: <4F9B81DE-C93F-44C7-B370-DABD4CD49034@oracle.com> References: <4F9B81DE-C93F-44C7-B370-DABD4CD49034@oracle.com> Message-ID: On 14 March 2018 at 16:30, Gavin Bierman wrote: >> On 13 Mar 2018, at 23:15, Stephen Colebourne wrote: >> >> A recent email to amber-spec implies that expression switch will >> support fall-through. >> http://mail.openjdk.java.net/pipermail/amber-spec-experts/2018-March/000332.html >> >> Is this true? If so, on what basis is this justified? >> >> (Fall through in switch is the spawn of the devil. Allowing it in the >> brand new expression switch would be horrific IMO.) > > If you write all your expression switches using the -> form exclusively, then you will be able to forget that fall-through even exists. Yes. I am reading amber-spec. But being outside the spec-experts loop I can't comment in real time. I do however find myself cheering Kevin on to keep challenging this. His recent mail about whether the design is targetting real developers, and not just compiler/language experts chimes exactly with where I'm at: http://mail.openjdk.java.net/pipermail/amber-spec-experts/2018-March/000343.html In both current design issues (raw string literals and expression switch) I fully understand why the Oracle team has come to the conclusions they have. From a pure language design POV they make sense - the explanations are all perfectly rational. Yet in both cases they feel like the wrong conclusions for Java developers beyond the expert level. And they don't have that sense of blending in. That judgement is of course highly subjective, which is why I spent time in another thread highlighting how the raw string literal design has far far too many puzzlers to take that discussion beyond pure subjectivity. Getting back to expression switches, Brian asked "In the meantime, let me probe for what's really uncomfortable about the current design point". This was wrt using break to return a value, but I think the narrow point is merely a manifestation of the difficultly of integrating the whole feature. Specifically: * Java has no way to return a value from a block, except where it is a method in disguise (lambdas, inner classes) * Java has never previously converted a block based control-flow statement to an expression * the closest parallel is if..else, which has a completely different form (ternary) as an expression * ternary expressions do not have blocks * I can't remember ever refactoring a ternary to if..else Given the above, the oft-repeated mantra of wanting to extend the existing switch statement is a very odd place to drive the feature from. Everyone wants a feature that fits well into Java, but it can be argued that a completely different syntax closer to ternary would make as much sense as extending switch. eg.: return (dayOfWeek) ?? case MONDAY: "M" case TUESDAY: "Tu" .... (Discussion of extending switch to expressions have so far assumed the first set of braces between switch and case, but it need not be that way. Since expressions in Java today don't have braces unless they are defining methods, I'd argue that expression switches fit much better without those braces. This syntax form would also look fine passed directly to a method in a way that a brace-based form would not.) Consider Kevin's comment "So, from my perspective, if we just adopt a style rule for Google Style that when using switch in an expression context one should stick to `->`, I might have basically what I want." This should be a huge red flag to the proposed design. That the best teams will ban the use of a significant proportion of the new feature on arrival is surely wrong. (Again, I completely understand the "N case is significantly different from the 2 case" argument for why blocks in expression switches might be more common. Yet I still contend as Kevin and Remi have that blocks are an edge case here that blocks a much nicer solution to the problem space. Lambdas are not a suitable comparison as they define methods, so form a completely different conceptual grouping for developers). In summary, a key issue here is that blocks in Java have never returned values. And the syntax choices for making them do so all seem poor. So, is using `break` to return a value wrong? Perhaps its simply the wrong question. Stephen From scolebourne at joda.org Wed Mar 14 18:02:34 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 14 Mar 2018 18:02:34 +0000 Subject: RFR: JDK-8198986 - 3.10.7: Raw string literals In-Reply-To: References: <3FF68A89-1FB4-4911-8DCD-715CCA38FB98@oracle.com> <12135679-F69E-44EE-AE90-3ED60F5D7A21@oracle.com> Message-ID: On 13 March 2018 at 23:36, Stephen Colebourne wrote: > On 13 March 2018 at 23:10, John Rose wrote: >> While you say you would prefer to hunt through >> the body of the string looking for doubled backticks, > > No, I expect my IDE to double the backticks when I paste. In the > unlikely event that there are backticks. For the record, I'd also find a syntax with a fixed number of backticks/quotes and no escaping preferable to the unlimited backticks of the Oracle proposal. `````` = "" (empty) ``````` = "`" (single backtick) ````AAA```` = "`AAA`" (AAA surrounded by backticks) While there is the very rare scenario it can't cope with, it is far less puzzler strewn. Stephen From brian.goetz at oracle.com Wed Mar 14 18:23:49 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 14 Mar 2018 14:23:49 -0400 Subject: RFR: JDK-8198986 - 3.10.7: Raw string literals In-Reply-To: References: <3FF68A89-1FB4-4911-8DCD-715CCA38FB98@oracle.com> <12135679-F69E-44EE-AE90-3ED60F5D7A21@oracle.com> Message-ID: <31e848a0-4742-c838-df6e-a646811995b8@oracle.com> For the record, we explored these approaches, and, while we also liked them for the first few minutes (probably for the same reason you do), they eventually lose their lustre. Stepping out of the micro-syntactic morass, the primary goal of this feature is to make it possible to take a well-formed snippet of some other language (SQL, Regex, JSON, HTML, Python, whatever) and embed it, _free of the need to do any localized fixups_, in Java code.? Picking a fixed number of ticks, plus an escaping rule, may make it less likely you have to do fixups, but ultimately just moves the problem down the road.? The N-delimiter solution allows you to pick a non-conflicting delimiter, and then not have to fuss with the snippet _at all, ever_. For common cases (where a program has no or single or double embedded ticks), the result is equivalent to having picked three ticks, but more flexible.? I get that you would rather not have that flexibility, but you must also get that there's room for reasonable people to disagree on where to draw the line? On 3/14/2018 2:02 PM, Stephen Colebourne wrote: > On 13 March 2018 at 23:36, Stephen Colebourne wrote: >> On 13 March 2018 at 23:10, John Rose wrote: >>> While you say you would prefer to hunt through >>> the body of the string looking for doubled backticks, >> No, I expect my IDE to double the backticks when I paste. In the >> unlikely event that there are backticks. > For the record, I'd also find a syntax with a fixed number of > backticks/quotes and no escaping preferable to the unlimited backticks > of the Oracle proposal. > > `````` = "" (empty) > ``````` = "`" (single backtick) > ````AAA```` = "`AAA`" (AAA surrounded by backticks) > > While there is the very rare scenario it can't cope with, it is far > less puzzler strewn. > > Stephen From john.r.rose at oracle.com Wed Mar 14 18:40:47 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 14 Mar 2018 11:40:47 -0700 Subject: RFR: JDK-8198986 - 3.10.7: Raw string literals In-Reply-To: <31e848a0-4742-c838-df6e-a646811995b8@oracle.com> References: <3FF68A89-1FB4-4911-8DCD-715CCA38FB98@oracle.com> <12135679-F69E-44EE-AE90-3ED60F5D7A21@oracle.com> <31e848a0-4742-c838-df6e-a646811995b8@oracle.com> Message-ID: <16BA3263-DBE1-47AA-9A4F-F13345C4F936@oracle.com> On Mar 14, 2018, at 11:23 AM, Brian Goetz wrote: > > Picking a fixed number of ticks, plus an escaping rule, may make it less likely > you have to do fixups, but ultimately just moves the problem down the road. ?where in terms of my previous message "the problem" is having to read through the bulk of the string for escapes, instead of consulting the edges only. One mental test that I have for these designs is to ask what a programmer would experience when nesting Java source code inside of Java source code. This is a stress test, but a significant proxy for embedding other random language in Java source code. Any fixed limit to the number of quotes also puts a fixed limit to nesting depth. Again, not that we want programmers to do deep nesting, but an arbitrary limit is a sharp edge, and the smaller the sharper. In practice, as Brian said, people won't often abuse the notation, except to win obfuscated code contests. It's easy to find abusable points throughout any language design, but "abusus non tollit usum". int o0olo1o0 = 01_01; // abusus int n = 5; // what people really write http://merriam-webster.com/dictionary/abusus+non+tollit+usum ? John From paul.sandoz at oracle.com Wed Mar 14 19:06:58 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 14 Mar 2018 12:06:58 -0700 Subject: RFR: JDK-8198986 - 3.10.7: Raw string literals In-Reply-To: <31e848a0-4742-c838-df6e-a646811995b8@oracle.com> References: <3FF68A89-1FB4-4911-8DCD-715CCA38FB98@oracle.com> <12135679-F69E-44EE-AE90-3ED60F5D7A21@oracle.com> <31e848a0-4742-c838-df6e-a646811995b8@oracle.com> Message-ID: <4430EF72-FBEF-4846-90FA-3BD8A7E3AAD5@oracle.com> > On Mar 14, 2018, at 11:23 AM, Brian Goetz wrote: > > Stepping out of the micro-syntactic morass, the primary goal of this feature is to make it possible to take a well-formed snippet of some other language (SQL, Regex, JSON, HTML, Python, whatever) and embed it, _free of the need to do any localized fixups_, in Java code. And also extract it _free of the need to revert any local fixups_. Further, it more easily enables some parser to directly operate on that source (e.g. what if the IDE recognizes the string literal contents as SQL or even Java source? perhaps it could provide syntax highlighting and basic correction?). Further, when using an IDE it?s easy to underestimate how useful tools like grep are. Paul. From scolebourne at joda.org Wed Mar 14 21:35:08 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 14 Mar 2018 21:35:08 +0000 Subject: Patterns and nulls In-Reply-To: <0cb9a653-d087-e4a2-da22-6d5395305580@oracle.com> References: <0cb9a653-d087-e4a2-da22-6d5395305580@oracle.com> Message-ID: On 14 March 2018 at 16:58, Brian Goetz wrote: > Let's assume we have the following declarations: > > record Box(Object); > Object o; > String s; > Box b; > > Implicitly, `Box` has a deconstruction pattern whose signature is `Box(out > Object o)`. > > What will users expect on the following? > > Box b = new Box(null); > switch (b) { > case Box(Candy x): ... > case Box(Frog f): ... > case Box(Object o): ... > } > > There are four non-ridiculous possibilities: > - NPE > - Match none > - Match Box(Candy) > - Match Box(Object) If records were changed to not allow null, then `new Box(null)` would not be possible. Would that not change the line of reasoning here? Stephen From brian.goetz at oracle.com Wed Mar 14 21:47:15 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 14 Mar 2018 17:47:15 -0400 Subject: Patterns and nulls In-Reply-To: References: <0cb9a653-d087-e4a2-da22-6d5395305580@oracle.com> Message-ID: <3d789aba-faf0-177e-e9e8-79a0314891b8@oracle.com> > If records were changed to not allow null, then `new Box(null)` would > not be possible. Would that not change the line of reasoning here? No.? Pattern matching is not just for records, it's just that (a) like other conveniences, records come with deconstruction patterns free out of the box, and (b) records are likely to be the first construct against which will be able to pattern match.? But over time, we'll extend pattern matching to arbitrary classes and more. Even if it did simplify pattern matching, it seems like an unreasonable restriction on records.? Like it or not, nulls are part of Java's data model.? (I understand the desire to not let _new_ constructs be polluted by old mistakes, but its a siren song. Everything is connected to everything else.? It's one more case where you can't refactor a class to a record.? And having complex rules of "you can have null in these classes but not those" just makes things more complicated.) And even if it were a reasonable restriction, it would further interact poorly with mutable records.? Mutable records could sport a no-arg constructor, but then you'd need a sensible non-null default for every field.? There might be sensible defaults for Strings (""), Lists (empty list), etc, but that's surely not true for all the things you might put in a record. From scolebourne at joda.org Wed Mar 14 21:52:11 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 14 Mar 2018 21:52:11 +0000 Subject: Record construction In-Reply-To: <977efc77-f9f5-6d7d-936e-9404dc7d4300@oracle.com> References: <977efc77-f9f5-6d7d-936e-9404dc7d4300@oracle.com> Message-ID: On 14 March 2018 at 14:55, Brian Goetz wrote: > The constructor syntax > > record Point(int x, int y) { > Point { > } > } > > is proposed as a shorthand for an explicit default constructor: Minor tweak, but class names can get quite long. A keyword would be preferable: record Point(int x, int y) { new { Preconditions.require(x > 0); } } It also does not clash with existing constructors if a way were found to add `new{}` to non-record classes. > This makes both validation and normalization work; if the constructor body > contains only: > > Preconditions.require(x > 0); This example skips some boilerplate, so is a little misleading. The correct invocation (using our library) would be one of these: ArgChecker.isTrue(x > 0, "x must be positive"); ArgChecker.isPositive(x, "x"); This repetition is one reason why `requires x > 0` has appeal (as it can infer the variable name). While not entirely off the fence, I'm coming down on the side that says it is better to allow any preconditions library, rather than just make it a language feature. Wild idea - is there a way for the library method to obtain the variable name and type used at the call site?: public static void isTrue(CallsiteBooleanExpression expr) { if (expr.isFalse()) { throw new IllegalArgumentExpression(expr.expressionString() + " must be true"); } } Stephen From brian.goetz at oracle.com Wed Mar 14 22:09:03 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 14 Mar 2018 18:09:03 -0400 Subject: Record construction In-Reply-To: References: <977efc77-f9f5-6d7d-936e-9404dc7d4300@oracle.com> Message-ID: <00801f13-dc4d-dcf4-c167-b673c69857e1@oracle.com> > Minor tweak, but class names can get quite long. A keyword would be preferable: Yes, that's also a realistic option, and reminiscent of instance initializers (but has the advantage of _not_ being an instance initializer, which already means something.)? From a readability perspective, I have a subjective preference for something that looks like a constructor, but I'd not rule this out.? (Remember, reading code is more important than writing code, so I don't think the length of the name is really the primary consideration.) > Wild idea - is there a way for the library method to obtain the > variable name and type used at the call site?: > > public static void isTrue(CallsiteBooleanExpression expr) { > if (expr.isFalse()) { > throw new IllegalArgumentExpression(expr.expressionString() + " > must be true"); > } > } > Putting it in terms of a variable name and type doesn't make sense, because there might be multiple variable names and types: ??? Preconditions.require(x < y); Doing so would require something like expression trees, a major project. A less intrusive path (but still not trivial) would be to factor out the dual "evaluate and textify" behavior of assert into a language feature, so that assert()-style methods could be written as ordinary library methods, and the method could receive, somehow, both the value and the text of a condition: ??? MyLibrary.require( x < y ) Not crazy, but you've got to go a long way for it, so "wild" sounds about right. From forax at univ-mlv.fr Wed Mar 14 23:49:32 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 15 Mar 2018 00:49:32 +0100 (CET) Subject: Record construction In-Reply-To: <00801f13-dc4d-dcf4-c167-b673c69857e1@oracle.com> References: <977efc77-f9f5-6d7d-936e-9404dc7d4300@oracle.com> <00801f13-dc4d-dcf4-c167-b673c69857e1@oracle.com> Message-ID: <91530965.1325760.1521071372428.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Brian Goetz" > ?: "Stephen Colebourne" , "amber-dev" > Envoy?: Mercredi 14 Mars 2018 23:09:03 > Objet: Re: Record construction >> Minor tweak, but class names can get quite long. A keyword would be preferable: > > Yes, that's also a realistic option, and reminiscent of instance > initializers (but has the advantage of _not_ being an instance > initializer, which already means something.)? From a readability > perspective, I have a subjective preference for something that looks > like a constructor, but I'd not rule this out.? (Remember, reading code > is more important than writing code, so I don't think the length of the > name is really the primary consideration.) > > >> Wild idea - is there a way for the library method to obtain the >> variable name and type used at the call site?: >> >> public static void isTrue(CallsiteBooleanExpression expr) { >> if (expr.isFalse()) { >> throw new IllegalArgumentExpression(expr.expressionString() + " >> must be true"); >> } >> } >> > > Putting it in terms of a variable name and type doesn't make sense, > because there might be multiple variable names and types: > > ??? Preconditions.require(x < y); > > Doing so would require something like expression trees, a major project. > > A less intrusive path (but still not trivial) would be to factor out the > dual "evaluate and textify" behavior of assert into a language feature, > so that assert()-style methods could be written as ordinary library > methods, and the method could receive, somehow, both the value and the > text of a condition: > > ??? MyLibrary.require( x < y ) > > Not crazy, but you've got to go a long way for it, so "wild" sounds > about right. an implicit textualisation, we have briefly talk about that when looking for serializing lambdas class MyLibrary { interface MyBooleanSupplier extends BooleanSupplier & Textualization { } void require(MyBooleanSupplier supplier) { supplier.toString() // get "x < y" suuplier.get() // call () -> x < y with x and y bounded ... } } R?mi From scolebourne at joda.org Thu Mar 15 08:32:57 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 15 Mar 2018 08:32:57 +0000 Subject: Record construction In-Reply-To: <00801f13-dc4d-dcf4-c167-b673c69857e1@oracle.com> References: <977efc77-f9f5-6d7d-936e-9404dc7d4300@oracle.com> <00801f13-dc4d-dcf4-c167-b673c69857e1@oracle.com> Message-ID: On 14 March 2018 at 22:09, Brian Goetz wrote: >> Minor tweak, but class names can get quite long. A keyword would be >> preferable: > > > Yes, that's also a realistic option, and reminiscent of instance > initializers (but has the advantage of _not_ being an instance initializer, > which already means something.) From a readability perspective, I have a > subjective preference for something that looks like a constructor, but I'd > not rule this out. (Remember, reading code is more important than writing > code, so I don't think the length of the name is really the primary > consideration.) In my experience reading code, constructors tend to blend into method too easily. A keyword (that is usually highlighted in a different colour) would make the initializer read as being more clearly different to methods. >> Wild idea - is there a way for the library method to obtain the >> variable name and type used at the call site?: > > Putting it in terms of a variable name and type doesn't make sense, because > there might be multiple variable names and types: > > Preconditions.require(x < y); > > Doing so would require something like expression trees, a major project. I wasn't thinking of that. Just something that can provide the expression as a sting. I think Remi captured it in his response. Its not an essential thing, but the repetition without it is a factor in judging the merits of a `requires` keyword.. Stephen From cushon at google.com Thu Mar 15 18:52:07 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Thu, 15 Mar 2018 18:52:07 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <4b7cb307-1af7-ccfe-02a0-3358dd75959c@oracle.com> Message-ID: On Wed, Mar 7, 2018 at 11:38 AM John Rose wrote: > > Note that the only risk is to point to another lambda in the source > > file but with the same code? > > ?Or if we take the option to globalize some "top ten" list of shapes. > Then you'd step into a lambda body in some java.base library. > (Or maybe not step at all?) > > If your single-stepping takes you to a lambda with the same shape > as the one you expect to single-step into, but the source location > you actually reach looks unreached, it's confusing but arguably OK. > > After all lambda identity (as an object) is not guaranteed, and can > be pre-allocated. To the debugging programmer, it looks like the > compiler and/or JVM chose to preallocate the same lambda but > in a different place. Exact source locations are debatable in a way > expression semantics are not. Again, full debuggability sometimes > conflicts with full performance. > I agree that stepping in to a lambda body and jumping to a different line with identical code might not be *too* surprising. Perhaps more concerning is the handling of breakpoints: if a breakpoint is set in a lambda body that gets deduplicated (or is implemented using one of the globalized lambdas), the breakpoint will never be hit. This seems more confusing and harder to work around than stepping to an incorrect but equivalent line. I don't have any good ideas here aside from something like the follow-on project you described of adding additional information to the capture site that could be used by the debugger. That sounds feasible but non-trivial, and I wonder whether lambda deduplication alone provides enough motivation to add support for that. From cushon at google.com Thu Mar 15 18:52:59 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Thu, 15 Mar 2018 18:52:59 +0000 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: I uploaded a webrev with the current sketch: http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.00/webrev/ * Nothing here is intended to be final, I'm interested in feedback on the direction and any suggestions for next steps. * It shows the approach used for ast comparison, including the lambda parameter handling I described. The lambda parameter handling is currently baked-in, but it could easily be refactored to allow the ast diffing to be used in other contexts. * There is no handling of debug info yet. On Sun, Mar 4, 2018 at 7:59 PM Liam Miller-Cushon wrote: > On Fri, Mar 2, 2018 at 8:01 PM, Vicente Romero > wrote: > >> 1) How to identifying duplicates: I have a prototype that runs during >>> lambda desugaring and identifies duplicates by diffing ASTs. Is that the >>> best place for deduplication, or it worth considering comparing generated >>> code instead of ASTs? >>> >> >> are you doing an exact diff? I assume that we want: s -> s to be equal to >> z -> z provided that the target is the same > > > Lambda parameters are currently handled as a special case. The diff tests > that the syntax for the two trees is the same, and that the symbols > associated with identifier and select nodes are equivalent. It accepts > symbols that correspond to the same parameter in each of the associated > lambda methods. > > E.g. when diffing the lambda bodies for `s -> s` and `x -> x` it sees that > `s` and `x` are both the first parameter of their enclosing lambdas. > From brian.goetz at oracle.com Thu Mar 15 19:00:28 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 15 Mar 2018 15:00:28 -0400 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: <24d4a6e3-0a7e-432a-dcb6-b9195883cd3e@oracle.com> This seems pleasantly straightforward. I think we wanted to exclude serializable lambdas from being deduplicated?? Hopefully that's one more test here: ?369???????? if (existing != null) { ?370???????????? sym = existing.symbol; Stats on its effectiveness on a representative corpus would be useful. Hopefully the hash code is discriminative enough that TreeDiffer doesn't actually run that often. On 3/15/2018 2:52 PM, Liam Miller-Cushon wrote: > I uploaded a webrev with the current sketch: > http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.00/webrev/ > > > * Nothing here is intended to be final, I'm interested in feedback on > the direction and any suggestions for next steps. > * It shows the approach used for ast comparison, including the lambda > parameter handling I described. The lambda parameter handling is > currently baked-in, but it could easily be refactored to allow the ast > diffing to be used in other contexts. > * There is no handling of debug info yet. > > On Sun, Mar 4, 2018 at 7:59 PM Liam Miller-Cushon > wrote: > > On Fri, Mar 2, 2018 at 8:01 PM, Vicente Romero > > wrote: > > 1) How to identifying duplicates: I have a prototype that > runs during lambda desugaring and identifies duplicates by > diffing ASTs. Is that the best place for deduplication, or > it worth considering comparing generated code instead of ASTs? > > > are you doing an exact diff? I assume that we want: s -> s to > be equal to z -> z provided that the target is the same > > > Lambda parameters are currently handled as a special case. The > diff tests that the syntax for the two trees is the same, and that > the symbols associated with identifier and select nodes are > equivalent. It accepts symbols that correspond to the same > parameter in each of the associated lambda methods. > > E.g. when diffing the lambda bodies for `s -> s` and `x -> x` it > sees that `s` and `x` are both the first parameter of their > enclosing lambdas. > From vicente.romero at oracle.com Thu Mar 15 20:31:59 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Thu, 15 Mar 2018 16:31:59 -0400 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> Hi Liam, overall looks good, some comments after eyeballing the patch: - it seems like: map dedupedLambda, could be a set. - TreeDiffer, consider the benefits / trade-offs of extending com.sun.tools.javac.tree.TreeScanner instead of implementing TreeVisitor. The code will change a bit and you will probably need to declare a boolean field for the result but you will have access to the subclasses of JCTree without casting and will be able to use the getTag() method in JCTree, in most cases being JCExpression the exception, to avoid most instanceof checks. - symbols are unique so you can use == to test equality - you don't need: Objects.equals(tree.isStatic(), that.isStatic()), to compare two booleans - names are also unique so you can use == instead of Objects.equals(tree.getLabel(), that.getLabel()) - in general there are complete implementations for trees that can't appear inside the method's body, but I think that they were provided for completeness. Thanks, Vicente On 03/15/2018 02:52 PM, Liam Miller-Cushon wrote: > I uploaded a webrev with the current sketch: > http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.00/webrev/ > > > * Nothing here is intended to be final, I'm interested in feedback on > the direction and any suggestions for next steps. > * It shows the approach used for ast comparison, including the lambda > parameter handling I described. The lambda parameter handling is > currently baked-in, but it could easily be refactored to allow the ast > diffing to be used in other contexts. > * There is no handling of debug info yet. > > On Sun, Mar 4, 2018 at 7:59 PM Liam Miller-Cushon > wrote: > > On Fri, Mar 2, 2018 at 8:01 PM, Vicente Romero > > wrote: > > 1) How to identifying duplicates: I have a prototype that > runs during lambda desugaring and identifies duplicates by > diffing ASTs. Is that the best place for deduplication, or > it worth considering comparing generated code instead of ASTs? > > > are you doing an exact diff? I assume that we want: s -> s to > be equal to z -> z provided that the target is the same > > > Lambda parameters are currently handled as a special case. The > diff tests that the syntax for the two trees is the same, and that > the symbols associated with identifier and select nodes are > equivalent. It accepts symbols that correspond to the same > parameter in each of the associated lambda methods. > > E.g. when diffing the lambda bodies for `s -> s` and `x -> x` it > sees that `s` and `x` are both the first parameter of their > enclosing lambdas. > From maurizio.cimadamore at oracle.com Thu Mar 15 21:10:05 2018 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 15 Mar 2018 21:10:05 +0000 Subject: hg: amber/amber: Automatic merge with default Message-ID: <201803152110.w2FLA5Al019469@aojmv0008.oracle.com> Changeset: d677c3c3ca5d Author: mcimadamore Date: 2018-03-15 22:06 +0100 URL: http://hg.openjdk.java.net/amber/amber/rev/d677c3c3ca5d Automatic merge with default - make/data/x11wrappergen/functions.txt - make/data/x11wrappergen/sizes.32 - make/data/x11wrappergen/sizes.64 - make/data/x11wrappergen/sizes.64-solaris-i386 - make/devkit/createSolarisDevkit.sh - make/hotspot/src/native/dtrace/generateJvmOffsetsMain.c - make/lib/LibosxLibraries.gmk - make/lib/NetworkingLibraries.gmk - make/lib/NioLibraries.gmk - make/lib/PlatformLibraries.gmk - make/lib/SecurityLibraries.gmk - src/hotspot/share/gc/parallel/cardTableExtension.cpp - src/hotspot/share/gc/parallel/cardTableExtension.hpp - src/hotspot/share/gc/shared/cardTableModRefBSForCTRS.cpp - src/hotspot/share/logging/logTagLevelExpression.cpp - src/hotspot/share/logging/logTagLevelExpression.hpp - src/hotspot/share/memory/universe_ext.cpp - src/hotspot/share/services/allocationContextService.hpp - src/java.base/share/classes/javax/security/auth/Policy.java - src/java.base/share/classes/sun/security/provider/AuthPolicyFile.java - src/java.base/unix/classes/java/io/FileDescriptor.java - src/java.base/windows/classes/java/io/FileDescriptor.java - src/java.management/share/classes/sun/management/TypeVersionMapper.java - src/java.sql/share/classes/javax/transaction/xa/XAException.java - src/java.sql/share/classes/javax/transaction/xa/XAResource.java - src/java.sql/share/classes/javax/transaction/xa/Xid.java - src/java.sql/share/classes/javax/transaction/xa/package.html ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties - test/hotspot/gtest/logging/test_logTagLevelExpression.cpp - test/jdk/com/sun/jdi/JDIScaffold.java - test/jdk/java/lang/System/ExitFinalizersAndJIT.java - test/jdk/java/lang/System/finalization/FinExit.java - test/jdk/java/lang/System/finalization/FinExit.sh - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/Launcher.c - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/Makefile - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/linux-i586/libLauncher.so - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/solaris-amd64/libLauncher.so - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/solaris-sparcv9/libLauncher.so - test/jdk/javax/transaction/testng/Driver.java - test/jdk/javax/transaction/testng/test/transaction/XAExceptionTests.java - test/jdk/javax/transaction/testng/util/SerializedTransactionExceptions.java - test/jdk/sanity/releaseFile/NegativeSOURCETest.java - test/jdk/sun/java2d/OpenGL/CopyAreaOOB.java - test/jdk/sun/java2d/OpenGL/CustomCompositeTest.java - test/jdk/sun/java2d/OpenGL/DrawBufImgOp.java - test/jdk/sun/java2d/OpenGL/DrawHugeImageTest.java - test/jdk/sun/java2d/OpenGL/GradientPaints.java - test/jdk/sun/java2d/OpenGL/bug7181438.java From maurizio.cimadamore at oracle.com Thu Mar 15 21:10:32 2018 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 15 Mar 2018 21:10:32 +0000 Subject: hg: amber/amber: Automatic merge with default Message-ID: <201803152110.w2FLAWVS020196@aojmv0008.oracle.com> Changeset: 310517a2be2e Author: mcimadamore Date: 2018-03-15 22:06 +0100 URL: http://hg.openjdk.java.net/amber/amber/rev/310517a2be2e Automatic merge with default - make/data/x11wrappergen/functions.txt - make/data/x11wrappergen/sizes.32 - make/data/x11wrappergen/sizes.64 - make/data/x11wrappergen/sizes.64-solaris-i386 - make/devkit/createSolarisDevkit.sh - make/hotspot/src/native/dtrace/generateJvmOffsetsMain.c - make/lib/LibosxLibraries.gmk - make/lib/NetworkingLibraries.gmk - make/lib/NioLibraries.gmk - make/lib/PlatformLibraries.gmk - make/lib/SecurityLibraries.gmk - src/hotspot/share/gc/parallel/cardTableExtension.cpp - src/hotspot/share/gc/parallel/cardTableExtension.hpp - src/hotspot/share/gc/shared/cardTableModRefBSForCTRS.cpp - src/hotspot/share/logging/logTagLevelExpression.cpp - src/hotspot/share/logging/logTagLevelExpression.hpp - src/hotspot/share/memory/universe_ext.cpp - src/hotspot/share/services/allocationContextService.hpp - src/java.base/share/classes/javax/security/auth/Policy.java - src/java.base/share/classes/sun/security/provider/AuthPolicyFile.java - src/java.base/unix/classes/java/io/FileDescriptor.java - src/java.base/windows/classes/java/io/FileDescriptor.java - src/java.management/share/classes/sun/management/TypeVersionMapper.java - src/java.sql/share/classes/javax/transaction/xa/XAException.java - src/java.sql/share/classes/javax/transaction/xa/XAResource.java - src/java.sql/share/classes/javax/transaction/xa/Xid.java - src/java.sql/share/classes/javax/transaction/xa/package.html ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Resolve.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/main/JavaCompiler.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties - test/hotspot/gtest/logging/test_logTagLevelExpression.cpp - test/jdk/com/sun/jdi/JDIScaffold.java - test/jdk/java/lang/System/ExitFinalizersAndJIT.java - test/jdk/java/lang/System/finalization/FinExit.java - test/jdk/java/lang/System/finalization/FinExit.sh - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/Launcher.c - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/Makefile - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/linux-i586/libLauncher.so - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/solaris-amd64/libLauncher.so - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/solaris-sparcv9/libLauncher.so - test/jdk/javax/transaction/testng/Driver.java - test/jdk/javax/transaction/testng/test/transaction/XAExceptionTests.java - test/jdk/javax/transaction/testng/util/SerializedTransactionExceptions.java - test/jdk/sanity/releaseFile/NegativeSOURCETest.java - test/jdk/sun/java2d/OpenGL/CopyAreaOOB.java - test/jdk/sun/java2d/OpenGL/CustomCompositeTest.java - test/jdk/sun/java2d/OpenGL/DrawBufImgOp.java - test/jdk/sun/java2d/OpenGL/DrawHugeImageTest.java - test/jdk/sun/java2d/OpenGL/GradientPaints.java - test/jdk/sun/java2d/OpenGL/bug7181438.java From maurizio.cimadamore at oracle.com Thu Mar 15 21:10:52 2018 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 15 Mar 2018 21:10:52 +0000 Subject: hg: amber/amber: Automatic merge with default Message-ID: <201803152110.w2FLArQX020500@aojmv0008.oracle.com> Changeset: 6448c7f1dc1a Author: mcimadamore Date: 2018-03-15 22:06 +0100 URL: http://hg.openjdk.java.net/amber/amber/rev/6448c7f1dc1a Automatic merge with default - make/data/x11wrappergen/functions.txt - make/data/x11wrappergen/sizes.32 - make/data/x11wrappergen/sizes.64 - make/data/x11wrappergen/sizes.64-solaris-i386 - make/devkit/createSolarisDevkit.sh - make/hotspot/src/native/dtrace/generateJvmOffsetsMain.c - make/lib/LibosxLibraries.gmk - make/lib/NetworkingLibraries.gmk - make/lib/NioLibraries.gmk - make/lib/PlatformLibraries.gmk - make/lib/SecurityLibraries.gmk - src/hotspot/share/gc/parallel/cardTableExtension.cpp - src/hotspot/share/gc/parallel/cardTableExtension.hpp - src/hotspot/share/gc/shared/cardTableModRefBSForCTRS.cpp - src/hotspot/share/logging/logTagLevelExpression.cpp - src/hotspot/share/logging/logTagLevelExpression.hpp - src/hotspot/share/memory/universe_ext.cpp - src/hotspot/share/services/allocationContextService.hpp - src/java.base/share/classes/javax/security/auth/Policy.java - src/java.base/share/classes/sun/security/provider/AuthPolicyFile.java - src/java.base/unix/classes/java/io/FileDescriptor.java - src/java.base/windows/classes/java/io/FileDescriptor.java - src/java.management/share/classes/sun/management/TypeVersionMapper.java - src/java.sql/share/classes/javax/transaction/xa/XAException.java - src/java.sql/share/classes/javax/transaction/xa/XAResource.java - src/java.sql/share/classes/javax/transaction/xa/Xid.java - src/java.sql/share/classes/javax/transaction/xa/package.html ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Resolve.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java - test/hotspot/gtest/logging/test_logTagLevelExpression.cpp - test/jdk/com/sun/jdi/JDIScaffold.java - test/jdk/java/lang/System/ExitFinalizersAndJIT.java - test/jdk/java/lang/System/finalization/FinExit.java - test/jdk/java/lang/System/finalization/FinExit.sh - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/Launcher.c - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/Makefile - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/linux-i586/libLauncher.so - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/solaris-amd64/libLauncher.so - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/solaris-sparcv9/libLauncher.so - test/jdk/javax/transaction/testng/Driver.java - test/jdk/javax/transaction/testng/test/transaction/XAExceptionTests.java - test/jdk/javax/transaction/testng/util/SerializedTransactionExceptions.java - test/jdk/sanity/releaseFile/NegativeSOURCETest.java - test/jdk/sun/java2d/OpenGL/CopyAreaOOB.java - test/jdk/sun/java2d/OpenGL/CustomCompositeTest.java - test/jdk/sun/java2d/OpenGL/DrawBufImgOp.java - test/jdk/sun/java2d/OpenGL/DrawHugeImageTest.java - test/jdk/sun/java2d/OpenGL/GradientPaints.java - test/jdk/sun/java2d/OpenGL/bug7181438.java From maurizio.cimadamore at oracle.com Thu Mar 15 21:11:13 2018 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 15 Mar 2018 21:11:13 +0000 Subject: hg: amber/amber: Automatic merge with default Message-ID: <201803152111.w2FLBDYt020815@aojmv0008.oracle.com> Changeset: c9ec6bd25825 Author: mcimadamore Date: 2018-03-15 22:07 +0100 URL: http://hg.openjdk.java.net/amber/amber/rev/c9ec6bd25825 Automatic merge with default - make/data/x11wrappergen/functions.txt - make/data/x11wrappergen/sizes.32 - make/data/x11wrappergen/sizes.64 - make/data/x11wrappergen/sizes.64-solaris-i386 - make/devkit/createSolarisDevkit.sh - make/hotspot/src/native/dtrace/generateJvmOffsetsMain.c - make/lib/LibosxLibraries.gmk - make/lib/NetworkingLibraries.gmk - make/lib/NioLibraries.gmk - make/lib/PlatformLibraries.gmk - make/lib/SecurityLibraries.gmk - src/hotspot/share/gc/parallel/cardTableExtension.cpp - src/hotspot/share/gc/parallel/cardTableExtension.hpp - src/hotspot/share/gc/shared/cardTableModRefBSForCTRS.cpp - src/hotspot/share/logging/logTagLevelExpression.cpp - src/hotspot/share/logging/logTagLevelExpression.hpp - src/hotspot/share/memory/universe_ext.cpp - src/hotspot/share/services/allocationContextService.hpp - src/java.base/share/classes/javax/security/auth/Policy.java - src/java.base/share/classes/sun/security/provider/AuthPolicyFile.java - src/java.base/unix/classes/java/io/FileDescriptor.java - src/java.base/windows/classes/java/io/FileDescriptor.java - src/java.management/share/classes/sun/management/TypeVersionMapper.java - src/java.sql/share/classes/javax/transaction/xa/XAException.java - src/java.sql/share/classes/javax/transaction/xa/XAResource.java - src/java.sql/share/classes/javax/transaction/xa/Xid.java - src/java.sql/share/classes/javax/transaction/xa/package.html ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Resolve.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassReader.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/model/JavacElements.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties ! src/jdk.jshell/share/classes/jdk/jshell/ReplParser.java - test/hotspot/gtest/logging/test_logTagLevelExpression.cpp - test/jdk/com/sun/jdi/JDIScaffold.java - test/jdk/java/lang/System/ExitFinalizersAndJIT.java - test/jdk/java/lang/System/finalization/FinExit.java - test/jdk/java/lang/System/finalization/FinExit.sh - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/Launcher.c - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/Makefile - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/linux-i586/libLauncher.so - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/solaris-amd64/libLauncher.so - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/solaris-sparcv9/libLauncher.so - test/jdk/javax/transaction/testng/Driver.java - test/jdk/javax/transaction/testng/test/transaction/XAExceptionTests.java - test/jdk/javax/transaction/testng/util/SerializedTransactionExceptions.java - test/jdk/sanity/releaseFile/NegativeSOURCETest.java - test/jdk/sun/java2d/OpenGL/CopyAreaOOB.java - test/jdk/sun/java2d/OpenGL/CustomCompositeTest.java - test/jdk/sun/java2d/OpenGL/DrawBufImgOp.java - test/jdk/sun/java2d/OpenGL/DrawHugeImageTest.java - test/jdk/sun/java2d/OpenGL/GradientPaints.java - test/jdk/sun/java2d/OpenGL/bug7181438.java ! test/langtools/tools/javac/parser/extend/TrialParser.java From maurizio.cimadamore at oracle.com Thu Mar 15 21:11:42 2018 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 15 Mar 2018 21:11:42 +0000 Subject: hg: amber/amber: Automatic merge with default Message-ID: <201803152111.w2FLBgIP021220@aojmv0008.oracle.com> Changeset: 956d81c32894 Author: mcimadamore Date: 2018-03-15 22:07 +0100 URL: http://hg.openjdk.java.net/amber/amber/rev/956d81c32894 Automatic merge with default ! make/CompileJavaModules.gmk - make/data/x11wrappergen/functions.txt - make/data/x11wrappergen/sizes.32 - make/data/x11wrappergen/sizes.64 - make/data/x11wrappergen/sizes.64-solaris-i386 - make/devkit/createSolarisDevkit.sh - make/hotspot/src/native/dtrace/generateJvmOffsetsMain.c - make/lib/LibosxLibraries.gmk - make/lib/NetworkingLibraries.gmk - make/lib/NioLibraries.gmk - make/lib/PlatformLibraries.gmk - make/lib/SecurityLibraries.gmk - src/hotspot/share/gc/parallel/cardTableExtension.cpp - src/hotspot/share/gc/parallel/cardTableExtension.hpp - src/hotspot/share/gc/shared/cardTableModRefBSForCTRS.cpp - src/hotspot/share/logging/logTagLevelExpression.cpp - src/hotspot/share/logging/logTagLevelExpression.hpp - src/hotspot/share/memory/universe_ext.cpp - src/hotspot/share/services/allocationContextService.hpp - src/java.base/share/classes/javax/security/auth/Policy.java - src/java.base/share/classes/sun/security/provider/AuthPolicyFile.java - src/java.base/unix/classes/java/io/FileDescriptor.java - src/java.base/windows/classes/java/io/FileDescriptor.java - src/java.management/share/classes/sun/management/TypeVersionMapper.java - src/java.sql/share/classes/javax/transaction/xa/XAException.java - src/java.sql/share/classes/javax/transaction/xa/XAResource.java - src/java.sql/share/classes/javax/transaction/xa/Xid.java - src/java.sql/share/classes/javax/transaction/xa/package.html ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties - test/hotspot/gtest/logging/test_logTagLevelExpression.cpp - test/jdk/com/sun/jdi/JDIScaffold.java - test/jdk/java/lang/System/ExitFinalizersAndJIT.java - test/jdk/java/lang/System/finalization/FinExit.java - test/jdk/java/lang/System/finalization/FinExit.sh - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/Launcher.c - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/Makefile - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/linux-i586/libLauncher.so - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/solaris-amd64/libLauncher.so - test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/lib/solaris-sparcv9/libLauncher.so - test/jdk/javax/transaction/testng/Driver.java - test/jdk/javax/transaction/testng/test/transaction/XAExceptionTests.java - test/jdk/javax/transaction/testng/util/SerializedTransactionExceptions.java - test/jdk/sanity/releaseFile/NegativeSOURCETest.java - test/jdk/sun/java2d/OpenGL/CopyAreaOOB.java - test/jdk/sun/java2d/OpenGL/CustomCompositeTest.java - test/jdk/sun/java2d/OpenGL/DrawBufImgOp.java - test/jdk/sun/java2d/OpenGL/DrawHugeImageTest.java - test/jdk/sun/java2d/OpenGL/GradientPaints.java - test/jdk/sun/java2d/OpenGL/bug7181438.java From brian.goetz at oracle.com Thu Mar 15 23:11:39 2018 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Thu, 15 Mar 2018 23:11:39 +0000 Subject: hg: amber/amber: Improvements to DynamicCallSiteRef Message-ID: <201803152311.w2FNBdML019377@aojmv0008.oracle.com> Changeset: 398f83f17970 Author: briangoetz Date: 2018-03-15 19:06 -0400 URL: http://hg.openjdk.java.net/amber/amber/rev/398f83f17970 Improvements to DynamicCallSiteRef ! src/java.base/share/classes/java/lang/invoke/constant/DynamicCallSiteRef.java From cushon at google.com Fri Mar 16 00:36:03 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 16 Mar 2018 00:36:03 +0000 Subject: deduplicating lambda methods In-Reply-To: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> Message-ID: Thanks for the comments. I uploaded a new webrev, and responses are inline. http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.01/ On Thu, Mar 15, 2018 at 12:00 PM Brian Goetz wrote: > I think we wanted to exclude serializable lambdas from being deduplicated? > Thanks for the reminder, done. > Stats on its effectiveness on a representative corpus would be useful. > Agreed, I will collect more data and see if matches what we'd expect from the research Louis did. > Hopefully the hash code is discriminative enough that TreeDiffer doesn't > actually run that often. > I hope so too, and will try to quantify the compile-time performance overhead. On Thu, Mar 15, 2018 at 1:31 PM Vicente Romero wrote: > - it seems like: map dedupedLambda, could be a set. > When a duplicate is detected, we want the symbol of the first equivalent method that was seen. Currently the implementation is getting that from the return value of putIfAbsent. I'm not sure how to get that from the Set API? > - TreeDiffer, consider the benefits / trade-offs of extending > com.sun.tools.javac.tree.TreeScanner instead of implementing TreeVisitor. > The code will change a bit and you will probably need to declare a boolean > field for the result but you will have access to the subclasses of JCTree > without casting and will be able to use the getTag() method in JCTree, in > most cases being JCExpression the exception, to avoid most instanceof > checks. > Thanks, I'll investigate that. My initial impression is that using fields to replace the return value and the second parameter of the visit methods in TreeVisitor will complicate control flow somewhat. Have you considered adding a TreeVisitor-like API that for JCTrees to javac? > - symbols are unique so you can use == to test equality > - you don't need: Objects.equals(tree.isStatic(), that.isStatic()), to > compare two booleans > - names are also unique so you can use == instead of > Objects.equals(tree.getLabel(), that.getLabel()) > Thanks, fixed. - in general there are complete implementations for trees that can't appear > inside the method's body, but I think that they were provided for > completeness. > Right, my sense was that this might be useful for things besides lambda de-duplication, so it was worth handling nodes that won't appear in method bodies. If you think that should be deferred until it's actually needed, I can remove it. From maurizio.cimadamore at oracle.com Fri Mar 16 18:46:28 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 16 Mar 2018 18:46:28 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> Message-ID: On 16/03/18 00:36, Liam Miller-Cushon wrote: > Thanks, I'll investigate that. My initial impression is that using fields > to replace the return value and the second parameter of the visit methods > in TreeVisitor will complicate control flow somewhat. > > Have you considered adding a TreeVisitor-like API that for JCTrees to javac? Having a tree scanner accepting a visitor parameter would be good - although can be done outside the scope of this patch. Note that, in the meantime, it's rather easy to mimic the visitor parameter idiom - just define a 'scan' method like this: ``` Object parameter; //visitor field scan(JCTree tree, Object parameter) { ?? Object prev = this.parameter; ?? try { ?????? this.parameter = parameter; ??? ?? super.scan(tree); ?? } finally { ?????? this.parameter = prev; ?? } } ``` And then your code should not be altered much (I hope! :-)). Maurizio From vicente.romero at oracle.com Fri Mar 16 19:17:46 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 16 Mar 2018 15:17:46 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> Message-ID: Hi Liam, Thanks for the updates to the patch. I also agree with Maurizio about using TreeScanner. Additional comment: TreeHasher should skip parenthesis of expressions, to obtain the same hash for, for example: return (i + j); and return i + j; Also, as a bonus, consider constant expressions, we probably want: return 5; and return 2 + 3; to be equal. I think that these two use cases will need a "normalization" step to be applied to the tree and store in the map, set, only a normalized version of the tree. I understand that at least considering constants could be out of the scope of this patch, so I can see it as a follow up development. Thanks, Vicente On 03/16/2018 02:46 PM, Maurizio Cimadamore wrote: > > > On 16/03/18 00:36, Liam Miller-Cushon wrote: >> Thanks, I'll investigate that. My initial impression is that using >> fields >> to replace the return value and the second parameter of the visit >> methods >> in TreeVisitor will complicate control flow somewhat. >> >> Have you considered adding a TreeVisitor-like API that for JCTrees to >> javac? > Having a tree scanner accepting a visitor parameter would be good - > although can be done outside the scope of this patch. > > Note that, in the meantime, it's rather easy to mimic the visitor > parameter idiom - just define a 'scan' method like this: > > ``` > Object parameter; //visitor field > > scan(JCTree tree, Object parameter) { > ?? Object prev = this.parameter; > ?? try { > ?????? this.parameter = parameter; > ??? ?? super.scan(tree); > ?? } finally { > ?????? this.parameter = prev; > ?? } > } > ``` > > And then your code should not be altered much (I hope! :-)). > > Maurizio > From vicente.romero at oracle.com Fri Mar 16 19:21:13 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 16 Mar 2018 15:21:13 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> Message-ID: hum, that one lost the format, reformatting: Hi Liam, Thanks for the updates to the patch. I also agree with Maurizio about using TreeScanner. Additional comment: TreeHasher should skip parenthesis of expressions, to obtain the same hash for, for example: return (i + j); and return i + j; Also, as a bonus, consider constant expressions, we probably want: return 5; and return 2 + 3; to be equal. I think that these two use cases will need a "normalization" step to be applied to the tree and store in the map, set, only a normalized version of the tree. I understand that at least considering constants could be out of the scope of this patch, so I can see it as a follow up development. Thanks, Vicente On 03/16/2018 03:17 PM, Vicente Romero wrote: > Hi Liam, Thanks for the updates to the patch. I also agree with > Maurizio about using TreeScanner. Additional comment: TreeHasher > should skip parenthesis of expressions, to obtain the same hash for, > for example: return (i + j); and return i + j; Also, as a bonus, > consider constant expressions, we probably want: return 5; and return > 2 + 3; to be equal. I think that these two use cases will need a > "normalization" step to be applied to the tree and store in the map, > set, only a normalized version of the tree. I understand that at least > considering constants could be out of the scope of this patch, so I > can see it as a follow up development. Thanks, Vicente > > > > On 03/16/2018 02:46 PM, Maurizio Cimadamore wrote: >> >> >> On 16/03/18 00:36, Liam Miller-Cushon wrote: >>> Thanks, I'll investigate that. My initial impression is that using >>> fields >>> to replace the return value and the second parameter of the visit >>> methods >>> in TreeVisitor will complicate control flow somewhat. >>> >>> Have you considered adding a TreeVisitor-like API that for JCTrees >>> to javac? >> Having a tree scanner accepting a visitor parameter would be good - >> although can be done outside the scope of this patch. >> >> Note that, in the meantime, it's rather easy to mimic the visitor >> parameter idiom - just define a 'scan' method like this: >> >> ``` >> Object parameter; //visitor field >> >> scan(JCTree tree, Object parameter) { >> ?? Object prev = this.parameter; >> ?? try { >> ?????? this.parameter = parameter; >> ??? ?? super.scan(tree); >> ?? } finally { >> ?????? this.parameter = prev; >> ?? } >> } >> ``` >> >> And then your code should not be altered much (I hope! :-)). >> >> Maurizio >> > From vicente.romero at oracle.com Fri Mar 16 20:08:27 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 16 Mar 2018 16:08:27 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> Message-ID: <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> Hi again, I could be wrong but it seems to me that TreeHasher generates too many false positives as the hash number obtained won't have any topological information about the tree. In other words, it seems to me that the hash generated will be the same regardless of the way the tree is traversed. I think that we are interested in obtaining a hash that is a topological descriptor of the tree. This will probably imply making TreeHasher a full fledged visitor. Thanks, Vicente On 03/16/2018 03:21 PM, Vicente Romero wrote: > hum, that one lost the format, reformatting: > > Hi Liam, > > Thanks for the updates to the patch. I also agree with Maurizio about > using TreeScanner. Additional comment: TreeHasher should skip > parenthesis of expressions, to obtain the same hash for, for example: > return (i + j); and return i + j; > > Also, as a bonus, consider constant expressions, we probably want: > return 5; and return 2 + 3; to be equal. > > I think that these two use cases will need a "normalization" step to > be applied to the tree and store in the map, set, only a normalized > version of the tree. I understand that at least considering constants > could be out of the scope of this patch, so I can see it as a follow > up development. > > Thanks, > Vicente > > On 03/16/2018 03:17 PM, Vicente Romero wrote: >> Hi Liam, Thanks for the updates to the patch. I also agree with >> Maurizio about using TreeScanner. Additional comment: TreeHasher >> should skip parenthesis of expressions, to obtain the same hash for, >> for example: return (i + j); and return i + j; Also, as a bonus, >> consider constant expressions, we probably want: return 5; and return >> 2 + 3; to be equal. I think that these two use cases will need a >> "normalization" step to be applied to the tree and store in the map, >> set, only a normalized version of the tree. I understand that at >> least considering constants could be out of the scope of this patch, >> so I can see it as a follow up development. Thanks, Vicente >> >> >> >> On 03/16/2018 02:46 PM, Maurizio Cimadamore wrote: >>> >>> >>> On 16/03/18 00:36, Liam Miller-Cushon wrote: >>>> Thanks, I'll investigate that. My initial impression is that using >>>> fields >>>> to replace the return value and the second parameter of the visit >>>> methods >>>> in TreeVisitor will complicate control flow somewhat. >>>> >>>> Have you considered adding a TreeVisitor-like API that for JCTrees >>>> to javac? >>> Having a tree scanner accepting a visitor parameter would be good - >>> although can be done outside the scope of this patch. >>> >>> Note that, in the meantime, it's rather easy to mimic the visitor >>> parameter idiom - just define a 'scan' method like this: >>> >>> ``` >>> Object parameter; //visitor field >>> >>> scan(JCTree tree, Object parameter) { >>> ?? Object prev = this.parameter; >>> ?? try { >>> ?????? this.parameter = parameter; >>> ??? ?? super.scan(tree); >>> ?? } finally { >>> ?????? this.parameter = prev; >>> ?? } >>> } >>> ``` >>> >>> And then your code should not be altered much (I hope! :-)). >>> >>> Maurizio >>> >> > From cushon at google.com Sat Mar 17 01:48:32 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Sat, 17 Mar 2018 01:48:32 +0000 Subject: deduplicating lambda methods In-Reply-To: <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> Message-ID: Hi, An updated version is at: http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.02/ On Fri, Mar 16, 2018 at 11:46 AM Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > Having a tree scanner accepting a visitor parameter would be good - > although can be done outside the scope of this patch. > Sounds good, I'll add it to my list to investigate later. Note that, in the meantime, it's rather easy to mimic the visitor > parameter idiom - just define a 'scan' method like this: > Thanks! I switched to com.sun.tools.javac.comp.TreeScanner using your approach. On Fri, Mar 16, 2018 at 12:20 PM Vicente Romero wrote: > TreeHasher should skip > parenthesis of expressions, to obtain the same hash for, for example: > return (i + j); and return i + j; > Done, thanks. > Also, as a bonus, consider constant expressions, we probably want: > return 5; and return 2 + 3; to be equal. > > I think that these two use cases will need a "normalization" step to be > applied to the tree and store in the map, set, only a normalized version > of the tree. I understand that at least considering constants could be > out of the scope of this patch, so I can see it as a follow up development. > Are you envisioning this as something that would happen as part of tree comparision, or something that would fall out of other work around constant folding? If support for comprehensive compile-time constant folding happens, and it's performed prior to lambda deduplication, then your examples would be supported for free without the need for a more sophisticated diff. On Fri, Mar 16, 2018 at 1:07 PM Vicente Romero wrote: > I could be wrong but it seems to me that TreeHasher generates too many > false positives as the hash number obtained won't have any topological > information about the tree. In other words, it seems to me that the hash > generated will be the same regardless of the way the tree is traversed. > I think that we are interested in obtaining a hash that is a topological > descriptor of the tree. This will probably imply making TreeHasher a > full fledged visitor. > The hash function needs more work. I am still planning to collect data on how often collisions are happening as part of quantifying the compile-time performance overhead. However, the collisions I am seeing right now are mostly from identifiers and literals, and not from differences in the topology. I'm having trouble thinking of uncontrived examples where the topology of two ASTs is different, but the result of traversing all nodes and recording their tags is the same. Do you think that's going to be common, and did you have examples in mind? From bsrbnd at gmail.com Sat Mar 17 15:02:43 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sat, 17 Mar 2018 16:02:43 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <4b7cb307-1af7-ccfe-02a0-3358dd75959c@oracle.com> Message-ID: On 15 March 2018 at 19:52, Liam Miller-Cushon wrote: > On Wed, Mar 7, 2018 at 11:38 AM John Rose wrote: >> >> > Note that the only risk is to point to another lambda in the source >> > file but with the same code? >> >> ?Or if we take the option to globalize some "top ten" list of shapes. >> Then you'd step into a lambda body in some java.base library. >> (Or maybe not step at all?) >> >> If your single-stepping takes you to a lambda with the same shape >> as the one you expect to single-step into, but the source location >> you actually reach looks unreached, it's confusing but arguably OK. >> >> After all lambda identity (as an object) is not guaranteed, and can >> be pre-allocated. To the debugging programmer, it looks like the >> compiler and/or JVM chose to preallocate the same lambda but >> in a different place. Exact source locations are debatable in a way >> expression semantics are not. Again, full debuggability sometimes >> conflicts with full performance. > > > I agree that stepping in to a lambda body and jumping to a different line > with identical code might not be *too* surprising. > > Perhaps more concerning is the handling of breakpoints: if a breakpoint > is set in a lambda body that gets deduplicated (or is implemented > using one of the globalized lambdas), the breakpoint will never be hit. > This seems more confusing and harder to work around than stepping to > an incorrect but equivalent line. In this case, I think the debugger (or the debug API?) would have to map all corresponding breakpoints to the reference (merged or shape) lambda maybe producing some unnecessary breaks when debugging but without misses. Bernard > I don't have any good ideas here aside from something like the follow-on > project you described of adding additional information to the capture site > that could be used by the debugger. That sounds feasible but non-trivial, > and I wonder whether lambda deduplication alone provides enough > motivation to add support for that. From vicente.romero at oracle.com Sat Mar 17 18:44:53 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Sat, 17 Mar 2018 14:44:53 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> Message-ID: <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> Hi Liam, On 03/16/2018 09:48 PM, Liam Miller-Cushon wrote: > Hi, > > An updated version is at: > http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.02/ > overall looks good, but I think that a set of regression tests should be included along with the patch. > > On Fri, Mar 16, 2018 at 11:46 AM Maurizio Cimadamore > > wrote: > > Having a tree scanner accepting a visitor parameter would be good - > although can be done outside the scope of this patch. > > > Sounds good, I'll add it to my list to investigate later. > > Note that, in the meantime, it's rather easy to mimic the visitor > parameter idiom - just define a 'scan' method like this: > > > Thanks! I switched to com.sun.tools.javac.comp.TreeScanner using your > approach. > > On Fri, Mar 16, 2018 at 12:20 PM Vicente Romero > > wrote: > > TreeHasher should skip > parenthesis of expressions, to obtain the same hash for, for example: > return (i + j); and return i + j; > > > Done, thanks. > > Also, as a bonus, consider constant expressions, we probably want: > return 5; and return 2 + 3; to be equal. > > I think that these two use cases will need a "normalization" step > to be > applied to the tree and store in the map, set, only a normalized > version > of the tree. I understand that at least considering constants could be > out of the scope of this patch, so I can see it as a follow up > development. > > > Are you envisioning this as something that would happen as part of tree > comparision, yes because the current effort on constant folding won't be rewriting the trees, at least not all of them only a small fraction of them. Although as I said there is no need to do this now I don't think that the constant folding project will make this task easier in the future. Unless we decide to rewrite all the trees holding a constant value. > or something that would fall out of other work around constant > folding? If support for comprehensive compile-time constant folding > happens, > and it's performed prior to lambda deduplication, then your examples would > be supported for free without the need for a more sophisticated diff. > On Fri, Mar 16, 2018 at 1:07 PM Vicente Romero > > wrote: > > I could be wrong but it seems to me that TreeHasher generates too many > false positives as the hash number obtained won't have any topological > information about the tree. In other words, it seems to me that > the hash > generated will be the same regardless of the way the tree is > traversed. > I think that we are interested in obtaining a hash that is a > topological > descriptor of the tree. This will probably imply making TreeHasher a > full fledged visitor. > > > The hash function needs more work. I am still planning to collect data > on how often collisions are happening as part of quantifying the > compile-time performance overhead. > > However, the collisions I am seeing right now are mostly from identifiers > and literals, and not from differences in the topology. I'm having trouble > thinking of uncontrived examples where the topology of two ASTs is > different, but the result of traversing all nodes and recording their tags > is the same. Do you think that's going to be common, and did you have > examples in mind? at first glance it seemed to me that the hash function would return the same value for: (int i) -> i / 5; and (int i) -> 5 / i; I haven't test it but this was my impression. Vicente From brian.goetz at oracle.com Sat Mar 17 18:49:01 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 17 Mar 2018 14:49:01 -0400 Subject: deduplicating lambda methods In-Reply-To: <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> Message-ID: <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> >> >> ??? Also, as a bonus, consider constant expressions, we probably want: >> ??? return 5; and return 2 + 3; to be equal. >> >> ??? I think that these two use cases will need a "normalization" step >> ??? to be >> ??? applied to the tree and store in the map, set, only a normalized >> ??? version >> ??? of the tree. I understand that at least considering constants >> could be >> ??? out of the scope of this patch, so I can see it as a follow up >> ??? development. I don't think we want to be duplicating the constant folding logic in multiple places, but I also don't think we have to -- doesn't L2M run after constant folding already?? I ask because I've already seen constant propagation turn capturing lambdas into noncapturing. From vicente.romero at oracle.com Sat Mar 17 18:57:26 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Sat, 17 Mar 2018 14:57:26 -0400 Subject: deduplicating lambda methods In-Reply-To: <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> Message-ID: <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> On 03/17/2018 02:49 PM, Brian Goetz wrote: > >>> >>> ??? Also, as a bonus, consider constant expressions, we probably want: >>> ??? return 5; and return 2 + 3; to be equal. >>> >>> ??? I think that these two use cases will need a "normalization" step >>> ??? to be >>> ??? applied to the tree and store in the map, set, only a normalized >>> ??? version >>> ??? of the tree. I understand that at least considering constants >>> could be >>> ??? out of the scope of this patch, so I can see it as a follow up >>> ??? development. > > I don't think we want to be duplicating the constant folding logic in > multiple places, but I also don't think we have to -- doesn't L2M run > after constant folding already?? I ask because I've already seen > constant propagation turn capturing lambdas into noncapturing. I don't think we need to duplicate the constant folding logic for this, I just think that the hasher and the differ should be blind regarding the substructure of a tree as soon as a constant is found, be it in a variable or as the constant value of a tree. Liam has a good point, if our new constant folding phase were rewriting the trees then the differ and the hasher he is proposing would have this for free. I guess that this is a use case that can make us consider doing tree rewriting as part of the new constant folding phase, but we are not doing it right now, only for some trees: basically ldc() indy() and condys. That's why I was saying that considering constants in the tree differ / hasher could wait and that it don't have to be part of this patch. From brian.goetz at oracle.com Sat Mar 17 19:00:36 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 17 Mar 2018 15:00:36 -0400 Subject: deduplicating lambda methods In-Reply-To: <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> Message-ID: <23d97d15-af9c-a38c-5982-37060beaaa85@oracle.com> >> I don't think we want to be duplicating the constant folding logic in >> multiple places, but I also don't think we have to -- doesn't L2M run >> after constant folding already?? I ask because I've already seen >> constant propagation turn capturing lambdas into noncapturing. > > I don't think we need to duplicate the constant folding logic for > this, I just think that the hasher and the differ should be blind > regarding the substructure of a tree as soon as a constant is found, > be it in a variable or as the constant value of a tree. Liam has a > good point, if our new constant folding phase were rewriting the trees > then the differ and the hasher he is proposing would have this for > free. I guess that this is a use case that can make us consider doing > tree rewriting as part of the new constant folding phase, but we are > not doing it right now, only for some trees: basically ldc() indy() > and condys. That's why I was saying that considering constants in the > tree differ / hasher could wait and that it don't have to be part of > this patch. > When you say blind, obviously it has to take the constants into account, since x -> x+2 and x->x+3 are different lambdas.? I think for now, not trying to do anything special here is the right move; this could conceivably fail to deduplicate x -> x + 2? and x -> x + TWO? when TWO is a static final, but that's an acceptable loss, since deduplication is merely optimistic. From vicente.romero at oracle.com Sat Mar 17 19:10:37 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Sat, 17 Mar 2018 15:10:37 -0400 Subject: deduplicating lambda methods In-Reply-To: <23d97d15-af9c-a38c-5982-37060beaaa85@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <23d97d15-af9c-a38c-5982-37060beaaa85@oracle.com> Message-ID: On 03/17/2018 03:00 PM, Brian Goetz wrote: > > >>> I don't think we want to be duplicating the constant folding logic >>> in multiple places, but I also don't think we have to -- doesn't L2M >>> run after constant folding already?? I ask because I've already seen >>> constant propagation turn capturing lambdas into noncapturing. >> >> I don't think we need to duplicate the constant folding logic for >> this, I just think that the hasher and the differ should be blind >> regarding the substructure of a tree as soon as a constant is found, >> be it in a variable or as the constant value of a tree. Liam has a >> good point, if our new constant folding phase were rewriting the >> trees then the differ and the hasher he is proposing would have this >> for free. I guess that this is a use case that can make us consider >> doing tree rewriting as part of the new constant folding phase, but >> we are not doing it right now, only for some trees: basically ldc() >> indy() and condys. That's why I was saying that considering constants >> in the tree differ / hasher could wait and that it don't have to be >> part of this patch. >> > > When you say blind, obviously it has to take the constants into account, right but it doesn't have to redo what constant folding is doing, the constant will be already there sitting in the tree, > since x -> x+2 and x->x+3 are different lambdas.? I think for now, not > trying to do anything special here is the right move; I totally agree that's why I started saying that this could be a follow-up development > this could conceivably fail to deduplicate x -> x + 2? and x -> x + > TWO? when TWO is a static final, but that's an acceptable loss, since > deduplication is merely optimistic. right it would fail to do that, but it could probably handle it by adding a couple of LOC more compared to the current patch, but again I'm not proposing to do it now > > > From vicente.romero at oracle.com Sat Mar 17 19:42:50 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Sat, 17 Mar 2018 15:42:50 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <23d97d15-af9c-a38c-5982-37060beaaa85@oracle.com> Message-ID: <8f779cc9-4246-5461-ffd2-bb6dd794c3b0@oracle.com> Hi, Attached there is a patch, as in hack-ish, quick, use with care, to be applied on top of last Liam's patch that considers constant values in the trees. This patch allows to deduplicate lambdas like: Function l1 = i -> 2 + 3;??? and: Function l2 = i -> 5; Thanks, Vicente On 03/17/2018 03:10 PM, Vicente Romero wrote: > > > On 03/17/2018 03:00 PM, Brian Goetz wrote: >> >> >>>> I don't think we want to be duplicating the constant folding logic >>>> in multiple places, but I also don't think we have to -- doesn't >>>> L2M run after constant folding already?? I ask because I've already >>>> seen constant propagation turn capturing lambdas into noncapturing. >>> >>> I don't think we need to duplicate the constant folding logic for >>> this, I just think that the hasher and the differ should be blind >>> regarding the substructure of a tree as soon as a constant is found, >>> be it in a variable or as the constant value of a tree. Liam has a >>> good point, if our new constant folding phase were rewriting the >>> trees then the differ and the hasher he is proposing would have this >>> for free. I guess that this is a use case that can make us consider >>> doing tree rewriting as part of the new constant folding phase, but >>> we are not doing it right now, only for some trees: basically ldc() >>> indy() and condys. That's why I was saying that considering >>> constants in the tree differ / hasher could wait and that it don't >>> have to be part of this patch. >>> >> >> When you say blind, obviously it has to take the constants into account, > > right but it doesn't have to redo what constant folding is doing, the > constant will be already there sitting in the tree, > >> since x -> x+2 and x->x+3 are different lambdas.? I think for now, >> not trying to do anything special here is the right move; > > I totally agree that's why I started saying that this could be a > follow-up development > >> this could conceivably fail to deduplicate x -> x + 2? and x -> x + >> TWO? when TWO is a static final, but that's an acceptable loss, since >> deduplication is merely optimistic. > > right it would fail to do that, but it could probably handle it by > adding a couple of LOC more compared to the current patch, but again > I'm not proposing to do it now > >> >> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: considering.constants.patch Type: text/x-patch Size: 1878 bytes Desc: not available URL: From cushon at google.com Sun Mar 18 02:04:04 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Sun, 18 Mar 2018 02:04:04 +0000 Subject: deduplicating lambda methods In-Reply-To: <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> Message-ID: Hi Vicente, On Sat, Mar 17, 2018 at 11:44 AM Vicente Romero wrote: > overall looks good, but I think that a set of regression tests should be > included along with the patch. > Definitely, I have some functional tests that I'll work on turning into a jtreg test. Do you have advice about how much test coverage is appropriate for tree comparison? Getting to 100% branch coverage would require a lot of test cases. I could probably generate some of them programatically. FWIW a certain amount of TreeDiffer was generated and not hand-written, so any bugs are probably structural problems rather than typos. (That doesn't lessen the need for tests, of course--there's still the risk of adding typos when changes are made in the future.) > at first glance it seemed to me that the hash function would return the > same value for: > (int i) -> i / 5; and (int i) -> 5 / i; I haven't test it but this was my > impression. > The hash function currently traverses all nodes and hashes their tags, so for those two examples it would see something like `[DIV, IDENT, LITERAL]` and `[DIV, LITERAL, IDENT]`, which would not result in the same value. This does mean that `x + 42` and `y + 43` both hashed to the same value. I updated the hash function to include literal values and symbols (with the same handling of method parameters as the diff). With that change I am no longer seeing hash collisions that result in unnecessary diffing. There are still cases where two lambda bodies hash to the same value but have different target types, but that case doesn't require a tree diff (the check that the target types are the same happens first). The updates to hashing are here: http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.03/ From cushon at google.com Sun Mar 18 02:04:13 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Sun, 18 Mar 2018 02:04:13 +0000 Subject: deduplicating lambda methods In-Reply-To: <8f779cc9-4246-5461-ffd2-bb6dd794c3b0@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <23d97d15-af9c-a38c-5982-37060beaaa85@oracle.com> <8f779cc9-4246-5461-ffd2-bb6dd794c3b0@oracle.com> Message-ID: On Sat, Mar 17, 2018 at 12:42 PM Vicente Romero wrote: > Attached there is a patch, as in hack-ish, quick, use with care, to be > applied on top of last Liam's patch that considers constant values in > the trees. This patch allows to deduplicate lambdas like: > > Function l1 = i -> 2 + 3; and: > Function l2 = i -> 5; > Nice! I understand now, that's less complicated than what I had been imagining. You mentioned a 'normalization' step earlier. Is that something that would still be necessary with this approach? From bsrbnd at gmail.com Sun Mar 18 15:17:17 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 18 Mar 2018 16:17:17 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> Message-ID: Hi Liam, On 18 March 2018 at 03:04, Liam Miller-Cushon wrote: [...] > The updates to hashing are here: > http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.03/ Still looking at the stats [1], we see that the highest duplication frequency of more than 10e3 appears on *cross-file* lambdas but falls to 10e2 on *intra-file* lambdas and your patch suggest an *intra-class* de-duplication with an even lower frequency (note also that the JIT profile issue [2] is still unaddressed)... So, I'm wondering if a set of well defined JIT compliant global shapes (somewhere in 'java.base') would be more efficient in term of class size reduction and performance? Bernard [1] https://drive.google.com/file/d/1abAR_bueU0Zxy4e9XfLVzVe2nwy_POQm/view?usp=sharing [2] http://mail.openjdk.java.net/pipermail/amber-dev/2018-March/002745.html From brian.goetz at oracle.com Sun Mar 18 15:43:59 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 18 Mar 2018 11:43:59 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> Message-ID: <54c0edc4-61e4-7386-ea0a-d7633b836a0d@oracle.com> Clearly that's outside the scope of the current project, so this is all speculation. One of the tradeoffs in attempting to deduplicate across classes is that there's a coordination cost.? If class A captures a lambda, and class B captures the same lambda, there's some benefit to them doing so independently; doing otherwise requires a cache, coordination, lookups, locks, etc.? You'd have to show that the hit rate is good enough to make up for this. One place where it likely is -- and we don't currently do this -- is in the string concatenation bootstrap.? We currently spin a class for each string concatenation site, even though cases like "string and int" are surely duplicated many times.? Caching common shapes here is probably a much bigger win than caching duplicated lambdas; there is likely to be far more duplication. For the few lambdas and method refs that are out at the "used a million times in the same app" end of the spectrum, the winning move there is probably some sort of jlink plugin that identifies the most common ones, factors them into static final fields of a synthetic class, and replaces captures with field references. Doing this in the JDK, though, is probably the wrong move, as any list of "famous lambdas" is likely to be wrong for 95% of applications, so this is something you want to generate by analysis over a specific application. On 3/18/2018 11:17 AM, B. Blaser wrote: > Hi Liam, > > On 18 March 2018 at 03:04, Liam Miller-Cushon wrote: > > [...] > >> The updates to hashing are here: >> http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.03/ > Still looking at the stats [1], we see that the highest duplication > frequency of more than 10e3 appears on *cross-file* lambdas but falls > to 10e2 on *intra-file* lambdas and your patch suggest an > *intra-class* de-duplication with an even lower frequency (note also > that the JIT profile issue [2] is still unaddressed)... > > So, I'm wondering if a set of well defined JIT compliant global shapes > (somewhere in 'java.base') would be more efficient in term of class > size reduction and performance? > > Bernard > > [1] https://drive.google.com/file/d/1abAR_bueU0Zxy4e9XfLVzVe2nwy_POQm/view?usp=sharing > [2] http://mail.openjdk.java.net/pipermail/amber-dev/2018-March/002745.html From bsrbnd at gmail.com Sun Mar 18 21:02:29 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 18 Mar 2018 22:02:29 +0100 Subject: deduplicating lambda methods In-Reply-To: <54c0edc4-61e4-7386-ea0a-d7633b836a0d@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <54c0edc4-61e4-7386-ea0a-d7633b836a0d@oracle.com> Message-ID: Well, I'm not disputing your analysis, but let's take the most frequent *cross-file* (and probably *cross-app*) example [3]: Runnable r = () -> {}; I think this should be possible to define something like: diff --git a/src/java.base/share/classes/java/lang/Runnable.java b/src/java.base/share/classes/java/lang/Runnable.java --- a/src/java.base/share/classes/java/lang/Runnable.java +++ b/src/java.base/share/classes/java/lang/Runnable.java @@ -66,4 +66,5 @@ * @see java.lang.Thread#run() */ public abstract void run(); + public Runnable NOP = () -> {}; } and use it within the compiler with some 'getstatic' instead of multiple 'invokedynamic' (which also might be used directly by programmers)? Bernard [3] http://mail.openjdk.java.net/pipermail/amber-dev/2018-March/002731.html On 18 March 2018 at 16:43, Brian Goetz wrote: > Clearly that's outside the scope of the current project, so this is all > speculation. > > One of the tradeoffs in attempting to deduplicate across classes is that > there's a coordination cost. If class A captures a lambda, and class B > captures the same lambda, there's some benefit to them doing so > independently; doing otherwise requires a cache, coordination, lookups, > locks, etc. You'd have to show that the hit rate is good enough to make up > for this. > > One place where it likely is -- and we don't currently do this -- is in the > string concatenation bootstrap. We currently spin a class for each string > concatenation site, even though cases like "string and int" are surely > duplicated many times. Caching common shapes here is probably a much bigger > win than caching duplicated lambdas; there is likely to be far more > duplication. > > For the few lambdas and method refs that are out at the "used a million > times in the same app" end of the spectrum, the winning move there is > probably some sort of jlink plugin that identifies the most common ones, > factors them into static final fields of a synthetic class, and replaces > captures with field references. Doing this in the JDK, though, is probably > the wrong move, as any list of "famous lambdas" is likely to be wrong for > 95% of applications, so this is something you want to generate by analysis > over a specific application. > > > > > On 3/18/2018 11:17 AM, B. Blaser wrote: >> >> Hi Liam, >> >> On 18 March 2018 at 03:04, Liam Miller-Cushon wrote: >> >> [...] >> >>> The updates to hashing are here: >>> http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.03/ >> >> Still looking at the stats [1], we see that the highest duplication >> frequency of more than 10e3 appears on *cross-file* lambdas but falls >> to 10e2 on *intra-file* lambdas and your patch suggest an >> *intra-class* de-duplication with an even lower frequency (note also >> that the JIT profile issue [2] is still unaddressed)... >> >> So, I'm wondering if a set of well defined JIT compliant global shapes >> (somewhere in 'java.base') would be more efficient in term of class >> size reduction and performance? >> >> Bernard >> >> [1] >> https://drive.google.com/file/d/1abAR_bueU0Zxy4e9XfLVzVe2nwy_POQm/view?usp=sharing >> [2] >> http://mail.openjdk.java.net/pipermail/amber-dev/2018-March/002745.html > > From vicente.romero at oracle.com Mon Mar 19 12:55:11 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Mon, 19 Mar 2018 08:55:11 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <23d97d15-af9c-a38c-5982-37060beaaa85@oracle.com> <8f779cc9-4246-5461-ffd2-bb6dd794c3b0@oracle.com> Message-ID: <8c5b6a61-7d7f-b227-41db-d150fb281af3@oracle.com> On 03/17/2018 10:04 PM, Liam Miller-Cushon wrote: > On Sat, Mar 17, 2018 at 12:42 PM Vicente Romero > > wrote: > > Attached there is a patch, as in hack-ish, quick, use with care, to be > applied on top of last Liam's patch that considers constant values in > the trees. This patch allows to deduplicate lambdas like: > > Function l1 = i -> 2 + 3;??? and: > Function l2 = i -> 5; > > > Nice! I understand now, that's less complicated than what I had been > imagining. :) > > You mentioned a 'normalization' step earlier. Is that something that would > still be necessary with this approach? no I don't think it will be necessary anymore Vicente From vicente.romero at oracle.com Mon Mar 19 13:10:02 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Mon, 19 Mar 2018 09:10:02 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> Message-ID: On 03/17/2018 10:04 PM, Liam Miller-Cushon wrote: > Hi Vicente, > > On Sat, Mar 17, 2018 at 11:44 AM Vicente Romero > > wrote: > > overall looks good, but I think that a set of regression tests > should be included along with the patch. > > > Definitely, I have some functional tests that I'll work on turning > into a jtreg test. > > Do you have advice about how much test coverage is appropriate for > tree comparison? > Getting to 100% branch coverage would require a lot of test cases. I > could probably > generate some of them programatically. FWIW a certain amount of > TreeDiffer was > generated and not hand-written, so any bugs are probably structural > problems rather > than typos. (That doesn't lessen the need for tests, of > course--there's still the risk of > adding typos when changes are made in the future.) well I assume that covering most common cases with a combo test should be the general guideline here. I don't think we need 100% coverage to validate the approach. I would also add some corner cases, probably in separate tests, those corner cases should include lambdas returning lambdas, lambdas with inner classes in their bodies, etc. > at first glance it seemed to me that the hash function would > return the same value for: > (int i) -> i / 5; and (int i) -> 5 / i; I haven't test it but this > was my impression. > > > The hash function currently traverses all nodes and hashes their tags, > so for those two > examples it would see something like `[DIV, IDENT, LITERAL]` and > `[DIV, LITERAL, IDENT]`, > which would not result in the same value. yep I got that but I didn't test it manually, thanks for checking it > > This does mean that `x?+ 42` and `y?+ 43` both hashed to the same > value. I updated the > hash function to include literal values and symbols (with the same > handling of method > parameters as the diff). With that change I am no longer seeing hash > collisions that result in > unnecessary diffing. There are still cases where two lambda bodies > hash to the same > value but have different target types, but that case doesn't require a > tree diff (the check > that the target types are the same happens first). > > The updates to hashing are here: > http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.03/ > From bsrbnd at gmail.com Mon Mar 19 14:41:40 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 19 Mar 2018 15:41:40 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> Message-ID: On 19 March 2018 at 14:10, Vicente Romero wrote: > > > On 03/17/2018 10:04 PM, Liam Miller-Cushon wrote: >> >> Hi Vicente, >> >> On Sat, Mar 17, 2018 at 11:44 AM Vicente Romero > > wrote: >> >> overall looks good, but I think that a set of regression tests >> should be included along with the patch. >> >> >> Definitely, I have some functional tests that I'll work on turning into a >> jtreg test. >> >> Do you have advice about how much test coverage is appropriate for tree >> comparison? >> Getting to 100% branch coverage would require a lot of test cases. I could >> probably >> generate some of them programatically. FWIW a certain amount of TreeDiffer >> was >> generated and not hand-written, so any bugs are probably structural >> problems rather >> than typos. (That doesn't lessen the need for tests, of course--there's >> still the risk of >> adding typos when changes are made in the future.) > > > well I assume that covering most common cases with a combo test should be > the general guideline here. I don't think we need 100% coverage to validate > the approach. I would also add some corner cases, probably in separate > tests, those corner cases should include lambdas returning lambdas, lambdas > with inner classes in their bodies, etc. Just playing a bit with this patch, I note that the following example isn't de-duplicated correctly: Runnable r = () -> { int i = 0; if (i==0) {} }; r = () -> { int i = 0; if (i==0) {} }; because symbols don't seem to be '==' for local variables (is that right?). I think we should probably consider using the name instead for local variables, as next. Bernard $ diff -u ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java --- ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 2018-03-19 14:59:49.351288750 +0100 +++ ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java 2018-03-19 15:03:36.416434155 +0100 @@ -172,6 +172,10 @@ result = index == otherIndex; return; } + else { + result = symbol.name == otherSymbol.name; + return; + } } result = tree.sym == that.sym; } $ diff -u ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java --- ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 2018-03-19 15:00:02.743120393 +0100 +++ ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java 2018-03-19 15:01:35.581953251 +0100 @@ -84,6 +84,10 @@ hash(idx); return; } + else { + hash(tree.sym.name); + return; + } } hash(sym); } From vicente.romero at oracle.com Mon Mar 19 15:00:32 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Mon, 19 Mar 2018 11:00:32 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> Message-ID: <727afea4-10ab-b000-8cc4-87c675b98f85@oracle.com> On 03/19/2018 10:41 AM, B. Blaser wrote: > On 19 March 2018 at 14:10, Vicente Romero wrote: >> >> On 03/17/2018 10:04 PM, Liam Miller-Cushon wrote: >>> Hi Vicente, >>> >>> On Sat, Mar 17, 2018 at 11:44 AM Vicente Romero >> > wrote: >>> >>> overall looks good, but I think that a set of regression tests >>> should be included along with the patch. >>> >>> >>> Definitely, I have some functional tests that I'll work on turning into a >>> jtreg test. >>> >>> Do you have advice about how much test coverage is appropriate for tree >>> comparison? >>> Getting to 100% branch coverage would require a lot of test cases. I could >>> probably >>> generate some of them programatically. FWIW a certain amount of TreeDiffer >>> was >>> generated and not hand-written, so any bugs are probably structural >>> problems rather >>> than typos. (That doesn't lessen the need for tests, of course--there's >>> still the risk of >>> adding typos when changes are made in the future.) >> >> well I assume that covering most common cases with a combo test should be >> the general guideline here. I don't think we need 100% coverage to validate >> the approach. I would also add some corner cases, probably in separate >> tests, those corner cases should include lambdas returning lambdas, lambdas >> with inner classes in their bodies, etc. > Just playing a bit with this patch, I note that the following example > isn't de-duplicated correctly: > > Runnable r = () -> { int i = 0; if (i==0) {} }; > r = () -> { int i = 0; if (i==0) {} }; > > because symbols don't seem to be '==' for local variables (is that > right?). right each local should have a different symbol > I think we should probably consider using the name instead > for local variables, as next. working with names could be brittle although it should probably discriminate a lot of cases due to copy / paste, but I guess if we could use other info like type and some positional information > > Bernard Vicente > > > $ diff -u ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 > ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java > --- ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 > 2018-03-19 14:59:49.351288750 +0100 > +++ ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java > 2018-03-19 15:03:36.416434155 +0100 > @@ -172,6 +172,10 @@ > result = index == otherIndex; > return; > } > + else { > + result = symbol.name == otherSymbol.name; > + return; > + } > } > result = tree.sym == that.sym; > } > > $ diff -u ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 > ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java > --- ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 > 2018-03-19 15:00:02.743120393 +0100 > +++ ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java > 2018-03-19 15:01:35.581953251 +0100 > @@ -84,6 +84,10 @@ > hash(idx); > return; > } > + else { > + hash(tree.sym.name); > + return; > + } > } > hash(sym); > } From maurizio.cimadamore at oracle.com Mon Mar 19 21:31:42 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 19 Mar 2018 21:31:42 +0000 Subject: deduplicating lambda methods In-Reply-To: <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> Message-ID: <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> On 17/03/18 18:57, Vicente Romero wrote: > > > On 03/17/2018 02:49 PM, Brian Goetz wrote: >> >>>> >>>> ??? Also, as a bonus, consider constant expressions, we probably want: >>>> ??? return 5; and return 2 + 3; to be equal. >>>> >>>> ??? I think that these two use cases will need a "normalization" step >>>> ??? to be >>>> ??? applied to the tree and store in the map, set, only a normalized >>>> ??? version >>>> ??? of the tree. I understand that at least considering constants >>>> could be >>>> ??? out of the scope of this patch, so I can see it as a follow up >>>> ??? development. >> >> I don't think we want to be duplicating the constant folding logic in >> multiple places, but I also don't think we have to -- doesn't L2M run >> after constant folding already?? I ask because I've already seen >> constant propagation turn capturing lambdas into noncapturing. > > I don't think we need to duplicate the constant folding logic for > this, I just think that the hasher and the differ should be blind > regarding the substructure of a tree as soon as a constant is found, > be it in a variable or as the constant value of a tree. Liam has a > good point, if our new constant folding phase were rewriting the trees > then the differ and the hasher he is proposing would have this for > free. I guess that this is a use case that can make us consider doing > tree rewriting as part of the new constant folding phase, but we are > not doing it right now, only for some trees: basically ldc() indy() > and condys. That's why I was saying that considering constants in the > tree differ / hasher could wait and that it don't have to be part of > this patch. I think this is where I see things going as well - as I said, right now we kind of handle folding in different places - it's not a lot of lines of code - but there are lines scattered in quite few places where things have to be special cased if a type happens to be constant - this is true for the FreeVarCollector in Lower, which is also subclassed by the collector in L2M. If we had an early step such as the one we are envisioning for the condy-folding effort which _reifies_ folding choices in the AST (rather than resorting to special types), it would be much more straightforward for the remaining pieces of the pipeline to just do the 'right thing'. That said, until we get there I also agree with what Vicente says: visitors should not descend into constant values - and that probably includes the hasher/differ visitors. But we should mark this special code for 'removal' for when things will be handled in a more uniform fashion. Also, a side note: we have learned during condy-folding that rewriting trees is not always the right approach - if you are compiling with -g, you might need the trees to remain in place (as you want the compiler to perform less aggressive folding) - so I'm not sure the compiler will always be able to rely on constants to be rewritten in the constant folding phase. Maurizio From maurizio.cimadamore at oracle.com Mon Mar 19 21:37:49 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 19 Mar 2018 21:37:49 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> Message-ID: <99180a8e-0446-f59d-a54c-6aff208638df@oracle.com> On 18/03/18 02:04, Liam Miller-Cushon wrote: > Hi Vicente, > > On Sat, Mar 17, 2018 at 11:44 AM Vicente Romero > > wrote: > > overall looks good, but I think that a set of regression tests > should be included along with the patch. > > > Definitely, I have some functional tests that I'll work on turning > into a jtreg test. > > Do you have advice about how much test coverage is appropriate for > tree comparison? > Getting to 100% branch coverage would require a lot of test cases. I > could probably > generate some of them programatically. FWIW a certain amount of > TreeDiffer was > generated and not hand-written, so any bugs are probably structural > problems rather > than typos. (That doesn't lessen the need for tests, of > course--there's still the risk of > adding typos when changes are made in the future.) How about a test harness that takes a source file, compiles, takes its AST and then uses it as a 'golden' AST to compare it against other AST obtained from: 1) compiling same source file with some random harmless alteration (e.g. alpha renaming of identifiers) 2) compiling same source file with some structural alterations (e.g. drop all bodies from nested blocks) 3) or, compile another user-defined alteration You could get both positive tests (compare AST against itself, or against alpha-renamed trees), and negative tests (compare AST against structural alterations). If we got good at generating such random alteration I bet we could achieve quite good coverage by simply feeding already existing javac test sources to the fuzzing machinery. Maurizio > > at first glance it seemed to me that the hash function would > return the same value for: > (int i) -> i / 5; and (int i) -> 5 / i; I haven't test it but this > was my impression. > > > The hash function currently traverses all nodes and hashes their tags, > so for those two > examples it would see something like `[DIV, IDENT, LITERAL]` and > `[DIV, LITERAL, IDENT]`, > which would not result in the same value. > > This does mean that `x?+ 42` and `y?+ 43` both hashed to the same > value. I updated the > hash function to include literal values and symbols (with the same > handling of method > parameters as the diff). With that change I am no longer seeing hash > collisions that result in > unnecessary diffing. There are still cases where two lambda bodies > hash to the same > value but have different target types, but that case doesn't require a > tree diff (the check > that the target types are the same happens first). > > The updates to hashing are here: > http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.03/ > From vicente.romero at oracle.com Tue Mar 20 12:42:56 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 20 Mar 2018 08:42:56 -0400 Subject: deduplicating lambda methods In-Reply-To: <99180a8e-0446-f59d-a54c-6aff208638df@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <99180a8e-0446-f59d-a54c-6aff208638df@oracle.com> Message-ID: On 03/19/2018 05:37 PM, Maurizio Cimadamore wrote: > > > > On 18/03/18 02:04, Liam Miller-Cushon wrote: >> Hi Vicente, >> >> On Sat, Mar 17, 2018 at 11:44 AM Vicente Romero >> > wrote: >> >> overall looks good, but I think that a set of regression tests >> should be included along with the patch. >> >> >> Definitely, I have some functional tests that I'll work on turning >> into a jtreg test. >> >> Do you have advice about how much test coverage is appropriate for >> tree comparison? >> Getting to 100% branch coverage would require a lot of test cases. I >> could probably >> generate some of them programatically. FWIW a certain amount of >> TreeDiffer was >> generated and not hand-written, so any bugs are probably structural >> problems rather >> than typos. (That doesn't lessen the need for tests, of >> course--there's still the risk of >> adding typos when changes are made in the future.) > How about a test harness that takes a source file, compiles, takes its > AST and then uses it as a 'golden' AST to compare it against other AST > obtained from: > > 1) compiling same source file with some random harmless alteration > (e.g. alpha renaming of identifiers) > 2) compiling same source file with some structural alterations (e.g. > drop all bodies from nested blocks) > 3) or, compile another user-defined alteration > > You could get both positive tests (compare AST against itself, or > against alpha-renamed trees), and negative tests (compare AST against > structural alterations). > > If we got good at generating such random alteration I bet we could > achieve quite good coverage by simply feeding already existing javac > test sources to the fuzzing machinery. neat! > > Maurizio Vicente >> at first glance it seemed to me that the hash function would >> return the same value for: >> (int i) -> i / 5; and (int i) -> 5 / i; I haven't test it but >> this was my impression. >> >> >> The hash function currently traverses all nodes and hashes their >> tags, so for those two >> examples it would see something like `[DIV, IDENT, LITERAL]` and >> `[DIV, LITERAL, IDENT]`, >> which would not result in the same value. >> >> This does mean that `x?+ 42` and `y?+ 43` both hashed to the same >> value. I updated the >> hash function to include literal values and symbols (with the same >> handling of method >> parameters as the diff). With that change I am no longer seeing hash >> collisions that result in >> unnecessary diffing. There are still cases where two lambda bodies >> hash to the same >> value but have different target types, but that case doesn't require >> a tree diff (the check >> that the target types are the same happens first). >> >> The updates to hashing are here: >> http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.03/ >> > From federico.peralta at gmail.com Tue Mar 20 13:57:13 2018 From: federico.peralta at gmail.com (Federico Peralta Schaffner) Date: Tue, 20 Mar 2018 13:57:13 +0000 Subject: Records -- current status In-Reply-To: References: Message-ID: Hi all, First time posting (though I've been silently following discussions for quite some time). As this is my very first message, I might be missing some important points already discussed (my apologies if this is the case). Please see my comments inline, thanks for letting the community participate in these discussions. Regards, fps.- 2018-03-16 15:55 GMT-03:00 Brian Goetz : > > - Extension. The proposal outlines a notion of abstract record, which provides a "width subtyped" hierarchy. Some have questioned whether this carries its weight, especially given how Scala doesn't support case-to-case extension (some see this as a bug, others as an existence proof.) Records can implement interfaces. In the linked document ( http://cr.openjdk.java.net/~briangoetz/amber/datum.html), it says that if a concrete record extends an abstract record, the state vector of the abstract record must be a prefix of the state vector of the concrete record. An example: abstract record Base(int x, int y); record Sub(int x, int y, int z) extends Base(x, y); So, my question is why Base(x, y) and why not i.e. Base(y, z) or Base (x, z) or even Base (y, x), etc? Initially, as fields are referenced by name, I saw no reason for concrete records to be able to only append new fields to the state vector. However, when I was to think of a concrete example where it would be useful to relax the "prefix constraint", I must admit that I couldn't find any practical example... Anyway, although only from a theoretical point of view, it would be nice to know the rationale behind this constraint. > > - Concrete records are final. Relaxing this adds complexity to the equality story; I'm not seeing good reasons to do so. Absolutely agree. > > - Additional constructors. I don't see any reason why additional constructors are problematic, especially if they are constrained to delegate to the default constructor (which in turn is made far simpler if there can be statements ahead of the this() call.) Users may find the lack of additional constructors to be an arbitrary limitation (and they'd probably be right.) I also agree here. A typical use case would be to use additional constructors accepting less fields to provide default values to the principal or default constructor. Besides this, I think that copy constructors are widely used (much more than Cloneable and .clone()), which makes me think about 2 things: 1. What about automatically providing a copy constructor for records? 2. If providing a built-in copy constructor is not desired for all records (after all, not every record needs to be copied), it would be very useful to have a succint way to express the need of a copy constructor. For example: Instead of writing: record Point(int x, int y) { public Point(Point p) { default.this(p.x, p.y); } } We could only write: record Point(int x, int y) { public Point(Point p); } Here the delegation to the principal constructor could be implemented by the compiler. Besides, in a copy constructor, there's no need to perform any validation on the arguments, as they come from an already constructed record instance. Whether the fields of the source record (the one to be copied) satisfy the invariants imposed in the overriden default constructor (if there is one actually defined), or if the fields of the source record don't satisfy these invariants, e.g. if some mutable field has been reassigned in the source record by means of a mutator method invoked after construction (more about this below), it is irrelevant to the record instance; in any case, there's no need to re-validate the fields against the invariants. I think that the built-in implementation of copy constructors should perform a shallow copy. If the programmer wants to implement a deep copy mechanism, the copy constructor implementation could be explicitly provided: record Address(String street, String city, String country) { } record Person(String name, Address address) { public Person(Person p) { default.this(p.name, new Address(p.address)); } } > > - Static fields. Static fields seem harmless. Yes, no problem with static fields. > > - Additional instance fields. These are a much bigger concern. While the primary arguments against them are of the "slippery slope" variety, I still have deep misgivings about supporting unrestricted non-principal instance fields, and I also haven't found a reasonable set of restrictions that makes this less risky. I'd like to keep looking for a better story here, before just caving on this, as I worry doing so will end up biting us in the back. I think that additional fields can be very dangerous for the little benefit they might bring to records. In fact, I don't see a real value for them. Suppose you have: record Name(String first, String last) { public firstAndLast() { return first + " " + last; } } Why caching the result of first + " " + last in an additional firstAndLast instance field? For records, it should be enough to expose derived state via accessor methods. If you want to cache some value (because it's expensive to calculate), you can always resort to classes, or could maybe use another record. For the example above, you could have: record FullName(Name name, String fullName) { @Override public FullName(Name name, String fullName) { if (!name.firstAndLast().equals(fullName)) { throw new IllegalArgumentException(); } default.this(name, name.firstAndLast()); } public FullName(Name name) { this(name, name.firstAndLast()); // delegates to the overriden default constructor } } And use it in a decorator fashion: FullName someone = new FullName(new Name("John", "Doe")); String fullName = someone.fullName(); // John Doe String first = someone.name().first(); // John String last = someone.name().last(); // Doe > > - Mutability and accessibility. I'd like to propose an odd choice here, which is: fields are final and package (protected for abstract records) by default, but finality can be explicitly opted out of (non-final) and accessibility can be explicitly widened (public). I love final fields by default, I think that finality should be encouraged by the language. However, I find it very unpleasant to use a keyword (or a reserved word, reserved type name, etc) to change a field's default finality. Maybe I'm asking for an impossible here, but isn't there a mechanism similar to effectively-final variables, to detect whether a final field is being mutated? Certainly, the compiler is able to detect these cases, such as when an effectively-final variable is being modified inside a lambda. My proposal is to use similar detection mechanics. Instead of throwing a compilation error when a field is being mutated, the compiler could internally mark that field as non-final. Of course, fields could only be reassigned from within explicitly declared mutator methods. So, if this is at all possible, we wouldn't need to declare fields as neither non-final nor final: the compiler would automatically figure it out. In other words, the intention of the programmer to explicitly reassign a field within a method of the record would make the field implicitly non-final. (Disclaimer: please bear with me if I'm fantasizing too much here, as I'm not aware of the internals of the compiler or whether this is a viable choice. Anyways, just imagining this possibility has been fun). ... With regards to accessibility, I think that all fields should be public. Yes, public! But only for reading purposes (fields should only be reassigned from within setter/mutator methods). If I'm getting the idea correctly, the spirit of records is to be transparent. We don't need any encapsulation here. So, why making fields package-private or protected, and providing public, automatically-built accessors? Instead, we should let the information (and just the information) be always exposed. I'm well aware that the Java world hates public fields, and that is OK in the context of classes. But this is records, not classes, we're living in a totally different world. Exposing the fields publicly (they'll be public via deconstructors, after all) would show the clear intention that records are all about transparent information, and that they sit at the opposite side of encapsulation (aka information-hiding). Simply put, records with public fields enforce the concept of information transparency. But we need to support the uniform access principle... Really? Do we? Why? Records are a new functionality. We will be already providing access to the fields via deconstructors when pattern-matching is available, so why should we respect that principle here as well? This is just an opinion and I don't want to create polemic, but whereas the uniform access principle is one of those principles that are beautiful in theory, it's worth noting that it's been the source of nasty bugs in practice. I'm thinking specifically about OR mapper frameworks. You invoke a simple getter and you get a query that performs 12 joins in the background. And you'd better pray the gods if you were invoking that getter inside a loop... Do we really want to support a principle that promotes these kind of behavior? Classes are classes and records are records. They are very different each other, why not making these differences more evident by changing the way fields are accessed? If you want to return a defensive copy of some field, you can always write an explicit accessor that i.e. uses the copy constructor (see above) to return a field that is in turn a record of another type, or an unmidifiable map, etc. This exceptional behavior should be explicit IMO, particularly if fields cannot be reassigned from outside of their enclosing record. Last but not least, I think it's a de-facto pseudo-standard to use fieldName() to return Optional for single instance fields. The proposed fieldName() built-in accessors would collide with this pseudo-standard. > > - Accessors. Perhaps the most controversial aspect is that records are inherently transparent to read; if something wants to truly encapsulate state, it's not a record. Records will eventually have pattern deconstructors, which will expose their state, so we should go out of the gate with the equivalent. The obvious choice is to expose read accessors automatically. (These will not be named getXxx; we are not burning the ill-advised Javabean naming conventions into the language, no matter how much people think it already is.) The obvious naming choice for these accessors is fieldName(). No provision for write accessors; that's bring-your-own. I think that this quite supports my previous arguments, except for the read accessors. If we don't want to burn Javabean naming conventions into the language, why should we burn any other convention into it? Agree with explicit write accessors, with the caveat of my above proposal to make reassigned fields effectively-non-final. > > - Core methods. Records will get equals, hashCode, and toString. There's a good argument for making equals/hashCode final (so they can't be explicitly redeclared); this gives us stronger preservation of the data invariants that allow us to safely and mechanically snapshot / serialize / marshal (we'd definitely want this if we ever allowed additional instance fields.) No reason to suppress override of toString, though. Absolutely agree. If you want to provide a custom implementation for equals/hashCode, just use a class. Besides, this opens the possibility of hash code randomization for records, i.e. to use some seed dependent on the current run of the JVM to be used as the base to calculate hash codes (something akin JDK9's immutable sets and maps). > Records could be safely made cloneable() with automatic support too (like arrays), but not clear if this is worth it (its darn useful for arrays, though.) I think the auto-generated getters should be final too; this leaves arrays as second-class components, but I am not sure that bothers me. I've never used neither the Clonable interface nor the .clone() method in my life. If I ever had the need to create copies of an object, I just declared a copy constructor and used it instead. I only talk from my experience, but I doubt .clone() and Clonable are widely used. Besides it's (seen as) broken, implementing a copy constructor is safer and easier than implementing Cloneable and overriding the .clone() method (without forgetting to make it public). If deep cloning is needed, this should be accomplished in the copy constructor. From brian.goetz at oracle.com Tue Mar 20 14:35:42 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 20 Mar 2018 10:35:42 -0400 Subject: Records -- current status In-Reply-To: References: Message-ID: <595e8fa7-12b0-2a26-78f7-214861cf2c33@oracle.com> > In the linked document ( > http://cr.openjdk.java.net/~briangoetz/amber/datum.html), it says that if a > concrete record extends an abstract record, the state vector of the > abstract record must be a prefix of the state vector of the concrete > record. Correct. > So, my question is why Base(x, y) and why not i.e. Base(y, z) or Base (x, > z) or even Base (y, x), etc? The motivation is to suggest "width subtyping" (https://en.wikipedia.org/wiki/Subtyping#Width_and_depth_subtyping); while there is no reason we couldn't do as you suggest, the more constrained variant guides users towards the sorts of subtyping that make sense, which is, "this record subsumes that record."? And if it does, it is clearer if its all in one place. I think users are going to have a hard time reasoning about the boundary of when to use a record vs when they have to "fall back to" a class (and, the fact that records get the pretty syntax will unfortunately multiply this.)? The clearer the boundaries and restrictions, the easier it is to form a mental model of what records are for, which is: "it's just data". > > 1. What about automatically providing a copy constructor for records? This starts to press on a more general point, which is: what if I want a record, all of whose fields are the same as another record, but with one changed?? This is the equivalent of "setter" for immutable objects, sometimes called a "wither".? One way to get to withers is the same way we get to setters and getters; another is to treat it as a form of constructor where, if some fields are not specified, they are copied from another instance.? (C# supports this through the combination of named parameters, default values, and "receiver-defaults.")? This is something we're thinking about, especially as we approach value types, but don't have a clear story on yet. > 2. If providing a built-in copy constructor is not desired for all records > (after all, not every record needs to be copied), it would be very useful > to have a succint way to express the need of a copy constructor. For > example: > > Instead of writing: > > record Point(int x, int y) { > public Point(Point p) { > default.this(p.x, p.y); > } > } > > We could only write: > > record Point(int x, int y) { > public Point(Point p); > } Yeah, that's cute.? And possibly same for factories (if there's a factory that exposes the same signature as a constructor, infer the obvious constructor body.) > I think that the built-in implementation of copy constructors should > perform a shallow copy. I think this is the only think we could realistically do.? If you want deeper, provide it yourself (and same for clone.) > > Why caching the result of first + " " + last in an additional firstAndLast > instance field? For records, it should be enough to expose derived state > via accessor methods. My hope is that this is good enough for V1.? Yes, it gives up some optimization opportunities.? (OTOH, developers tend to significantly over-value optimization, and if they are so obsessed with performance, than maybe what they have is not a record, but a class.? And, just like bad programmers will gleefully use one-element arrays to "fool" the "dumb" compiler's rules about mutable capture in lambdas, some will take the same glee in outsmarting us with a static HashMap.? (The thing that makes them bad is not the trick, but the glee.? If you feel clever doing this, you're doing it wrong.)) > > I love final fields by default, I think that finality should be encouraged > by the language. However, I find it very unpleasant to use a keyword (or a > reserved word, reserved type name, etc) to change a field's default > finality. Maybe I'm asking for an impossible here, but isn't there a > mechanism similar to effectively-final variables, to detect whether a final > field is being mutated? Certainly, the compiler is able to detect these > cases, such as when an effectively-final variable is being modified inside > a lambda. Yeah, that's asking too much :)? Fields can be public (in which case you can't find all the writes at compile time); they can be modified reflectively; etc.? I think its reasonable to ask people to be explicit here. > With regards to accessibility, I think that all fields should be public. > Yes, public! But only for reading purposes (fields should only be > reassigned from within setter/mutator methods). Heh, we explored both of these ideas for a while -- both public, and a variant public-read, private-write.? (If they were always final, then public final would be a credible option, but only if there were no chance we'd ever relax finality.)? But, I don't think think Java developers would ever accept it.? I'd much rather position records as "like the classes you've been writing all along, but less annoying." > Last but not least, I think it's a de-facto pseudo-standard to use > fieldName() to return Optional for single instance fields. I don't believe this is remotely as prevalent as you're suggesting (though there are surely regional dialects).? Yes, its becoming more common to name accessors without the silly "get" prefix, which I support.? And, some accessors choose to normalize the type, and one such normalization is wrapping with Optional.? But these are two separate issues. > > Absolutely agree. If you want to provide a custom implementation for > equals/hashCode, just use a class. Besides, this opens the possibility of > hash code randomization for records, i.e. to use some seed dependent on the > current run of the JVM to be used as the base to calculate hash codes > (something akin JDK9's immutable sets and maps). We also have plans to make equals/hashCode easier to write, so that you don't have to fall all the way down the syntactic cliff to downgrade from records to classes.? This should make the landing less unpleasant. > > I've never used neither the Clonable interface nor the .clone() method in > my life. I use clone() for arrays 100% of the time when I build APIs that use arrays.? In the constructor, you can just say ??? this.array = array.clone() and in the read accessor you can say ??? return array.clone() It's really convenient, safe, and 100% clear what's going on. From cyrill.brunner at hispeed.ch Tue Mar 20 14:42:47 2018 From: cyrill.brunner at hispeed.ch (Cyrill) Date: Tue, 20 Mar 2018 15:42:47 +0100 Subject: Records -- current status In-Reply-To: <595e8fa7-12b0-2a26-78f7-214861cf2c33@oracle.com> References: <595e8fa7-12b0-2a26-78f7-214861cf2c33@oracle.com> Message-ID: <834442a8-e82c-d674-36fd-d8286127a7be@hispeed.ch> >> 1. What about automatically providing a copy constructor for records? > > [...]? (C# supports this through the combination of named parameters, > default values, and "receiver-defaults.")? This is something we're > thinking about, especially as we approach value types, but don't have > a clear story on yet. Speaking of default values & named params, is there an effort for these yet? Or is it just, as you say, not yet a clear story and thus maybe a side-topic in the valhalla ML? I'd be interested to follow the discussions once they come up. Kind regards, Cyrill Brunner From brian.goetz at oracle.com Tue Mar 20 14:48:39 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 20 Mar 2018 10:48:39 -0400 Subject: Records -- current status In-Reply-To: <834442a8-e82c-d674-36fd-d8286127a7be@hispeed.ch> References: <595e8fa7-12b0-2a26-78f7-214861cf2c33@oracle.com> <834442a8-e82c-d674-36fd-d8286127a7be@hispeed.ch> Message-ID: It's certainly on our list of things we'd like to think about. But we've got so much on our plate, we'd like to focus on getting records and pattern matching on a much firmer footing first. On 3/20/2018 10:42 AM, Cyrill wrote: > >>> 1. What about automatically providing a copy constructor for records? >> >> [...]? (C# supports this through the combination of named parameters, >> default values, and "receiver-defaults.")? This is something we're >> thinking about, especially as we approach value types, but don't have >> a clear story on yet. > Speaking of default values & named params, is there an effort for > these yet? Or is it just, as you say, not yet a clear story and thus > maybe a side-topic in the valhalla ML? I'd be interested to follow the > discussions once they come up. > > Kind regards, > Cyrill Brunner From james.laskey at oracle.com Tue Mar 20 17:50:25 2018 From: james.laskey at oracle.com (james.laskey at oracle.com) Date: Tue, 20 Mar 2018 17:50:25 +0000 Subject: hg: amber/amber: 0000000: Change name of trimXXXX routines to stripXXXX Message-ID: <201803201750.w2KHoP5L016387@aojmv0008.oracle.com> Changeset: bc38df004048 Author: jlaskey Date: 2018-03-20 14:45 -0300 URL: http://hg.openjdk.java.net/amber/amber/rev/bc38df004048 0000000: Change name of trimXXXX routines to stripXXXX ! src/java.base/share/classes/java/lang/String.java ! src/java.base/share/classes/java/lang/StringLatin1.java ! src/java.base/share/classes/java/lang/StringUTF16.java ! test/jdk/java/lang/String/RawStringLiteralLib.java From bsrbnd at gmail.com Wed Mar 21 12:24:30 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 21 Mar 2018 13:24:30 +0100 Subject: deduplicating lambda methods In-Reply-To: <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> Message-ID: On 19 March 2018 at 22:31, Maurizio Cimadamore wrote: > > [...] > > Also, a side note: we have learned during condy-folding that rewriting trees > is not always the right approach - if you are compiling with -g, you might > need the trees to remain in place (as you want the compiler to perform less > aggressive folding) - so I'm not sure the compiler will always be able to > rely on constants to be rewritten in the constant folding phase. > > Maurizio This makes me think we could disable dedup with '-g' to preserve full debug-ability (as next), is this partially what you meant? Bernard $ diff -u ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java --- ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 2018-03-21 12:51:14.338021489 +0100 +++ ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java 2018-03-21 12:51:56.209495093 +0100 @@ -113,6 +113,8 @@ /** force serializable representation, for stress testing **/ private final boolean forceSerializable; + private final boolean debug; + /** Flag for alternate metafactories indicating the lambda object is intended to be serializable */ public static final int FLAG_SERIALIZABLE = 1 << 0; @@ -149,6 +151,7 @@ dumpLambdaToMethodStats = options.isSet("debug.dumpLambdaToMethodStats"); attr = Attr.instance(context); forceSerializable = options.isSet("forceSerializable"); + debug = options.isSet(Option.G) || options.isSet(Option.G_CUSTOM, "source"); } // @@ -365,7 +368,7 @@ lambdaDecl.body = translate(makeLambdaBody(tree, lambdaDecl)); boolean dedupe = false; - if (!localContext.isSerializable()) { + if (!localContext.isSerializable() && !debug) { DedupedLambda dedupedLambda = new DedupedLambda(lambdaDecl.sym, lambdaDecl.body); DedupedLambda existing = kInfo.dedupedLambdas.putIfAbsent(dedupedLambda, dedupedLambda); if (existing != null) { From maurizio.cimadamore at oracle.com Wed Mar 21 12:48:32 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 21 Mar 2018 12:48:32 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> Message-ID: <2375f941-2ee2-4d54-3914-72f8c1a6a679@oracle.com> On 21/03/18 12:24, B. Blaser wrote: > On 19 March 2018 at 22:31, Maurizio Cimadamore > wrote: >> [...] >> >> Also, a side note: we have learned during condy-folding that rewriting trees >> is not always the right approach - if you are compiling with -g, you might >> need the trees to remain in place (as you want the compiler to perform less >> aggressive folding) - so I'm not sure the compiler will always be able to >> rely on constants to be rewritten in the constant folding phase. >> >> Maurizio > This makes me think we could disable dedup with '-g' to preserve full > debug-ability (as next), is this partially what you meant? Yes and no, but I agree that it's better to conservatively disable deduplication with -g. Maurizio > > Bernard > > > $ diff -u ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 > ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > --- ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 > 2018-03-21 12:51:14.338021489 +0100 > +++ ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > 2018-03-21 12:51:56.209495093 +0100 > @@ -113,6 +113,8 @@ > /** force serializable representation, for stress testing **/ > private final boolean forceSerializable; > > + private final boolean debug; > + > /** Flag for alternate metafactories indicating the lambda object > is intended to be serializable */ > public static final int FLAG_SERIALIZABLE = 1 << 0; > > @@ -149,6 +151,7 @@ > dumpLambdaToMethodStats = > options.isSet("debug.dumpLambdaToMethodStats"); > attr = Attr.instance(context); > forceSerializable = options.isSet("forceSerializable"); > + debug = options.isSet(Option.G) || > options.isSet(Option.G_CUSTOM, "source"); > } > // > > @@ -365,7 +368,7 @@ > lambdaDecl.body = translate(makeLambdaBody(tree, lambdaDecl)); > > boolean dedupe = false; > - if (!localContext.isSerializable()) { > + if (!localContext.isSerializable() && !debug) { > DedupedLambda dedupedLambda = new > DedupedLambda(lambdaDecl.sym, lambdaDecl.body); > DedupedLambda existing = > kInfo.dedupedLambdas.putIfAbsent(dedupedLambda, dedupedLambda); > if (existing != null) { From james.laskey at oracle.com Wed Mar 21 13:53:32 2018 From: james.laskey at oracle.com (james.laskey at oracle.com) Date: Wed, 21 Mar 2018 13:53:32 +0000 Subject: hg: amber/amber: 0000000: Rename trimIndent, trimMarkers to stripIndent, stripMarkers Message-ID: <201803211353.w2LDrXEP001674@aojmv0008.oracle.com> Changeset: f2ec59847c27 Author: jlaskey Date: 2018-03-21 10:35 -0300 URL: http://hg.openjdk.java.net/amber/amber/rev/f2ec59847c27 0000000: Rename trimIndent, trimMarkers to stripIndent, stripMarkers ! src/java.base/share/classes/java/lang/String.java ! test/jdk/java/lang/String/RawStringLiteralLib.java From james.laskey at oracle.com Wed Mar 21 15:10:09 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 21 Mar 2018 12:10:09 -0300 Subject: Raw String Literals Message-ID: <10B03F59-E391-4F94-B4FE-FD76537D72E8@oracle.com> We think things have "settled in" with respect to Raw String Literals language changes and library support. If all things fall in place, we will probably move http://openjdk.java.net/jeps/326 forward soon. We are hoping to make builds available, but if you want to test the waters. hg clone http://hg.openjdk.java.net/amber/amber amber-raw-string-literals cd amber-raw-string-literals hg update raw-string-literal ./configure ? make images docs export PLATFORM= ... java home is at build/${PLATFORM}/images/jdk javadoc for String library support is at build/${PLATFORM}/images/docs/api/java.base/java/lang/String.html Cheers, ? Jim From brian.goetz at oracle.com Wed Mar 21 16:07:23 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 21 Mar 2018 12:07:23 -0400 Subject: Stats: Effect of translating lambda with condy, and folding, on JDK Message-ID: <2ff59ea9-5097-efbe-5485-cc7aa661085a@oracle.com> Vicente has run some statistics on the effect of two optimizations, using the JDK as a corpus: ?- translating non-capturing lambdas and method refs using condy instead of indy ?- constant folding Translating lambdas using condy instead of indy should result in two improvements: ?- condy is lighter weight at link time (faster startup) ?- better sharing of duplicated method references / lambdas Currently, if a class has two identical method references, they will each get their own indy site, which will share a Constant_InvokeDynamic_info (and bootstrap methods entry.)? But, since each indy is linked independently, linkage will be done redundantly, resulting in duplicate proxy classes being spun.? If we convert this over to condy, we'll have two LDCs of the same constant, which only gets resolved once. As a baseline, we collected counts of { lambda, method ref } x { capturing, not } x { serializable, not }, in the corpus (the JDK): ? ? ? ? ? ?? ?? ????? Ser????? NonSer Lambda??? Stateless??? 7?????? 953 ? ? ?? ?? Stateful??? 10?????? 1688 Ref ? ? ? Stateless?? 23?? ? ? 712 ? ?? ? ?? Stateful??? ? ? ? ?? 528 And we collected counts of indy constant pool entries and indy instructions that correspond to LambdaMetafactory: #indyCP:?? 3683 #indyInsn: 3966 We would expect to see a count of indy instructions equal to the number of nonserializable lambdas/mrefs plus twice the number of serializable lambdas/mrefs (since there is a twin capture site in the $deserialize$ method.)? We got close; we expected 3961, but saw 3966.? The remaining five are accounted by lambdas that were in contexts that got duplicated by the compiler (finally blocks, instance initializers.) The difference between the indy instructions and CP entries has to do with duplicated lambdas or methods refs.? We'd expect to see these for serializable, for those that are implicitly duplicated as above, and also when method refs are explicitly duplicated within a classfile.? So we saw ~230 reuses that must be the result of explicit source duplication of method refs. We then ran the same stats with translating non-capturing lambdas and non-capturing (static, unbound, and constructor) method references using condy. #indyCP? ? ? 2156 // shared=125 #indyInsn ?? 2281 #condyCP ? ? 1527 // shared=158 #condyLDC ?? 1685 What this means is that approximately 40% of lambdas and method refs could be diverted from indy to condy.? Any explicitly duplicated method refs would get the benefit of constant pool sharing. When the lambda deduplication effort completes, the numbers should get better, as there will be more candidates for sharing of CP entries (currently, even identical lambdas get their own lambda$nn method.) Looking ahead, the richer constant propagation will help as well, as it may push some lambdas from the "capturing" to the "non-capturing" column.? Not only are non-capturing lambdas more performant today, but then they also become candidates for condy translation. The JDK is probably not an ideal corpus to run this on; if someone wants to try the stats gathering patch on a different corpus, that would be great. From bsrbnd at gmail.com Wed Mar 21 19:00:15 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 21 Mar 2018 20:00:15 +0100 Subject: deduplicating lambda methods In-Reply-To: <2375f941-2ee2-4d54-3914-72f8c1a6a679@oracle.com> References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> <2375f941-2ee2-4d54-3914-72f8c1a6a679@oracle.com> Message-ID: On 21 March 2018 at 13:48, Maurizio Cimadamore wrote: > > > On 21/03/18 12:24, B. Blaser wrote: >> >> On 19 March 2018 at 22:31, Maurizio Cimadamore >> wrote: >>> >>> [...] >>> >>> Also, a side note: we have learned during condy-folding that rewriting >>> trees >>> is not always the right approach - if you are compiling with -g, you >>> might >>> need the trees to remain in place (as you want the compiler to perform >>> less >>> aggressive folding) - so I'm not sure the compiler will always be able to >>> rely on constants to be rewritten in the constant folding phase. >>> >>> Maurizio >> >> This makes me think we could disable dedup with '-g' to preserve full >> debug-ability (as next), is this partially what you meant? > > Yes and no, but I agree that it's better to conservatively disable > deduplication with -g. > > Maurizio I note also that we may want to skip empty blocs '{}' and statements ';'. The following examples would then produce only 2 lambda methods instead of currently 8: Runnable r = () -> {}; r = () -> { ; }; r = () -> { {} }; r = () -> { { ; } }; r = () -> { int i = 0; if (i==0) ; }; r = () -> { int i = 0; if (i==0) {} }; r = () -> { int i = 0; if (i==0) { ; } }; r = () -> { int i = 0; if (i==0) { {} } }; I had to write a 'NoOpFinder' class (does something like that already exist?) and I added a default action in 'TreeScanner' for that (see below). Does this look reasonable (I did only some quick tests)? Bernard $ diff -u src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java --- src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 2018-03-19 14:59:49.351288750 +0100 +++ src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java 2018-03-21 19:04:46.282040040 +0100 @@ -94,6 +94,8 @@ import java.util.Iterator; import java.util.Objects; +import static com.sun.tools.javac.comp.NoOpFinder.isNoOp; + /** A visitor that compares two lambda bodies for structural equality. */ public class TreeDiffer extends TreeScanner { @@ -109,8 +111,9 @@ private boolean result; public boolean scan(JCTree tree, JCTree parameter) { - if (tree == null || parameter == null) { - return tree == null && parameter == null; + boolean nop1 = isNoOp(tree), nop2 = isNoOp(parameter); + if (nop1 || nop2) { + return nop1 && nop2; } tree = TreeInfo.skipParens(tree); parameter = TreeInfo.skipParens(parameter); @@ -172,6 +175,10 @@ result = index == otherIndex; return; } + else { + result = symbol.name == otherSymbol.name; + return; + } } result = tree.sym == that.sym; } $ diff -u src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java --- src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 2018-03-19 15:00:02.743120393 +0100 +++ src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java 2018-03-21 19:04:25.454301881 +0100 @@ -61,7 +61,7 @@ @Override public void scan(JCTree tree) { - if (tree == null) { + if (tree == null || NoOpFinder.isNoOp(tree)) { return; } tree = TreeInfo.skipParens(tree); @@ -84,6 +84,10 @@ hash(idx); return; } + else { + hash(tree.sym.name); + return; + } } hash(sym); } diff --git a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java new file mode 100644 --- /dev/null +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java @@ -0,0 +1,29 @@ +package com.sun.tools.javac.comp; + +import com.sun.tools.javac.tree.JCTree; +import com.sun.tools.javac.tree.TreeScanner; + +public class NoOpFinder extends TreeScanner { + + public static boolean isNoOp(JCTree tree) { + NoOpFinder nop = new NoOpFinder(); + if (tree != null) + nop.scan(tree); + return nop.result; + } + + private boolean result = true; + + private NoOpFinder() { + defaultAction = () -> { result = false; }; + } + + @Override + public void visitSkip(JCTree.JCSkip tree) { + } + + @Override + public void visitBlock(JCTree.JCBlock tree) { + scan(tree.stats); + } +} diff --git a/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java b/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java @@ -43,6 +43,8 @@ */ public class TreeScanner extends Visitor { + protected Runnable defaultAction = () -> {}; + /** Visitor method: Scan a single node. */ public void scan(JCTree tree) { @@ -63,16 +65,19 @@ ****************************************************************************/ public void visitTopLevel(JCCompilationUnit tree) { + defaultAction.run(); scan(tree.defs); } public void visitPackageDef(JCPackageDecl tree) { + defaultAction.run(); scan(tree.annotations); scan(tree.pid); } @Override public void visitModuleDef(JCModuleDecl tree) { + defaultAction.run(); scan(tree.mods); scan(tree.qualId); scan(tree.directives); @@ -80,37 +85,44 @@ @Override public void visitExports(JCExports tree) { + defaultAction.run(); scan(tree.qualid); scan(tree.moduleNames); } @Override public void visitOpens(JCOpens tree) { + defaultAction.run(); scan(tree.qualid); scan(tree.moduleNames); } @Override public void visitProvides(JCProvides tree) { + defaultAction.run(); scan(tree.serviceName); scan(tree.implNames); } @Override public void visitRequires(JCRequires tree) { + defaultAction.run(); scan(tree.moduleName); } @Override public void visitUses(JCUses tree) { + defaultAction.run(); scan(tree.qualid); } public void visitImport(JCImport tree) { + defaultAction.run(); scan(tree.qualid); } public void visitClassDef(JCClassDecl tree) { + defaultAction.run(); scan(tree.mods); scan(tree.typarams); scan(tree.extending); @@ -119,6 +131,7 @@ } public void visitMethodDef(JCMethodDecl tree) { + defaultAction.run(); scan(tree.mods); scan(tree.restype); scan(tree.typarams); @@ -130,6 +143,7 @@ } public void visitVarDef(JCVariableDecl tree) { + defaultAction.run(); scan(tree.mods); scan(tree.vartype); scan(tree.nameexpr); @@ -137,23 +151,28 @@ } public void visitSkip(JCSkip tree) { + defaultAction.run(); } public void visitBlock(JCBlock tree) { + defaultAction.run(); scan(tree.stats); } public void visitDoLoop(JCDoWhileLoop tree) { + defaultAction.run(); scan(tree.body); scan(tree.cond); } public void visitWhileLoop(JCWhileLoop tree) { + defaultAction.run(); scan(tree.cond); scan(tree.body); } public void visitForLoop(JCForLoop tree) { + defaultAction.run(); scan(tree.init); scan(tree.cond); scan(tree.step); @@ -161,31 +180,37 @@ } public void visitForeachLoop(JCEnhancedForLoop tree) { + defaultAction.run(); scan(tree.var); scan(tree.expr); scan(tree.body); } public void visitLabelled(JCLabeledStatement tree) { + defaultAction.run(); scan(tree.body); } public void visitSwitch(JCSwitch tree) { + defaultAction.run(); scan(tree.selector); scan(tree.cases); } public void visitCase(JCCase tree) { + defaultAction.run(); scan(tree.pat); scan(tree.stats); } public void visitSynchronized(JCSynchronized tree) { + defaultAction.run(); scan(tree.lock); scan(tree.body); } public void visitTry(JCTry tree) { + defaultAction.run(); scan(tree.resources); scan(tree.body); scan(tree.catchers); @@ -193,52 +218,63 @@ } public void visitCatch(JCCatch tree) { + defaultAction.run(); scan(tree.param); scan(tree.body); } public void visitConditional(JCConditional tree) { + defaultAction.run(); scan(tree.cond); scan(tree.truepart); scan(tree.falsepart); } public void visitIf(JCIf tree) { + defaultAction.run(); scan(tree.cond); scan(tree.thenpart); scan(tree.elsepart); } public void visitExec(JCExpressionStatement tree) { + defaultAction.run(); scan(tree.expr); } public void visitBreak(JCBreak tree) { + defaultAction.run(); } public void visitContinue(JCContinue tree) { + defaultAction.run(); } public void visitReturn(JCReturn tree) { + defaultAction.run(); scan(tree.expr); } public void visitThrow(JCThrow tree) { + defaultAction.run(); scan(tree.expr); } public void visitAssert(JCAssert tree) { + defaultAction.run(); scan(tree.cond); scan(tree.detail); } public void visitApply(JCMethodInvocation tree) { + defaultAction.run(); scan(tree.typeargs); scan(tree.meth); scan(tree.args); } public void visitNewClass(JCNewClass tree) { + defaultAction.run(); scan(tree.encl); scan(tree.typeargs); scan(tree.clazz); @@ -247,6 +283,7 @@ } public void visitNewArray(JCNewArray tree) { + defaultAction.run(); scan(tree.annotations); scan(tree.elemtype); scan(tree.dims); @@ -256,90 +293,110 @@ } public void visitLambda(JCLambda tree) { + defaultAction.run(); scan(tree.body); scan(tree.params); } public void visitParens(JCParens tree) { + defaultAction.run(); scan(tree.expr); } public void visitAssign(JCAssign tree) { + defaultAction.run(); scan(tree.lhs); scan(tree.rhs); } public void visitAssignop(JCAssignOp tree) { + defaultAction.run(); scan(tree.lhs); scan(tree.rhs); } public void visitUnary(JCUnary tree) { + defaultAction.run(); scan(tree.arg); } public void visitBinary(JCBinary tree) { + defaultAction.run(); scan(tree.lhs); scan(tree.rhs); } public void visitTypeCast(JCTypeCast tree) { + defaultAction.run(); scan(tree.clazz); scan(tree.expr); } public void visitTypeTest(JCInstanceOf tree) { + defaultAction.run(); scan(tree.expr); scan(tree.clazz); } public void visitIndexed(JCArrayAccess tree) { + defaultAction.run(); scan(tree.indexed); scan(tree.index); } public void visitSelect(JCFieldAccess tree) { + defaultAction.run(); scan(tree.selected); } public void visitReference(JCMemberReference tree) { + defaultAction.run(); scan(tree.expr); scan(tree.typeargs); } public void visitIdent(JCIdent tree) { + defaultAction.run(); } public void visitLiteral(JCLiteral tree) { + defaultAction.run(); } public void visitTypeIdent(JCPrimitiveTypeTree tree) { + defaultAction.run(); } public void visitTypeArray(JCArrayTypeTree tree) { + defaultAction.run(); scan(tree.elemtype); } public void visitTypeApply(JCTypeApply tree) { + defaultAction.run(); scan(tree.clazz); scan(tree.arguments); } public void visitTypeUnion(JCTypeUnion tree) { + defaultAction.run(); scan(tree.alternatives); } public void visitTypeIntersection(JCTypeIntersection tree) { + defaultAction.run(); scan(tree.bounds); } public void visitTypeParameter(JCTypeParameter tree) { + defaultAction.run(); scan(tree.annotations); scan(tree.bounds); } @Override public void visitWildcard(JCWildcard tree) { + defaultAction.run(); scan(tree.kind); if (tree.inner != null) scan(tree.inner); @@ -347,26 +404,32 @@ @Override public void visitTypeBoundKind(TypeBoundKind that) { + defaultAction.run(); } public void visitModifiers(JCModifiers tree) { + defaultAction.run(); scan(tree.annotations); } public void visitAnnotation(JCAnnotation tree) { + defaultAction.run(); scan(tree.annotationType); scan(tree.args); } public void visitAnnotatedType(JCAnnotatedType tree) { + defaultAction.run(); scan(tree.annotations); scan(tree.underlyingType); } public void visitErroneous(JCErroneous tree) { + defaultAction.run(); } public void visitLetExpr(LetExpr tree) { + defaultAction.run(); scan(tree.defs); scan(tree.expr); } >> >> Bernard >> >> >> $ diff -u >> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 >> >> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >> --- >> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 >> 2018-03-21 12:51:14.338021489 +0100 >> +++ >> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >> 2018-03-21 12:51:56.209495093 +0100 >> @@ -113,6 +113,8 @@ >> /** force serializable representation, for stress testing **/ >> private final boolean forceSerializable; >> >> + private final boolean debug; >> + >> /** Flag for alternate metafactories indicating the lambda object >> is intended to be serializable */ >> public static final int FLAG_SERIALIZABLE = 1 << 0; >> >> @@ -149,6 +151,7 @@ >> dumpLambdaToMethodStats = >> options.isSet("debug.dumpLambdaToMethodStats"); >> attr = Attr.instance(context); >> forceSerializable = options.isSet("forceSerializable"); >> + debug = options.isSet(Option.G) || >> options.isSet(Option.G_CUSTOM, "source"); >> } >> // >> >> @@ -365,7 +368,7 @@ >> lambdaDecl.body = translate(makeLambdaBody(tree, lambdaDecl)); >> >> boolean dedupe = false; >> - if (!localContext.isSerializable()) { >> + if (!localContext.isSerializable() && !debug) { >> DedupedLambda dedupedLambda = new >> DedupedLambda(lambdaDecl.sym, lambdaDecl.body); >> DedupedLambda existing = >> kInfo.dedupedLambdas.putIfAbsent(dedupedLambda, dedupedLambda); >> if (existing != null) { > > From brian.goetz at oracle.com Wed Mar 21 19:19:38 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 21 Mar 2018 15:19:38 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> <2375f941-2ee2-4d54-3914-72f8c1a6a679@oracle.com> Message-ID: I think incremental optimizations like this (which add complexity, and worse, beget more incremental optimizations) should be justified by data that shows that these cases are common enough to warrant such a thing. On 3/21/2018 3:00 PM, B. Blaser wrote: > I note also that we may want to skip empty blocs '{}' and statements ';'. > The following examples would then produce only 2 lambda methods > instead of currently 8: > > Runnable r = () -> {}; > r = () -> { ; }; > r = () -> { {} }; > r = () -> { { ; } }; > > r = () -> { int i = 0; if (i==0) ; }; > r = () -> { int i = 0; if (i==0) {} }; > r = () -> { int i = 0; if (i==0) { ; } }; > r = () -> { int i = 0; if (i==0) { {} } }; > > I had to write a 'NoOpFinder' class (does something like that already > exist?) and I added a default action in 'TreeScanner' for that (see > below). > > Does this look reasonable (I did only some quick tests)? From bsrbnd at gmail.com Wed Mar 21 20:15:03 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 21 Mar 2018 21:15:03 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> <2375f941-2ee2-4d54-3914-72f8c1a6a679@oracle.com> Message-ID: I'm not very imaginative but for example: java.util.function.BiFunction f = (s,c) -> { Integer i=0; for (; i { Integer i=0; for (; i wrote: > I think incremental optimizations like this (which add complexity, and > worse, beget more incremental optimizations) should be justified by data > that shows that these cases are common enough to warrant such a thing. > > > On 3/21/2018 3:00 PM, B. Blaser wrote: >> >> I note also that we may want to skip empty blocs '{}' and statements ';'. >> The following examples would then produce only 2 lambda methods >> instead of currently 8: >> >> Runnable r = () -> {}; >> r = () -> { ; }; >> r = () -> { {} }; >> r = () -> { { ; } }; >> >> r = () -> { int i = 0; if (i==0) ; }; >> r = () -> { int i = 0; if (i==0) {} }; >> r = () -> { int i = 0; if (i==0) { ; } }; >> r = () -> { int i = 0; if (i==0) { {} } }; >> >> I had to write a 'NoOpFinder' class (does something like that already >> exist?) and I added a default action in 'TreeScanner' for that (see >> below). >> >> Does this look reasonable (I did only some quick tests)? > > From maurizio.cimadamore at oracle.com Wed Mar 21 21:09:03 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 21 Mar 2018 21:09:03 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> <2375f941-2ee2-4d54-3914-72f8c1a6a679@oracle.com> Message-ID: I think the way I'd like this to be addressed (not now, though) is to enhance the tree differ to be smarter about certain edge cases: * number of enclosing parens - e.g. (expr) === (((expr))) * number of enclosing braces - e.g. { statement } === {{{ statement }}} * skipping semicolons - e.g. { statement ; statement } === { statement ;;;;;; statement } There are probably many more of this kind - so I would not like to do something ad-hoc each time. Maurizio On 21/03/18 19:00, B. Blaser wrote: > On 21 March 2018 at 13:48, Maurizio Cimadamore > wrote: >> >> On 21/03/18 12:24, B. Blaser wrote: >>> On 19 March 2018 at 22:31, Maurizio Cimadamore >>> wrote: >>>> [...] >>>> >>>> Also, a side note: we have learned during condy-folding that rewriting >>>> trees >>>> is not always the right approach - if you are compiling with -g, you >>>> might >>>> need the trees to remain in place (as you want the compiler to perform >>>> less >>>> aggressive folding) - so I'm not sure the compiler will always be able to >>>> rely on constants to be rewritten in the constant folding phase. >>>> >>>> Maurizio >>> This makes me think we could disable dedup with '-g' to preserve full >>> debug-ability (as next), is this partially what you meant? >> Yes and no, but I agree that it's better to conservatively disable >> deduplication with -g. >> >> Maurizio > I note also that we may want to skip empty blocs '{}' and statements ';'. > The following examples would then produce only 2 lambda methods > instead of currently 8: > > Runnable r = () -> {}; > r = () -> { ; }; > r = () -> { {} }; > r = () -> { { ; } }; > > r = () -> { int i = 0; if (i==0) ; }; > r = () -> { int i = 0; if (i==0) {} }; > r = () -> { int i = 0; if (i==0) { ; } }; > r = () -> { int i = 0; if (i==0) { {} } }; > > I had to write a 'NoOpFinder' class (does something like that already > exist?) and I added a default action in 'TreeScanner' for that (see > below). > > Does this look reasonable (I did only some quick tests)? > > Bernard > > $ diff -u src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java > --- src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 > 2018-03-19 14:59:49.351288750 +0100 > +++ src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java > 2018-03-21 19:04:46.282040040 +0100 > @@ -94,6 +94,8 @@ > import java.util.Iterator; > import java.util.Objects; > > +import static com.sun.tools.javac.comp.NoOpFinder.isNoOp; > + > /** A visitor that compares two lambda bodies for structural equality. */ > public class TreeDiffer extends TreeScanner { > > @@ -109,8 +111,9 @@ > private boolean result; > > public boolean scan(JCTree tree, JCTree parameter) { > - if (tree == null || parameter == null) { > - return tree == null && parameter == null; > + boolean nop1 = isNoOp(tree), nop2 = isNoOp(parameter); > + if (nop1 || nop2) { > + return nop1 && nop2; > } > tree = TreeInfo.skipParens(tree); > parameter = TreeInfo.skipParens(parameter); > @@ -172,6 +175,10 @@ > result = index == otherIndex; > return; > } > + else { > + result = symbol.name == otherSymbol.name; > + return; > + } > } > result = tree.sym == that.sym; > } > > $ diff -u src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java > --- src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 > 2018-03-19 15:00:02.743120393 +0100 > +++ src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java > 2018-03-21 19:04:25.454301881 +0100 > @@ -61,7 +61,7 @@ > > @Override > public void scan(JCTree tree) { > - if (tree == null) { > + if (tree == null || NoOpFinder.isNoOp(tree)) { > return; > } > tree = TreeInfo.skipParens(tree); > @@ -84,6 +84,10 @@ > hash(idx); > return; > } > + else { > + hash(tree.sym.name); > + return; > + } > } > hash(sym); > } > > > diff --git a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java > b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java > new file mode 100644 > --- /dev/null > +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java > @@ -0,0 +1,29 @@ > +package com.sun.tools.javac.comp; > + > +import com.sun.tools.javac.tree.JCTree; > +import com.sun.tools.javac.tree.TreeScanner; > + > +public class NoOpFinder extends TreeScanner { > + > + public static boolean isNoOp(JCTree tree) { > + NoOpFinder nop = new NoOpFinder(); > + if (tree != null) > + nop.scan(tree); > + return nop.result; > + } > + > + private boolean result = true; > + > + private NoOpFinder() { > + defaultAction = () -> { result = false; }; > + } > + > + @Override > + public void visitSkip(JCTree.JCSkip tree) { > + } > + > + @Override > + public void visitBlock(JCTree.JCBlock tree) { > + scan(tree.stats); > + } > +} > diff --git a/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java > b/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java > --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java > +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java > @@ -43,6 +43,8 @@ > */ > public class TreeScanner extends Visitor { > > + protected Runnable defaultAction = () -> {}; > + > /** Visitor method: Scan a single node. > */ > public void scan(JCTree tree) { > @@ -63,16 +65,19 @@ > ****************************************************************************/ > > public void visitTopLevel(JCCompilationUnit tree) { > + defaultAction.run(); > scan(tree.defs); > } > > public void visitPackageDef(JCPackageDecl tree) { > + defaultAction.run(); > scan(tree.annotations); > scan(tree.pid); > } > > @Override > public void visitModuleDef(JCModuleDecl tree) { > + defaultAction.run(); > scan(tree.mods); > scan(tree.qualId); > scan(tree.directives); > @@ -80,37 +85,44 @@ > > @Override > public void visitExports(JCExports tree) { > + defaultAction.run(); > scan(tree.qualid); > scan(tree.moduleNames); > } > > @Override > public void visitOpens(JCOpens tree) { > + defaultAction.run(); > scan(tree.qualid); > scan(tree.moduleNames); > } > > @Override > public void visitProvides(JCProvides tree) { > + defaultAction.run(); > scan(tree.serviceName); > scan(tree.implNames); > } > > @Override > public void visitRequires(JCRequires tree) { > + defaultAction.run(); > scan(tree.moduleName); > } > > @Override > public void visitUses(JCUses tree) { > + defaultAction.run(); > scan(tree.qualid); > } > > public void visitImport(JCImport tree) { > + defaultAction.run(); > scan(tree.qualid); > } > > public void visitClassDef(JCClassDecl tree) { > + defaultAction.run(); > scan(tree.mods); > scan(tree.typarams); > scan(tree.extending); > @@ -119,6 +131,7 @@ > } > > public void visitMethodDef(JCMethodDecl tree) { > + defaultAction.run(); > scan(tree.mods); > scan(tree.restype); > scan(tree.typarams); > @@ -130,6 +143,7 @@ > } > > public void visitVarDef(JCVariableDecl tree) { > + defaultAction.run(); > scan(tree.mods); > scan(tree.vartype); > scan(tree.nameexpr); > @@ -137,23 +151,28 @@ > } > > public void visitSkip(JCSkip tree) { > + defaultAction.run(); > } > > public void visitBlock(JCBlock tree) { > + defaultAction.run(); > scan(tree.stats); > } > > public void visitDoLoop(JCDoWhileLoop tree) { > + defaultAction.run(); > scan(tree.body); > scan(tree.cond); > } > > public void visitWhileLoop(JCWhileLoop tree) { > + defaultAction.run(); > scan(tree.cond); > scan(tree.body); > } > > public void visitForLoop(JCForLoop tree) { > + defaultAction.run(); > scan(tree.init); > scan(tree.cond); > scan(tree.step); > @@ -161,31 +180,37 @@ > } > > public void visitForeachLoop(JCEnhancedForLoop tree) { > + defaultAction.run(); > scan(tree.var); > scan(tree.expr); > scan(tree.body); > } > > public void visitLabelled(JCLabeledStatement tree) { > + defaultAction.run(); > scan(tree.body); > } > > public void visitSwitch(JCSwitch tree) { > + defaultAction.run(); > scan(tree.selector); > scan(tree.cases); > } > > public void visitCase(JCCase tree) { > + defaultAction.run(); > scan(tree.pat); > scan(tree.stats); > } > > public void visitSynchronized(JCSynchronized tree) { > + defaultAction.run(); > scan(tree.lock); > scan(tree.body); > } > > public void visitTry(JCTry tree) { > + defaultAction.run(); > scan(tree.resources); > scan(tree.body); > scan(tree.catchers); > @@ -193,52 +218,63 @@ > } > > public void visitCatch(JCCatch tree) { > + defaultAction.run(); > scan(tree.param); > scan(tree.body); > } > > public void visitConditional(JCConditional tree) { > + defaultAction.run(); > scan(tree.cond); > scan(tree.truepart); > scan(tree.falsepart); > } > > public void visitIf(JCIf tree) { > + defaultAction.run(); > scan(tree.cond); > scan(tree.thenpart); > scan(tree.elsepart); > } > > public void visitExec(JCExpressionStatement tree) { > + defaultAction.run(); > scan(tree.expr); > } > > public void visitBreak(JCBreak tree) { > + defaultAction.run(); > } > > public void visitContinue(JCContinue tree) { > + defaultAction.run(); > } > > public void visitReturn(JCReturn tree) { > + defaultAction.run(); > scan(tree.expr); > } > > public void visitThrow(JCThrow tree) { > + defaultAction.run(); > scan(tree.expr); > } > > public void visitAssert(JCAssert tree) { > + defaultAction.run(); > scan(tree.cond); > scan(tree.detail); > } > > public void visitApply(JCMethodInvocation tree) { > + defaultAction.run(); > scan(tree.typeargs); > scan(tree.meth); > scan(tree.args); > } > > public void visitNewClass(JCNewClass tree) { > + defaultAction.run(); > scan(tree.encl); > scan(tree.typeargs); > scan(tree.clazz); > @@ -247,6 +283,7 @@ > } > > public void visitNewArray(JCNewArray tree) { > + defaultAction.run(); > scan(tree.annotations); > scan(tree.elemtype); > scan(tree.dims); > @@ -256,90 +293,110 @@ > } > > public void visitLambda(JCLambda tree) { > + defaultAction.run(); > scan(tree.body); > scan(tree.params); > } > > public void visitParens(JCParens tree) { > + defaultAction.run(); > scan(tree.expr); > } > > public void visitAssign(JCAssign tree) { > + defaultAction.run(); > scan(tree.lhs); > scan(tree.rhs); > } > > public void visitAssignop(JCAssignOp tree) { > + defaultAction.run(); > scan(tree.lhs); > scan(tree.rhs); > } > > public void visitUnary(JCUnary tree) { > + defaultAction.run(); > scan(tree.arg); > } > > public void visitBinary(JCBinary tree) { > + defaultAction.run(); > scan(tree.lhs); > scan(tree.rhs); > } > > public void visitTypeCast(JCTypeCast tree) { > + defaultAction.run(); > scan(tree.clazz); > scan(tree.expr); > } > > public void visitTypeTest(JCInstanceOf tree) { > + defaultAction.run(); > scan(tree.expr); > scan(tree.clazz); > } > > public void visitIndexed(JCArrayAccess tree) { > + defaultAction.run(); > scan(tree.indexed); > scan(tree.index); > } > > public void visitSelect(JCFieldAccess tree) { > + defaultAction.run(); > scan(tree.selected); > } > > public void visitReference(JCMemberReference tree) { > + defaultAction.run(); > scan(tree.expr); > scan(tree.typeargs); > } > > public void visitIdent(JCIdent tree) { > + defaultAction.run(); > } > > public void visitLiteral(JCLiteral tree) { > + defaultAction.run(); > } > > public void visitTypeIdent(JCPrimitiveTypeTree tree) { > + defaultAction.run(); > } > > public void visitTypeArray(JCArrayTypeTree tree) { > + defaultAction.run(); > scan(tree.elemtype); > } > > public void visitTypeApply(JCTypeApply tree) { > + defaultAction.run(); > scan(tree.clazz); > scan(tree.arguments); > } > > public void visitTypeUnion(JCTypeUnion tree) { > + defaultAction.run(); > scan(tree.alternatives); > } > > public void visitTypeIntersection(JCTypeIntersection tree) { > + defaultAction.run(); > scan(tree.bounds); > } > > public void visitTypeParameter(JCTypeParameter tree) { > + defaultAction.run(); > scan(tree.annotations); > scan(tree.bounds); > } > > @Override > public void visitWildcard(JCWildcard tree) { > + defaultAction.run(); > scan(tree.kind); > if (tree.inner != null) > scan(tree.inner); > @@ -347,26 +404,32 @@ > > @Override > public void visitTypeBoundKind(TypeBoundKind that) { > + defaultAction.run(); > } > > public void visitModifiers(JCModifiers tree) { > + defaultAction.run(); > scan(tree.annotations); > } > > public void visitAnnotation(JCAnnotation tree) { > + defaultAction.run(); > scan(tree.annotationType); > scan(tree.args); > } > > public void visitAnnotatedType(JCAnnotatedType tree) { > + defaultAction.run(); > scan(tree.annotations); > scan(tree.underlyingType); > } > > public void visitErroneous(JCErroneous tree) { > + defaultAction.run(); > } > > public void visitLetExpr(LetExpr tree) { > + defaultAction.run(); > scan(tree.defs); > scan(tree.expr); > } > >>> Bernard >>> >>> >>> $ diff -u >>> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 >>> >>> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >>> --- >>> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 >>> 2018-03-21 12:51:14.338021489 +0100 >>> +++ >>> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >>> 2018-03-21 12:51:56.209495093 +0100 >>> @@ -113,6 +113,8 @@ >>> /** force serializable representation, for stress testing **/ >>> private final boolean forceSerializable; >>> >>> + private final boolean debug; >>> + >>> /** Flag for alternate metafactories indicating the lambda object >>> is intended to be serializable */ >>> public static final int FLAG_SERIALIZABLE = 1 << 0; >>> >>> @@ -149,6 +151,7 @@ >>> dumpLambdaToMethodStats = >>> options.isSet("debug.dumpLambdaToMethodStats"); >>> attr = Attr.instance(context); >>> forceSerializable = options.isSet("forceSerializable"); >>> + debug = options.isSet(Option.G) || >>> options.isSet(Option.G_CUSTOM, "source"); >>> } >>> // >>> >>> @@ -365,7 +368,7 @@ >>> lambdaDecl.body = translate(makeLambdaBody(tree, lambdaDecl)); >>> >>> boolean dedupe = false; >>> - if (!localContext.isSerializable()) { >>> + if (!localContext.isSerializable() && !debug) { >>> DedupedLambda dedupedLambda = new >>> DedupedLambda(lambdaDecl.sym, lambdaDecl.body); >>> DedupedLambda existing = >>> kInfo.dedupedLambdas.putIfAbsent(dedupedLambda, dedupedLambda); >>> if (existing != null) { >> From kevinb at google.com Wed Mar 21 21:59:41 2018 From: kevinb at google.com (Kevin Bourrillion) Date: Wed, 21 Mar 2018 14:59:41 -0700 Subject: Records -- current status In-Reply-To: References: Message-ID: On Tue, Mar 20, 2018 at 6:57 AM, Federico Peralta Schaffner < federico.peralta at gmail.com> wrote: Besides this, I think that copy constructors are widely used (much more > than Cloneable and .clone()), which makes me think about 2 things: > imho, copy constructors are useful to support heterogeneous copies (e.g. `new HashMap<>(aTreeMap)`), which we have no need of, and that's about it. Aside from that, instance methods are the clearly conventional way to do things like this, and we should call it "clone" because that's the convention. That said, I'm getting more worried about the idea of the clone method on records. A regular Object doesn't automatically get clone() unless it asks for it, because we don't know how deep it expects that clone to be. By making it opt in, we have more reason to assume the user probably thought about that. What really makes records different here? We could continue to require at least an explicit `implements Cloneable`? > - Additional instance fields. These are a much bigger concern. While the > primary arguments against them are of the "slippery slope" variety, I still > have deep misgivings about supporting unrestricted non-principal instance > fields, and I also haven't found a reasonable set of restrictions that > makes this less risky. I'd like to keep looking for a better story here, > before just caving on this, as I worry doing so will end up biting us in > the back. > > I think that additional fields can be very dangerous for the little benefit > they might bring to records. In fact, I don't see a real value for them. > Suppose you have: > > record Name(String first, String last) { > public firstAndLast() { return first + " " + last; } > } > > Why caching the result of first + " " + last in an additional firstAndLast > instance field? For records, it should be enough to expose derived state > via accessor methods. If you want to cache some value (because it's > expensive to calculate), you can always resort to classes, or could maybe > use another record. For the example above, you could have: > > record FullName(Name name, String fullName) { > @Override > public FullName(Name name, String fullName) { > if (!name.firstAndLast().equals(fullName)) { > throw new IllegalArgumentException(); > } > default.this(name, name.firstAndLast()); > } > > public FullName(Name name) { > this(name, name.firstAndLast()); // delegates to the overriden > default constructor > } > } > I would have described this as an example of the kind of unfortunate complexity users would go to without support for extra derived fields; i.e., as an argument why we *should* allow them. And it requires eager initialization, and it unnecessarily checks fullName in equals() and hashCode(). I continue to seek an understanding of what our basis for forbidding these fields really is. They seem harmless to me. Some things make sense to lazily initialize. But we need to support the uniform access principle... Really? Do we? Why? > We don't *need* to, but it's still nice to. I don't want developers to have to keep track of whether they use the parens or not on this class vs. that class. Just always use them. The fact that one *can* make terrible design decisions and do massive work in a method that looks like an accessor is beside the point I think. -- Kevin Bourrillion | Java Librarian | Google, Inc. | kevinb at google.com From scolebourne at joda.org Thu Mar 22 08:32:40 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 22 Mar 2018 08:32:40 +0000 Subject: Raw String Literals In-Reply-To: <10B03F59-E391-4F94-B4FE-FD76537D72E8@oracle.com> References: <10B03F59-E391-4F94-B4FE-FD76537D72E8@oracle.com> Message-ID: On 21 March 2018 at 15:10, Jim Laskey wrote: > We think things have "settled in" with respect to Raw String Literals language changes and library support. If all things fall in place, we will probably move http://openjdk.java.net/jeps/326 forward soon. What does this print? public void varargs(String... strs) { System.out.println(Stream.of(strs).collect(joining())); } varargs(`Hello`, ``, ` World! `, ``); . scroll down for the answer... . . Answer: "Hello, `World!`, " (notice the extra commas, because there are only 2 arguments, not 4, due to no raw string literals) (thanks to Kevin B for the puzzler) As I have stated elsewhere, I believe the trade offs in the raw string literal proposal are poor for Java. Not allowing an empty string and not allowing strings to start or end with backticks is surprising, and I believe the puzzler demonstrates the nasty effects of excluding an empty string literal (as also demonstrated here http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-March/052052.html). I do understand the appeal of being able to self-embed (the "stress test"), I just happen to think it is the wrong trade off. Fortunately, I believe there is a way for Java to have both empty raw string literals and full embedding. Simply disallow double backtick as a delimiter. `foo` = "foo" (single backtick) `` = "" (empty string) ``foo`` = compile error ```foo``` - "foo" (triple backtick) `````` - compile error (no empty string here) `````foo````` - "foo" (five backticks) There is a major specification that separates one backtick from three backticks - Markdown: http://spec.commonmark.org/0.27/#code-spans http://spec.commonmark.org/0.27/#fenced-code-blocks As such, I think this would be readily accepted by developers, who are often familiar with Markdown. Were this approach to be adopted, I believe single backtick literals should not allow new lines. This would be a good simplification, and probably help IDE error recovery in many cases. The 3+ backtick literals would behave as per the current Oracle proposal, but with the benefit that when embedding just one or two backticks developers will not need to see or learn the rule about unlimited backticks. Thinks like regex would tend to use single backticks, while things like embedding XML or Javascript would use 3+ backticks. In summary, not allowing an empty raw string is going to result in nasty puzzlers, and IMO unlimited delimiters by themselves is not in the spirit of Java. Following Markdown's approach of separation between single and 3+ backticks would provide considerable benefits to Java. Stephen From bsrbnd at gmail.com Thu Mar 22 18:29:06 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Thu, 22 Mar 2018 19:29:06 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> <2375f941-2ee2-4d54-3914-72f8c1a6a679@oracle.com> Message-ID: You're right but I think all no-ops might also be treated as equivalent ( example: for(;;) ; === for(;;) {} ) without being afraid of added complexity. The default action in 'NoOpFinder' could simply throw some exception instead of visiting the full tree and about 99% of 'isNoOp()' invocations would be treated in constant time... Bernard On 21 March 2018 at 22:09, Maurizio Cimadamore wrote: > I think the way I'd like this to be addressed (not now, though) is to > enhance the tree differ to be smarter about certain edge cases: > > * number of enclosing parens - e.g. (expr) === (((expr))) > * number of enclosing braces - e.g. { statement } === {{{ statement }}} > * skipping semicolons - e.g. { statement ; statement } === { statement > ;;;;;; statement } > > There are probably many more of this kind - so I would not like to do > something ad-hoc each time. > > Maurizio > > > > On 21/03/18 19:00, B. Blaser wrote: >> >> On 21 March 2018 at 13:48, Maurizio Cimadamore >> wrote: >>> >>> >>> On 21/03/18 12:24, B. Blaser wrote: >>>> >>>> On 19 March 2018 at 22:31, Maurizio Cimadamore >>>> wrote: >>>>> >>>>> [...] >>>>> >>>>> Also, a side note: we have learned during condy-folding that rewriting >>>>> trees >>>>> is not always the right approach - if you are compiling with -g, you >>>>> might >>>>> need the trees to remain in place (as you want the compiler to perform >>>>> less >>>>> aggressive folding) - so I'm not sure the compiler will always be able >>>>> to >>>>> rely on constants to be rewritten in the constant folding phase. >>>>> >>>>> Maurizio >>>> >>>> This makes me think we could disable dedup with '-g' to preserve full >>>> debug-ability (as next), is this partially what you meant? >>> >>> Yes and no, but I agree that it's better to conservatively disable >>> deduplication with -g. >>> >>> Maurizio >> >> I note also that we may want to skip empty blocs '{}' and statements ';'. >> The following examples would then produce only 2 lambda methods >> instead of currently 8: >> >> Runnable r = () -> {}; >> r = () -> { ; }; >> r = () -> { {} }; >> r = () -> { { ; } }; >> >> r = () -> { int i = 0; if (i==0) ; }; >> r = () -> { int i = 0; if (i==0) {} }; >> r = () -> { int i = 0; if (i==0) { ; } }; >> r = () -> { int i = 0; if (i==0) { {} } }; >> >> I had to write a 'NoOpFinder' class (does something like that already >> exist?) and I added a default action in 'TreeScanner' for that (see >> below). >> >> Does this look reasonable (I did only some quick tests)? >> >> Bernard >> >> $ diff -u >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java >> --- >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 >> 2018-03-19 14:59:49.351288750 +0100 >> +++ >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java >> 2018-03-21 19:04:46.282040040 +0100 >> @@ -94,6 +94,8 @@ >> import java.util.Iterator; >> import java.util.Objects; >> >> +import static com.sun.tools.javac.comp.NoOpFinder.isNoOp; >> + >> /** A visitor that compares two lambda bodies for structural equality. >> */ >> public class TreeDiffer extends TreeScanner { >> >> @@ -109,8 +111,9 @@ >> private boolean result; >> >> public boolean scan(JCTree tree, JCTree parameter) { >> - if (tree == null || parameter == null) { >> - return tree == null && parameter == null; >> + boolean nop1 = isNoOp(tree), nop2 = isNoOp(parameter); >> + if (nop1 || nop2) { >> + return nop1 && nop2; >> } >> tree = TreeInfo.skipParens(tree); >> parameter = TreeInfo.skipParens(parameter); >> @@ -172,6 +175,10 @@ >> result = index == otherIndex; >> return; >> } >> + else { >> + result = symbol.name == otherSymbol.name; >> + return; >> + } >> } >> result = tree.sym == that.sym; >> } >> >> $ diff -u >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java >> --- >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 >> 2018-03-19 15:00:02.743120393 +0100 >> +++ >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java >> 2018-03-21 19:04:25.454301881 +0100 >> @@ -61,7 +61,7 @@ >> >> @Override >> public void scan(JCTree tree) { >> - if (tree == null) { >> + if (tree == null || NoOpFinder.isNoOp(tree)) { >> return; >> } >> tree = TreeInfo.skipParens(tree); >> @@ -84,6 +84,10 @@ >> hash(idx); >> return; >> } >> + else { >> + hash(tree.sym.name); >> + return; >> + } >> } >> hash(sym); >> } >> >> >> diff --git >> a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java >> b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java >> new file mode 100644 >> --- /dev/null >> +++ >> b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java >> @@ -0,0 +1,29 @@ >> +package com.sun.tools.javac.comp; >> + >> +import com.sun.tools.javac.tree.JCTree; >> +import com.sun.tools.javac.tree.TreeScanner; >> + >> +public class NoOpFinder extends TreeScanner { >> + >> + public static boolean isNoOp(JCTree tree) { >> + NoOpFinder nop = new NoOpFinder(); >> + if (tree != null) >> + nop.scan(tree); >> + return nop.result; >> + } >> + >> + private boolean result = true; >> + >> + private NoOpFinder() { >> + defaultAction = () -> { result = false; }; >> + } >> + >> + @Override >> + public void visitSkip(JCTree.JCSkip tree) { >> + } >> + >> + @Override >> + public void visitBlock(JCTree.JCBlock tree) { >> + scan(tree.stats); >> + } >> +} >> diff --git >> a/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java >> b/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java >> --- >> a/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java >> +++ >> b/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java >> @@ -43,6 +43,8 @@ >> */ >> public class TreeScanner extends Visitor { >> >> + protected Runnable defaultAction = () -> {}; >> + >> /** Visitor method: Scan a single node. >> */ >> public void scan(JCTree tree) { >> @@ -63,16 +65,19 @@ >> >> ****************************************************************************/ >> >> public void visitTopLevel(JCCompilationUnit tree) { >> + defaultAction.run(); >> scan(tree.defs); >> } >> >> public void visitPackageDef(JCPackageDecl tree) { >> + defaultAction.run(); >> scan(tree.annotations); >> scan(tree.pid); >> } >> >> @Override >> public void visitModuleDef(JCModuleDecl tree) { >> + defaultAction.run(); >> scan(tree.mods); >> scan(tree.qualId); >> scan(tree.directives); >> @@ -80,37 +85,44 @@ >> >> @Override >> public void visitExports(JCExports tree) { >> + defaultAction.run(); >> scan(tree.qualid); >> scan(tree.moduleNames); >> } >> >> @Override >> public void visitOpens(JCOpens tree) { >> + defaultAction.run(); >> scan(tree.qualid); >> scan(tree.moduleNames); >> } >> >> @Override >> public void visitProvides(JCProvides tree) { >> + defaultAction.run(); >> scan(tree.serviceName); >> scan(tree.implNames); >> } >> >> @Override >> public void visitRequires(JCRequires tree) { >> + defaultAction.run(); >> scan(tree.moduleName); >> } >> >> @Override >> public void visitUses(JCUses tree) { >> + defaultAction.run(); >> scan(tree.qualid); >> } >> >> public void visitImport(JCImport tree) { >> + defaultAction.run(); >> scan(tree.qualid); >> } >> >> public void visitClassDef(JCClassDecl tree) { >> + defaultAction.run(); >> scan(tree.mods); >> scan(tree.typarams); >> scan(tree.extending); >> @@ -119,6 +131,7 @@ >> } >> >> public void visitMethodDef(JCMethodDecl tree) { >> + defaultAction.run(); >> scan(tree.mods); >> scan(tree.restype); >> scan(tree.typarams); >> @@ -130,6 +143,7 @@ >> } >> >> public void visitVarDef(JCVariableDecl tree) { >> + defaultAction.run(); >> scan(tree.mods); >> scan(tree.vartype); >> scan(tree.nameexpr); >> @@ -137,23 +151,28 @@ >> } >> >> public void visitSkip(JCSkip tree) { >> + defaultAction.run(); >> } >> >> public void visitBlock(JCBlock tree) { >> + defaultAction.run(); >> scan(tree.stats); >> } >> >> public void visitDoLoop(JCDoWhileLoop tree) { >> + defaultAction.run(); >> scan(tree.body); >> scan(tree.cond); >> } >> >> public void visitWhileLoop(JCWhileLoop tree) { >> + defaultAction.run(); >> scan(tree.cond); >> scan(tree.body); >> } >> >> public void visitForLoop(JCForLoop tree) { >> + defaultAction.run(); >> scan(tree.init); >> scan(tree.cond); >> scan(tree.step); >> @@ -161,31 +180,37 @@ >> } >> >> public void visitForeachLoop(JCEnhancedForLoop tree) { >> + defaultAction.run(); >> scan(tree.var); >> scan(tree.expr); >> scan(tree.body); >> } >> >> public void visitLabelled(JCLabeledStatement tree) { >> + defaultAction.run(); >> scan(tree.body); >> } >> >> public void visitSwitch(JCSwitch tree) { >> + defaultAction.run(); >> scan(tree.selector); >> scan(tree.cases); >> } >> >> public void visitCase(JCCase tree) { >> + defaultAction.run(); >> scan(tree.pat); >> scan(tree.stats); >> } >> >> public void visitSynchronized(JCSynchronized tree) { >> + defaultAction.run(); >> scan(tree.lock); >> scan(tree.body); >> } >> >> public void visitTry(JCTry tree) { >> + defaultAction.run(); >> scan(tree.resources); >> scan(tree.body); >> scan(tree.catchers); >> @@ -193,52 +218,63 @@ >> } >> >> public void visitCatch(JCCatch tree) { >> + defaultAction.run(); >> scan(tree.param); >> scan(tree.body); >> } >> >> public void visitConditional(JCConditional tree) { >> + defaultAction.run(); >> scan(tree.cond); >> scan(tree.truepart); >> scan(tree.falsepart); >> } >> >> public void visitIf(JCIf tree) { >> + defaultAction.run(); >> scan(tree.cond); >> scan(tree.thenpart); >> scan(tree.elsepart); >> } >> >> public void visitExec(JCExpressionStatement tree) { >> + defaultAction.run(); >> scan(tree.expr); >> } >> >> public void visitBreak(JCBreak tree) { >> + defaultAction.run(); >> } >> >> public void visitContinue(JCContinue tree) { >> + defaultAction.run(); >> } >> >> public void visitReturn(JCReturn tree) { >> + defaultAction.run(); >> scan(tree.expr); >> } >> >> public void visitThrow(JCThrow tree) { >> + defaultAction.run(); >> scan(tree.expr); >> } >> >> public void visitAssert(JCAssert tree) { >> + defaultAction.run(); >> scan(tree.cond); >> scan(tree.detail); >> } >> >> public void visitApply(JCMethodInvocation tree) { >> + defaultAction.run(); >> scan(tree.typeargs); >> scan(tree.meth); >> scan(tree.args); >> } >> >> public void visitNewClass(JCNewClass tree) { >> + defaultAction.run(); >> scan(tree.encl); >> scan(tree.typeargs); >> scan(tree.clazz); >> @@ -247,6 +283,7 @@ >> } >> >> public void visitNewArray(JCNewArray tree) { >> + defaultAction.run(); >> scan(tree.annotations); >> scan(tree.elemtype); >> scan(tree.dims); >> @@ -256,90 +293,110 @@ >> } >> >> public void visitLambda(JCLambda tree) { >> + defaultAction.run(); >> scan(tree.body); >> scan(tree.params); >> } >> >> public void visitParens(JCParens tree) { >> + defaultAction.run(); >> scan(tree.expr); >> } >> >> public void visitAssign(JCAssign tree) { >> + defaultAction.run(); >> scan(tree.lhs); >> scan(tree.rhs); >> } >> >> public void visitAssignop(JCAssignOp tree) { >> + defaultAction.run(); >> scan(tree.lhs); >> scan(tree.rhs); >> } >> >> public void visitUnary(JCUnary tree) { >> + defaultAction.run(); >> scan(tree.arg); >> } >> >> public void visitBinary(JCBinary tree) { >> + defaultAction.run(); >> scan(tree.lhs); >> scan(tree.rhs); >> } >> >> public void visitTypeCast(JCTypeCast tree) { >> + defaultAction.run(); >> scan(tree.clazz); >> scan(tree.expr); >> } >> >> public void visitTypeTest(JCInstanceOf tree) { >> + defaultAction.run(); >> scan(tree.expr); >> scan(tree.clazz); >> } >> >> public void visitIndexed(JCArrayAccess tree) { >> + defaultAction.run(); >> scan(tree.indexed); >> scan(tree.index); >> } >> >> public void visitSelect(JCFieldAccess tree) { >> + defaultAction.run(); >> scan(tree.selected); >> } >> >> public void visitReference(JCMemberReference tree) { >> + defaultAction.run(); >> scan(tree.expr); >> scan(tree.typeargs); >> } >> >> public void visitIdent(JCIdent tree) { >> + defaultAction.run(); >> } >> >> public void visitLiteral(JCLiteral tree) { >> + defaultAction.run(); >> } >> >> public void visitTypeIdent(JCPrimitiveTypeTree tree) { >> + defaultAction.run(); >> } >> >> public void visitTypeArray(JCArrayTypeTree tree) { >> + defaultAction.run(); >> scan(tree.elemtype); >> } >> >> public void visitTypeApply(JCTypeApply tree) { >> + defaultAction.run(); >> scan(tree.clazz); >> scan(tree.arguments); >> } >> >> public void visitTypeUnion(JCTypeUnion tree) { >> + defaultAction.run(); >> scan(tree.alternatives); >> } >> >> public void visitTypeIntersection(JCTypeIntersection tree) { >> + defaultAction.run(); >> scan(tree.bounds); >> } >> >> public void visitTypeParameter(JCTypeParameter tree) { >> + defaultAction.run(); >> scan(tree.annotations); >> scan(tree.bounds); >> } >> >> @Override >> public void visitWildcard(JCWildcard tree) { >> + defaultAction.run(); >> scan(tree.kind); >> if (tree.inner != null) >> scan(tree.inner); >> @@ -347,26 +404,32 @@ >> >> @Override >> public void visitTypeBoundKind(TypeBoundKind that) { >> + defaultAction.run(); >> } >> >> public void visitModifiers(JCModifiers tree) { >> + defaultAction.run(); >> scan(tree.annotations); >> } >> >> public void visitAnnotation(JCAnnotation tree) { >> + defaultAction.run(); >> scan(tree.annotationType); >> scan(tree.args); >> } >> >> public void visitAnnotatedType(JCAnnotatedType tree) { >> + defaultAction.run(); >> scan(tree.annotations); >> scan(tree.underlyingType); >> } >> >> public void visitErroneous(JCErroneous tree) { >> + defaultAction.run(); >> } >> >> public void visitLetExpr(LetExpr tree) { >> + defaultAction.run(); >> scan(tree.defs); >> scan(tree.expr); >> } >> >>>> Bernard >>>> >>>> >>>> $ diff -u >>>> >>>> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 >>>> >>>> >>>> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >>>> --- >>>> >>>> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 >>>> 2018-03-21 12:51:14.338021489 +0100 >>>> +++ >>>> >>>> ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >>>> 2018-03-21 12:51:56.209495093 +0100 >>>> @@ -113,6 +113,8 @@ >>>> /** force serializable representation, for stress testing **/ >>>> private final boolean forceSerializable; >>>> >>>> + private final boolean debug; >>>> + >>>> /** Flag for alternate metafactories indicating the lambda object >>>> is intended to be serializable */ >>>> public static final int FLAG_SERIALIZABLE = 1 << 0; >>>> >>>> @@ -149,6 +151,7 @@ >>>> dumpLambdaToMethodStats = >>>> options.isSet("debug.dumpLambdaToMethodStats"); >>>> attr = Attr.instance(context); >>>> forceSerializable = options.isSet("forceSerializable"); >>>> + debug = options.isSet(Option.G) || >>>> options.isSet(Option.G_CUSTOM, "source"); >>>> } >>>> // >>>> >>>> @@ -365,7 +368,7 @@ >>>> lambdaDecl.body = translate(makeLambdaBody(tree, >>>> lambdaDecl)); >>>> >>>> boolean dedupe = false; >>>> - if (!localContext.isSerializable()) { >>>> + if (!localContext.isSerializable() && !debug) { >>>> DedupedLambda dedupedLambda = new >>>> DedupedLambda(lambdaDecl.sym, lambdaDecl.body); >>>> DedupedLambda existing = >>>> kInfo.dedupedLambdas.putIfAbsent(dedupedLambda, dedupedLambda); >>>> if (existing != null) { >>> >>> > From lowasser at google.com Thu Mar 22 18:40:12 2018 From: lowasser at google.com (Louis Wasserman) Date: Thu, 22 Mar 2018 18:40:12 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <2353d4a0-801f-844c-c742-68b20750ed36@oracle.com> <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> <2375f941-2ee2-4d54-3914-72f8c1a6a679@oracle.com> Message-ID: I'm with Brian: unless there's clear evidence that lambdas in the same file that are equivalent except for these kinds of differences *actually come up with regularity in real code,* it's not worth adding complexity to the tree diffing logic. I would guess that that's actively unlikely: since we're doing only intra-file duplication right now, lambdas in a given file are more likely than not to be written in the same style by the same developer. On Thu, Mar 22, 2018 at 11:29 AM B. Blaser wrote: > You're right but I think all no-ops might also be treated as > equivalent ( example: for(;;) ; === for(;;) {} ) without being afraid > of added complexity. The default action in 'NoOpFinder' could simply > throw some exception instead of visiting the full tree and about 99% > of 'isNoOp()' invocations would be treated in constant time... > > Bernard > > > On 21 March 2018 at 22:09, Maurizio Cimadamore > wrote: > > I think the way I'd like this to be addressed (not now, though) is to > > enhance the tree differ to be smarter about certain edge cases: > > > > * number of enclosing parens - e.g. (expr) === (((expr))) > > * number of enclosing braces - e.g. { statement } === {{{ statement }}} > > * skipping semicolons - e.g. { statement ; statement } === { statement > > ;;;;;; statement } > > > > There are probably many more of this kind - so I would not like to do > > something ad-hoc each time. > > > > Maurizio > > > > > > > > On 21/03/18 19:00, B. Blaser wrote: > >> > >> On 21 March 2018 at 13:48, Maurizio Cimadamore > >> wrote: > >>> > >>> > >>> On 21/03/18 12:24, B. Blaser wrote: > >>>> > >>>> On 19 March 2018 at 22:31, Maurizio Cimadamore > >>>> wrote: > >>>>> > >>>>> [...] > >>>>> > >>>>> Also, a side note: we have learned during condy-folding that > rewriting > >>>>> trees > >>>>> is not always the right approach - if you are compiling with -g, you > >>>>> might > >>>>> need the trees to remain in place (as you want the compiler to > perform > >>>>> less > >>>>> aggressive folding) - so I'm not sure the compiler will always be > able > >>>>> to > >>>>> rely on constants to be rewritten in the constant folding phase. > >>>>> > >>>>> Maurizio > >>>> > >>>> This makes me think we could disable dedup with '-g' to preserve full > >>>> debug-ability (as next), is this partially what you meant? > >>> > >>> Yes and no, but I agree that it's better to conservatively disable > >>> deduplication with -g. > >>> > >>> Maurizio > >> > >> I note also that we may want to skip empty blocs '{}' and statements > ';'. > >> The following examples would then produce only 2 lambda methods > >> instead of currently 8: > >> > >> Runnable r = () -> {}; > >> r = () -> { ; }; > >> r = () -> { {} }; > >> r = () -> { { ; } }; > >> > >> r = () -> { int i = 0; if (i==0) ; }; > >> r = () -> { int i = 0; if (i==0) {} }; > >> r = () -> { int i = 0; if (i==0) { ; } }; > >> r = () -> { int i = 0; if (i==0) { {} } }; > >> > >> I had to write a 'NoOpFinder' class (does something like that already > >> exist?) and I added a default action in 'TreeScanner' for that (see > >> below). > >> > >> Does this look reasonable (I did only some quick tests)? > >> > >> Bernard > >> > >> $ diff -u > >> > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 > >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java > >> --- > >> > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java.r03 > >> 2018-03-19 14:59:49.351288750 +0100 > >> +++ > >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java > >> 2018-03-21 19:04:46.282040040 +0100 > >> @@ -94,6 +94,8 @@ > >> import java.util.Iterator; > >> import java.util.Objects; > >> > >> +import static com.sun.tools.javac.comp.NoOpFinder.isNoOp; > >> + > >> /** A visitor that compares two lambda bodies for structural equality. > >> */ > >> public class TreeDiffer extends TreeScanner { > >> > >> @@ -109,8 +111,9 @@ > >> private boolean result; > >> > >> public boolean scan(JCTree tree, JCTree parameter) { > >> - if (tree == null || parameter == null) { > >> - return tree == null && parameter == null; > >> + boolean nop1 = isNoOp(tree), nop2 = isNoOp(parameter); > >> + if (nop1 || nop2) { > >> + return nop1 && nop2; > >> } > >> tree = TreeInfo.skipParens(tree); > >> parameter = TreeInfo.skipParens(parameter); > >> @@ -172,6 +175,10 @@ > >> result = index == otherIndex; > >> return; > >> } > >> + else { > >> + result = symbol.name == otherSymbol.name; > >> + return; > >> + } > >> } > >> result = tree.sym == that.sym; > >> } > >> > >> $ diff -u > >> > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 > >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java > >> --- > >> > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java.r03 > >> 2018-03-19 15:00:02.743120393 +0100 > >> +++ > >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java > >> 2018-03-21 19:04:25.454301881 +0100 > >> @@ -61,7 +61,7 @@ > >> > >> @Override > >> public void scan(JCTree tree) { > >> - if (tree == null) { > >> + if (tree == null || NoOpFinder.isNoOp(tree)) { > >> return; > >> } > >> tree = TreeInfo.skipParens(tree); > >> @@ -84,6 +84,10 @@ > >> hash(idx); > >> return; > >> } > >> + else { > >> + hash(tree.sym.name); > >> + return; > >> + } > >> } > >> hash(sym); > >> } > >> > >> > >> diff --git > >> > a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java > >> > b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java > >> new file mode 100644 > >> --- /dev/null > >> +++ > >> > b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/NoOpFinder.java > >> @@ -0,0 +1,29 @@ > >> +package com.sun.tools.javac.comp; > >> + > >> +import com.sun.tools.javac.tree.JCTree; > >> +import com.sun.tools.javac.tree.TreeScanner; > >> + > >> +public class NoOpFinder extends TreeScanner { > >> + > >> + public static boolean isNoOp(JCTree tree) { > >> + NoOpFinder nop = new NoOpFinder(); > >> + if (tree != null) > >> + nop.scan(tree); > >> + return nop.result; > >> + } > >> + > >> + private boolean result = true; > >> + > >> + private NoOpFinder() { > >> + defaultAction = () -> { result = false; }; > >> + } > >> + > >> + @Override > >> + public void visitSkip(JCTree.JCSkip tree) { > >> + } > >> + > >> + @Override > >> + public void visitBlock(JCTree.JCBlock tree) { > >> + scan(tree.stats); > >> + } > >> +} > >> diff --git > >> > a/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java > >> > b/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java > >> --- > >> > a/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java > >> +++ > >> > b/src/jdk.compiler/share/classes/com/sun/tools/javac/tree/TreeScanner.java > >> @@ -43,6 +43,8 @@ > >> */ > >> public class TreeScanner extends Visitor { > >> > >> + protected Runnable defaultAction = () -> {}; > >> + > >> /** Visitor method: Scan a single node. > >> */ > >> public void scan(JCTree tree) { > >> @@ -63,16 +65,19 @@ > >> > >> > ****************************************************************************/ > >> > >> public void visitTopLevel(JCCompilationUnit tree) { > >> + defaultAction.run(); > >> scan(tree.defs); > >> } > >> > >> public void visitPackageDef(JCPackageDecl tree) { > >> + defaultAction.run(); > >> scan(tree.annotations); > >> scan(tree.pid); > >> } > >> > >> @Override > >> public void visitModuleDef(JCModuleDecl tree) { > >> + defaultAction.run(); > >> scan(tree.mods); > >> scan(tree.qualId); > >> scan(tree.directives); > >> @@ -80,37 +85,44 @@ > >> > >> @Override > >> public void visitExports(JCExports tree) { > >> + defaultAction.run(); > >> scan(tree.qualid); > >> scan(tree.moduleNames); > >> } > >> > >> @Override > >> public void visitOpens(JCOpens tree) { > >> + defaultAction.run(); > >> scan(tree.qualid); > >> scan(tree.moduleNames); > >> } > >> > >> @Override > >> public void visitProvides(JCProvides tree) { > >> + defaultAction.run(); > >> scan(tree.serviceName); > >> scan(tree.implNames); > >> } > >> > >> @Override > >> public void visitRequires(JCRequires tree) { > >> + defaultAction.run(); > >> scan(tree.moduleName); > >> } > >> > >> @Override > >> public void visitUses(JCUses tree) { > >> + defaultAction.run(); > >> scan(tree.qualid); > >> } > >> > >> public void visitImport(JCImport tree) { > >> + defaultAction.run(); > >> scan(tree.qualid); > >> } > >> > >> public void visitClassDef(JCClassDecl tree) { > >> + defaultAction.run(); > >> scan(tree.mods); > >> scan(tree.typarams); > >> scan(tree.extending); > >> @@ -119,6 +131,7 @@ > >> } > >> > >> public void visitMethodDef(JCMethodDecl tree) { > >> + defaultAction.run(); > >> scan(tree.mods); > >> scan(tree.restype); > >> scan(tree.typarams); > >> @@ -130,6 +143,7 @@ > >> } > >> > >> public void visitVarDef(JCVariableDecl tree) { > >> + defaultAction.run(); > >> scan(tree.mods); > >> scan(tree.vartype); > >> scan(tree.nameexpr); > >> @@ -137,23 +151,28 @@ > >> } > >> > >> public void visitSkip(JCSkip tree) { > >> + defaultAction.run(); > >> } > >> > >> public void visitBlock(JCBlock tree) { > >> + defaultAction.run(); > >> scan(tree.stats); > >> } > >> > >> public void visitDoLoop(JCDoWhileLoop tree) { > >> + defaultAction.run(); > >> scan(tree.body); > >> scan(tree.cond); > >> } > >> > >> public void visitWhileLoop(JCWhileLoop tree) { > >> + defaultAction.run(); > >> scan(tree.cond); > >> scan(tree.body); > >> } > >> > >> public void visitForLoop(JCForLoop tree) { > >> + defaultAction.run(); > >> scan(tree.init); > >> scan(tree.cond); > >> scan(tree.step); > >> @@ -161,31 +180,37 @@ > >> } > >> > >> public void visitForeachLoop(JCEnhancedForLoop tree) { > >> + defaultAction.run(); > >> scan(tree.var); > >> scan(tree.expr); > >> scan(tree.body); > >> } > >> > >> public void visitLabelled(JCLabeledStatement tree) { > >> + defaultAction.run(); > >> scan(tree.body); > >> } > >> > >> public void visitSwitch(JCSwitch tree) { > >> + defaultAction.run(); > >> scan(tree.selector); > >> scan(tree.cases); > >> } > >> > >> public void visitCase(JCCase tree) { > >> + defaultAction.run(); > >> scan(tree.pat); > >> scan(tree.stats); > >> } > >> > >> public void visitSynchronized(JCSynchronized tree) { > >> + defaultAction.run(); > >> scan(tree.lock); > >> scan(tree.body); > >> } > >> > >> public void visitTry(JCTry tree) { > >> + defaultAction.run(); > >> scan(tree.resources); > >> scan(tree.body); > >> scan(tree.catchers); > >> @@ -193,52 +218,63 @@ > >> } > >> > >> public void visitCatch(JCCatch tree) { > >> + defaultAction.run(); > >> scan(tree.param); > >> scan(tree.body); > >> } > >> > >> public void visitConditional(JCConditional tree) { > >> + defaultAction.run(); > >> scan(tree.cond); > >> scan(tree.truepart); > >> scan(tree.falsepart); > >> } > >> > >> public void visitIf(JCIf tree) { > >> + defaultAction.run(); > >> scan(tree.cond); > >> scan(tree.thenpart); > >> scan(tree.elsepart); > >> } > >> > >> public void visitExec(JCExpressionStatement tree) { > >> + defaultAction.run(); > >> scan(tree.expr); > >> } > >> > >> public void visitBreak(JCBreak tree) { > >> + defaultAction.run(); > >> } > >> > >> public void visitContinue(JCContinue tree) { > >> + defaultAction.run(); > >> } > >> > >> public void visitReturn(JCReturn tree) { > >> + defaultAction.run(); > >> scan(tree.expr); > >> } > >> > >> public void visitThrow(JCThrow tree) { > >> + defaultAction.run(); > >> scan(tree.expr); > >> } > >> > >> public void visitAssert(JCAssert tree) { > >> + defaultAction.run(); > >> scan(tree.cond); > >> scan(tree.detail); > >> } > >> > >> public void visitApply(JCMethodInvocation tree) { > >> + defaultAction.run(); > >> scan(tree.typeargs); > >> scan(tree.meth); > >> scan(tree.args); > >> } > >> > >> public void visitNewClass(JCNewClass tree) { > >> + defaultAction.run(); > >> scan(tree.encl); > >> scan(tree.typeargs); > >> scan(tree.clazz); > >> @@ -247,6 +283,7 @@ > >> } > >> > >> public void visitNewArray(JCNewArray tree) { > >> + defaultAction.run(); > >> scan(tree.annotations); > >> scan(tree.elemtype); > >> scan(tree.dims); > >> @@ -256,90 +293,110 @@ > >> } > >> > >> public void visitLambda(JCLambda tree) { > >> + defaultAction.run(); > >> scan(tree.body); > >> scan(tree.params); > >> } > >> > >> public void visitParens(JCParens tree) { > >> + defaultAction.run(); > >> scan(tree.expr); > >> } > >> > >> public void visitAssign(JCAssign tree) { > >> + defaultAction.run(); > >> scan(tree.lhs); > >> scan(tree.rhs); > >> } > >> > >> public void visitAssignop(JCAssignOp tree) { > >> + defaultAction.run(); > >> scan(tree.lhs); > >> scan(tree.rhs); > >> } > >> > >> public void visitUnary(JCUnary tree) { > >> + defaultAction.run(); > >> scan(tree.arg); > >> } > >> > >> public void visitBinary(JCBinary tree) { > >> + defaultAction.run(); > >> scan(tree.lhs); > >> scan(tree.rhs); > >> } > >> > >> public void visitTypeCast(JCTypeCast tree) { > >> + defaultAction.run(); > >> scan(tree.clazz); > >> scan(tree.expr); > >> } > >> > >> public void visitTypeTest(JCInstanceOf tree) { > >> + defaultAction.run(); > >> scan(tree.expr); > >> scan(tree.clazz); > >> } > >> > >> public void visitIndexed(JCArrayAccess tree) { > >> + defaultAction.run(); > >> scan(tree.indexed); > >> scan(tree.index); > >> } > >> > >> public void visitSelect(JCFieldAccess tree) { > >> + defaultAction.run(); > >> scan(tree.selected); > >> } > >> > >> public void visitReference(JCMemberReference tree) { > >> + defaultAction.run(); > >> scan(tree.expr); > >> scan(tree.typeargs); > >> } > >> > >> public void visitIdent(JCIdent tree) { > >> + defaultAction.run(); > >> } > >> > >> public void visitLiteral(JCLiteral tree) { > >> + defaultAction.run(); > >> } > >> > >> public void visitTypeIdent(JCPrimitiveTypeTree tree) { > >> + defaultAction.run(); > >> } > >> > >> public void visitTypeArray(JCArrayTypeTree tree) { > >> + defaultAction.run(); > >> scan(tree.elemtype); > >> } > >> > >> public void visitTypeApply(JCTypeApply tree) { > >> + defaultAction.run(); > >> scan(tree.clazz); > >> scan(tree.arguments); > >> } > >> > >> public void visitTypeUnion(JCTypeUnion tree) { > >> + defaultAction.run(); > >> scan(tree.alternatives); > >> } > >> > >> public void visitTypeIntersection(JCTypeIntersection tree) { > >> + defaultAction.run(); > >> scan(tree.bounds); > >> } > >> > >> public void visitTypeParameter(JCTypeParameter tree) { > >> + defaultAction.run(); > >> scan(tree.annotations); > >> scan(tree.bounds); > >> } > >> > >> @Override > >> public void visitWildcard(JCWildcard tree) { > >> + defaultAction.run(); > >> scan(tree.kind); > >> if (tree.inner != null) > >> scan(tree.inner); > >> @@ -347,26 +404,32 @@ > >> > >> @Override > >> public void visitTypeBoundKind(TypeBoundKind that) { > >> + defaultAction.run(); > >> } > >> > >> public void visitModifiers(JCModifiers tree) { > >> + defaultAction.run(); > >> scan(tree.annotations); > >> } > >> > >> public void visitAnnotation(JCAnnotation tree) { > >> + defaultAction.run(); > >> scan(tree.annotationType); > >> scan(tree.args); > >> } > >> > >> public void visitAnnotatedType(JCAnnotatedType tree) { > >> + defaultAction.run(); > >> scan(tree.annotations); > >> scan(tree.underlyingType); > >> } > >> > >> public void visitErroneous(JCErroneous tree) { > >> + defaultAction.run(); > >> } > >> > >> public void visitLetExpr(LetExpr tree) { > >> + defaultAction.run(); > >> scan(tree.defs); > >> scan(tree.expr); > >> } > >> > >>>> Bernard > >>>> > >>>> > >>>> $ diff -u > >>>> > >>>> > ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 > >>>> > >>>> > >>>> > ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > >>>> --- > >>>> > >>>> > ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java.r03 > >>>> 2018-03-21 12:51:14.338021489 +0100 > >>>> +++ > >>>> > >>>> > ./src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > >>>> 2018-03-21 12:51:56.209495093 +0100 > >>>> @@ -113,6 +113,8 @@ > >>>> /** force serializable representation, for stress testing **/ > >>>> private final boolean forceSerializable; > >>>> > >>>> + private final boolean debug; > >>>> + > >>>> /** Flag for alternate metafactories indicating the lambda > object > >>>> is intended to be serializable */ > >>>> public static final int FLAG_SERIALIZABLE = 1 << 0; > >>>> > >>>> @@ -149,6 +151,7 @@ > >>>> dumpLambdaToMethodStats = > >>>> options.isSet("debug.dumpLambdaToMethodStats"); > >>>> attr = Attr.instance(context); > >>>> forceSerializable = options.isSet("forceSerializable"); > >>>> + debug = options.isSet(Option.G) || > >>>> options.isSet(Option.G_CUSTOM, "source"); > >>>> } > >>>> // > >>>> > >>>> @@ -365,7 +368,7 @@ > >>>> lambdaDecl.body = translate(makeLambdaBody(tree, > >>>> lambdaDecl)); > >>>> > >>>> boolean dedupe = false; > >>>> - if (!localContext.isSerializable()) { > >>>> + if (!localContext.isSerializable() && !debug) { > >>>> DedupedLambda dedupedLambda = new > >>>> DedupedLambda(lambdaDecl.sym, lambdaDecl.body); > >>>> DedupedLambda existing = > >>>> kInfo.dedupedLambdas.putIfAbsent(dedupedLambda, dedupedLambda); > >>>> if (existing != null) { > >>> > >>> > > > From brian.goetz at oracle.com Thu Mar 22 18:53:13 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 22 Mar 2018 14:53:13 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> <2375f941-2ee2-4d54-3914-72f8c1a6a679@oracle.com> Message-ID: <24d1c46a-6f04-a7ba-58ce-bc3033c1d828@oracle.com> > I would guess that that's actively unlikely: since we're doing only > intra-file duplication right now, lambdas in a given file are more likely > than not to be written in the same style by the same developer. ... if not outright cut-and-pasted. From james.laskey at oracle.com Thu Mar 22 19:23:29 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 22 Mar 2018 16:23:29 -0300 Subject: Raw String Literals In-Reply-To: References: <10B03F59-E391-4F94-B4FE-FD76537D72E8@oracle.com> Message-ID: Like all things, this is a tradeoff, and not necessarily one that admits to simple ?fixes" like "just ban double-ticks." Let?s break it down. 1. The assumption that developers are unable to see double-backtick as anything other than an empty string is not valid. (Consider that single-quotes are used for chars in Java; no one thinks '' is the "empty char".) This is something that is learned. And there already is a way to express ?empty string? in Java ? "". Developers can learn the basic idea that raw string literals should be used when ordinary string literals fail them -- multiple lines, or undesired interpretation of special characters. Think double quotes first. varargs(`Hello`, ``, ` World! `, ``); ==> varargs("Hello", "", " World! ", ""); 2. We were well aware of Markdown?s treatment of ticks; this did indeed influence the design. Markdown, in fact, does support double-backticks but gets blurred in the discussion between spanning and fencing. I myself use double-backticks in Markdown frequently, because that?s how I write up documentation on raw string literals. 3. This ?puzzler? isn?t really a puzzler; it?s just a speed bump on the way to learning the feature. Once you know the rules, it?s not confusing at all. And, IDEs will help with syntax coloring, that will make it obvious that this line doesn?t mean "pass an empty string?. It should also be possible for IDEs to choose the optimal/correct quoting for a given string body. String empty = ``; IDE: Did you mean? String empty = ""; 4. Let?s look at the proposed alternate solution, which is to disallow double-backticks. This may look like an obvious win, because it rescues those that assume that double-backtick is the empty string. But it also creates a cost, which is borne by all users: an anomaly. Learning the rule "any number of ticks" is easier than "any number of ticks, except the second most convenient number (two) you?d probably want to use.? 5. Any solution will introduce puzzlers: varargs(`Hello`, ````, ` World! `, ````);. Using a least puzzler criteria will not lead to a ?better" solution. Cheers, ? Jim > On Mar 22, 2018, at 5:32 AM, Stephen Colebourne wrote: > > On 21 March 2018 at 15:10, Jim Laskey wrote: >> We think things have "settled in" with respect to Raw String Literals language changes and library support. If all things fall in place, we will probably move http://openjdk.java.net/jeps/326 forward soon. > > What does this print? > > public void varargs(String... strs) { > System.out.println(Stream.of(strs).collect(joining())); > } > > varargs(`Hello`, ``, ` World! `, ``); > > . > > > scroll down for the answer... > > > . > > > . > > > Answer: "Hello, `World!`, " (notice the extra commas, because there > are only 2 arguments, not 4, due to no raw string literals) > (thanks to Kevin B for the puzzler) > > As I have stated elsewhere, I believe the trade offs in the raw string > literal proposal are poor for Java. Not allowing an empty string and > not allowing strings to start or end with backticks is surprising, and > I believe the puzzler demonstrates the nasty effects of excluding an > empty string literal (as also demonstrated here > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-March/052052.html). > I do understand the appeal of being able to self-embed (the "stress > test"), I just happen to think it is the wrong trade off. > > > Fortunately, I believe there is a way for Java to have both empty raw > string literals and full embedding. Simply disallow double backtick as > a delimiter. > > `foo` = "foo" (single backtick) > `` = "" (empty string) > ``foo`` = compile error > ```foo``` - "foo" (triple backtick) > `````` - compile error (no empty string here) > `````foo````` - "foo" (five backticks) > > There is a major specification that separates one backtick from three > backticks - Markdown: > http://spec.commonmark.org/0.27/#code-spans > http://spec.commonmark.org/0.27/#fenced-code-blocks > As such, I think this would be readily accepted by developers, who are > often familiar with Markdown. > > Were this approach to be adopted, I believe single backtick literals > should not allow new lines. This would be a good simplification, and > probably help IDE error recovery in many cases. The 3+ backtick > literals would behave as per the current Oracle proposal, but with the > benefit that when embedding just one or two backticks developers will > not need to see or learn the rule about unlimited backticks. Thinks > like regex would tend to use single backticks, while things like > embedding XML or Javascript would use 3+ backticks. > > In summary, not allowing an empty raw string is going to result in > nasty puzzlers, and IMO unlimited delimiters by themselves is not in > the spirit of Java. Following Markdown's approach of separation > between single and 3+ backticks would provide considerable benefits to > Java. > > Stephen From john.r.rose at oracle.com Thu Mar 22 19:29:20 2018 From: john.r.rose at oracle.com (John Rose) Date: Thu, 22 Mar 2018 12:29:20 -0700 Subject: Raw String Literals In-Reply-To: References: <10B03F59-E391-4F94-B4FE-FD76537D72E8@oracle.com> Message-ID: <621126EE-AD85-4245-9AE0-9135F9BF17FD@oracle.com> On Mar 22, 2018, at 1:32 AM, Stephen Colebourne wrote: > > public void varargs(String... strs) { > System.out.println(Stream.of(strs).collect(joining())); > } > > varargs(`Hello`, ``, ` World! `, ``); > > . > > > scroll down for the answer? I'll be blunt here. I didn't need to scroll down; it was obvious to me. Perhaps because I've spent a little more time gaining experience with Jim's proposal. But maybe that's just me? > [?More complex proposal omitted.] > > In summary, not allowing an empty raw string is going to result in > nasty puzzlers, and IMO unlimited delimiters by themselves is not in > the spirit of Java. Following Markdown's approach of separation > between single and 3+ backticks would provide considerable benefits to > Java. The only downside you mention is the puzzler you found. And the puzzler has, IMO, no shiningly obvious property that says, "Jim's proposal is just wrong". Can you guarantee that your more complex proposal has no such puzzlers of its own? It's not Jim's or my burden of proof to provide such a puzzler as a counter example, since that would prove only that we could play a non-terminating game of "find the puzzler in each more complex proposal and argue which puzzler is worse". Putting quotes inside of quotes is an opening move in that game. I claim there is no closing move. ? John From paul.sandoz at oracle.com Thu Mar 22 20:02:46 2018 From: paul.sandoz at oracle.com (paul.sandoz at oracle.com) Date: Thu, 22 Mar 2018 20:02:46 +0000 Subject: hg: amber/amber: For dynamically-computed constants constrain the bootstrap method's Message-ID: <201803222002.w2MK2kWk002468@aojmv0008.oracle.com> Changeset: 3623e1e0e5d8 Author: psandoz Date: 2018-03-22 12:35 -0700 URL: http://hg.openjdk.java.net/amber/amber/rev/3623e1e0e5d8 For dynamically-computed constants constrain the bootstrap method's first parameter to be MethodHandles.Lookup. This clears space in the future for the invocation of bootstrap methods with only the static arguments (the lookup, name and type are not stacked.) ! src/java.base/share/classes/java/lang/invoke/ConstantBootstraps.java - src/java.base/share/classes/java/lang/invoke/DynamicConstant.java ! src/java.base/share/classes/java/lang/invoke/package-info.java ! test/jdk/java/lang/invoke/condy/CondyBSMInvocation.java From paul.sandoz at oracle.com Thu Mar 22 22:38:01 2018 From: paul.sandoz at oracle.com (paul.sandoz at oracle.com) Date: Thu, 22 Mar 2018 22:38:01 +0000 Subject: hg: amber/amber: Make j.l.invoke.ConstantGroup private Message-ID: <201803222238.w2MMc1IR018097@aojmv0008.oracle.com> Changeset: 91e6df6f7571 Author: psandoz Date: 2018-03-22 15:37 -0700 URL: http://hg.openjdk.java.net/amber/amber/rev/91e6df6f7571 Make j.l.invoke.ConstantGroup private ! src/java.base/share/classes/java/lang/invoke/BootstrapCallInfo.java ! src/java.base/share/classes/java/lang/invoke/BootstrapMethodInvoker.java ! src/java.base/share/classes/java/lang/invoke/ConstantGroup.java From stuart.marks at oracle.com Fri Mar 23 06:15:05 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 22 Mar 2018 23:15:05 -0700 Subject: LVTI Style Guide Message-ID: Hi all, I've posted a style guide for LVTI: http://openjdk.java.net/projects/amber/LVTIstyle.html There has been quite a bit of chatter about LVTI surrounding the Java 10 release. This has tended to obscure thoughtful conversations that programmers need to have regarding when and how to use `var` effectively on their projects. This document is an attempt to provide a head start in those conversations. Enjoy, s'marks From scolebourne at joda.org Fri Mar 23 07:55:53 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 23 Mar 2018 07:55:53 +0000 Subject: Raw String Literals In-Reply-To: <621126EE-AD85-4245-9AE0-9135F9BF17FD@oracle.com> References: <10B03F59-E391-4F94-B4FE-FD76537D72E8@oracle.com> <621126EE-AD85-4245-9AE0-9135F9BF17FD@oracle.com> Message-ID: On 22 March 2018 at 19:29, John Rose wrote: > I didn't need to scroll down; it was obvious to me. Perhaps because > I've spent a little more time gaining experience with Jim's proposal. > But maybe that's just me? Yes it is, sorry. I've spent time with the feature too, yet it still got me on first showing. Rather than denying it, the correct question to be asking is why? Two things combine - a deep held expectation about empty strings, and the fact that in visual terms the comma grabs the eye as the argument separator. I saw 4 arguments, carried on reading the email and a few seconds later realised that I was being shown a puzzler, and had to go back and figure it out. > The only downside you mention is the puzzler you found. No. The puzzler is evidence of a design flaw in the language feature. Something that language designers should be very wary of. The problem/downside is breaking a fundamental expectation of users - that empty strings exist. In the spec-experts emails and beyond discussing this feature, I don't think the lack of empty raw string has been widely tested. > Can you guarantee that your more complex proposal has no such > puzzlers of its own? Any puzzler that applies to 3+ backticks in Oracle's proposal also applies to my proposal. Any puzzler that revolves around the empty string necessarily applies only to Oracle's proposal because of the lack of empty string support. Any puzzler that applies to single backtick blocks will have a smaller blast radius with my proposal because it cannot contain new lines. QED! But, just for fun, here is another one: public void foo() { output(`/n`, ``); var str = execute(); output(str + `/r/n`, ``); } OMG!!! (IDE colouring would help with this one, but I refer you to Stuart Marks' recent var style guide - "P3. Code readability shouldn't depend on IDEs") On 22 March 2018 at 19:23, Jim Laskey wrote: > 1. The assumption that developers are unable to see double-backtick as anything other than an empty string is not valid. You've done a survey to show this? Got some other proof? I'd argue forcefully that it is deeply ingrained into most developers thinking about strings that any delimiter repeated twice is the empty string. (a char isn't a string and developers know chars can't be empty, so its not comparable). > 3. This ?puzzler? isn?t really a puzzler; it?s just a speed bump on the way to learning the feature. A puzzler is something that if you put up on a screen, most people get wrong (or struggle with) potentially even if they know the rules. I believe that is definitely the case here. > And, IDEs will help with syntax coloring Not in the case of this puzzler - you are relying on users noticing the colour of two small commas. > 4. Let?s look at the proposed alternate solution, which is to disallow double-backticks. This may look like an obvious win, because it rescues those that assume that double-backtick is the empty string. But it also creates a cost, which is borne by all users: an anomaly. Learning the rule "any number of ticks" is easier than "any number of ticks, except the second most convenient number (two) you?d probably want to use.? Thats not how it would be described or explained though. Instead, just like Markdown does, there would be two distinct descriptions in one section, a simple single backtick variant and a complex multi-backtick one. With the new line restriction on single backticks this would be very obvious in code too. (IMO, a triple backtick a much better visual delimiter for multi-line raw strings anyway). > 5. Any solution will introduce puzzlers: varargs(`Hello`, ````, ` World! `, ````);. In that case the four backtick delimiter is of such a size that it draws the eye away from the comma and doesn't build on the deeply held empty string concept. And anyway, this example applies to both proposals. Stephen From amaembo at gmail.com Fri Mar 23 08:51:25 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Fri, 23 Mar 2018 15:51:25 +0700 Subject: LVTI Style Guide In-Reply-To: References: Message-ID: Hello! Thank you for this post. I was thinking about tuning our IntelliJ IDEA inspection "Local variable type can be omitted". Initially we enabled it, so every local variable type which can be replaced with 'var' was grayed out. Then Brian asked us not to do this, as many people just blindly apply the quick-fixes the IDE suggests. Now by default we display no highlighting, but quick-fix can be invoked from the drop-down menu. Still I think that we may have some intermediate solution when 'var' is suggested by default only in some places where it's applicable. My current thought is to limit default suggestions to these cases: * when the variable initializer is a new expression. * when the variable initializer is a static method, and its containing class is the same as the return value erased type. * when the variable initializer is an instance method and its name is "getXyz" where "Xyz" is the simple class name of the return value erased type. * when the variable initializer is a String/char/boolean constant. More questionable: * when the variable initializer is a String/boolean expression * when the variable initializer is a conditional operator and both branches are satisfied by any of these rules Even more questionable (too much magic): * when the variable initializer is a method which name is "getFoos" where "Foos" is the plural form (according to English grammar rules) of the declared variable name "foo" and type is either "Foo[]" or inheritor of "Iterable" Of course, this list is preliminary. If you have suggestions what to add or what to remove here, feel free to comment. The main concern we have is that the default behavior might look confusing for users: why var is suggested in one place, but not suggested in the other. So we probably should not make this feature too magical. Instead it should follow a clear list of rules which could be documented. Also there are open questions, whether we should have three modes ("never suggest var", "suggest var in some cases", "suggest var everywhere where possible") or more fine-grained setting like additional levels to include questionable cases or even set of checkboxes. Please share your opinions on this! Probably there should be an official recommendation from JDK team to IDE authors (not only IntelliJ) on whether the 'var' suggestion is considered a good style. Thank you, With best regards, Tagir Valeev. On Fri, Mar 23, 2018 at 1:15 PM, Stuart Marks wrote: > Hi all, > > I've posted a style guide for LVTI: > > http://openjdk.java.net/projects/amber/LVTIstyle.html > > There has been quite a bit of chatter about LVTI surrounding the Java 10 > release. This has tended to obscure thoughtful conversations that > programmers need to have regarding when and how to use `var` effectively on > their projects. This document is an attempt to provide a head start in > those conversations. > > Enjoy, > > s'marks > > From kevinb at google.com Fri Mar 23 10:34:37 2018 From: kevinb at google.com (Kevin Bourrillion) Date: Fri, 23 Mar 2018 03:34:37 -0700 Subject: LVTI Style Guide In-Reply-To: References: Message-ID: Stuart, This document is amazing and should emulated for every future language feature. My confidence that our codebase will actually profit from this new feature just shot up a hundredfold. Thanks so much for doing this! On Thu, Mar 22, 2018 at 11:15 PM, Stuart Marks wrote: > Hi all, > > I've posted a style guide for LVTI: > > http://openjdk.java.net/projects/amber/LVTIstyle.html > > There has been quite a bit of chatter about LVTI surrounding the Java 10 > release. This has tended to obscure thoughtful conversations that > programmers need to have regarding when and how to use `var` effectively on > their projects. This document is an attempt to provide a head start in > those conversations. > > Enjoy, > > s'marks > > -- Kevin Bourrillion | Java Librarian | Google, Inc. | kevinb at google.com From forax at univ-mlv.fr Fri Mar 23 10:50:40 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 23 Mar 2018 11:50:40 +0100 (CET) Subject: LVTI Style Guide In-Reply-To: References: Message-ID: <1846635255.2465778.1521802240785.JavaMail.zimbra@u-pem.fr> I agree with Kevin, the item G5 is important, i had several discussion with my students about it. R?mi ----- Mail original ----- > De: "Kevin Bourrillion" > ?: "Stuart Marks" > Cc: "amber-dev" > Envoy?: Vendredi 23 Mars 2018 11:34:37 > Objet: Re: LVTI Style Guide > Stuart, > > This document is amazing and should emulated for every future language > feature. My confidence that our codebase will actually profit from this new > feature just shot up a hundredfold. > > Thanks so much for doing this! > > > On Thu, Mar 22, 2018 at 11:15 PM, Stuart Marks > wrote: > >> Hi all, >> >> I've posted a style guide for LVTI: >> >> http://openjdk.java.net/projects/amber/LVTIstyle.html >> >> There has been quite a bit of chatter about LVTI surrounding the Java 10 >> release. This has tended to obscure thoughtful conversations that >> programmers need to have regarding when and how to use `var` effectively on >> their projects. This document is an attempt to provide a head start in >> those conversations. >> >> Enjoy, >> >> s'marks >> >> > > > -- > Kevin Bourrillion | Java Librarian | Google, Inc. | kevinb at google.com From orionllmain at gmail.com Fri Mar 23 11:06:35 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Fri, 23 Mar 2018 18:06:35 +0700 Subject: LVTI Style Guide In-Reply-To: References: Message-ID: What about this: - Suggest using var for generics (List, Map, PriorityQueue etc). - When a diamond operator is used, it is replaced with an explicit type after refactoring. - Users should be able to turn off this in preferences. - Suggest using var for type names longer than X characters (X should be configured in preferences). - Never suggest var for primitives (they are already very short). 2018-03-23 15:51 GMT+07:00 Tagir Valeev : > Hello! > > Thank you for this post. I was thinking about tuning our IntelliJ IDEA > inspection "Local variable type can be omitted". Initially we enabled it, > so every local variable type which can be replaced with 'var' was grayed > out. Then Brian asked us not to do this, as many people just blindly apply > the quick-fixes the IDE suggests. Now by default we display no > highlighting, but quick-fix can be invoked from the drop-down menu. Still I > think that we may have some intermediate solution when 'var' is suggested > by default only in some places where it's applicable. My current thought is > to limit default suggestions to these cases: > > * when the variable initializer is a new expression. > * when the variable initializer is a static method, and its containing > class is the same as the return value erased type. > * when the variable initializer is an instance method and its name is > "getXyz" where "Xyz" is the simple class name of the return value erased > type. > * when the variable initializer is a String/char/boolean constant. > > More questionable: > * when the variable initializer is a String/boolean expression > * when the variable initializer is a conditional operator and both branches > are satisfied by any of these rules > > Even more questionable (too much magic): > * when the variable initializer is a method which name is "getFoos" where > "Foos" is the plural form (according to English grammar rules) of the > declared variable name "foo" and type is either "Foo[]" or inheritor of > "Iterable" > > Of course, this list is preliminary. If you have suggestions what to add or > what to remove here, feel free to comment. > > The main concern we have is that the default behavior might look confusing > for users: why var is suggested in one place, but not suggested in the > other. So we probably should not make this feature too magical. Instead it > should follow a clear list of rules which could be documented. Also there > are open questions, whether we should have three modes ("never suggest > var", "suggest var in some cases", "suggest var everywhere where possible") > or more fine-grained setting like additional levels to include questionable > cases or even set of checkboxes. > > Please share your opinions on this! Probably there should be an official > recommendation from JDK team to IDE authors (not only IntelliJ) on whether > the 'var' suggestion is considered a good style. > > Thank you, > With best regards, > Tagir Valeev. > > On Fri, Mar 23, 2018 at 1:15 PM, Stuart Marks > wrote: > > > Hi all, > > > > I've posted a style guide for LVTI: > > > > http://openjdk.java.net/projects/amber/LVTIstyle.html > > > > There has been quite a bit of chatter about LVTI surrounding the Java 10 > > release. This has tended to obscure thoughtful conversations that > > programmers need to have regarding when and how to use `var` effectively > on > > their projects. This document is an attempt to provide a head start in > > those conversations. > > > > Enjoy, > > > > s'marks > > > > > From bsrbnd at gmail.com Fri Mar 23 11:11:42 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Fri, 23 Mar 2018 12:11:42 +0100 Subject: deduplicating lambda methods In-Reply-To: <24d1c46a-6f04-a7ba-58ce-bc3033c1d828@oracle.com> References: <92b5cf9a-f013-a4a2-ee1f-49cde24245c6@oracle.com> <752b1ab9-c741-26b6-6a15-3ff75c0f1655@oracle.com> <901bc5e4-bd22-84e4-637a-baa8c4720b83@oracle.com> <253f1071-4bd1-b0ca-1543-e3c62927c29b@oracle.com> <9f4b5b13-7d0c-2c45-6c30-0c7f0ba8f812@oracle.com> <2375f941-2ee2-4d54-3914-72f8c1a6a679@oracle.com> <24d1c46a-6f04-a7ba-58ce-bc3033c1d828@oracle.com> Message-ID: On 22 March 2018 at 19:53, Brian Goetz wrote: > >> I would guess that that's actively unlikely: since we're doing only >> intra-file duplication right now, lambdas in a given file are more likely >> than not to be written in the same style by the same developer. > > > ... if not outright cut-and-pasted. Opinions being divergent (I guess more than one developer is working on the same file ;-) and it shouldn't be too surprising to see two different styles for the same code even with many copy/paste), I think this additional no-op feature might be activated with some '-o' option by the most scrupulous ones? Bernard From jan.lahoda at oracle.com Fri Mar 23 13:07:50 2018 From: jan.lahoda at oracle.com (jan.lahoda at oracle.com) Date: Fri, 23 Mar 2018 13:07:50 +0000 Subject: hg: amber/amber: Avoiding crash in ArgumentAttr when a break inside a switch expression is not a value break. Message-ID: <201803231307.w2ND7ook014956@aojmv0008.oracle.com> Changeset: d6cbfe9e3030 Author: jlahoda Date: 2018-03-23 14:04 +0100 URL: http://hg.openjdk.java.net/amber/amber/rev/d6cbfe9e3030 Avoiding crash in ArgumentAttr when a break inside a switch expression is not a value break. ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/ArgumentAttr.java ! test/langtools/tools/javac/switchexpr/ExpressionSwitchBreaks2.java ! test/langtools/tools/javac/switchexpr/ExpressionSwitchBreaks2.out From amber-spec-experts-bounces at openjdk.java.net Tue Mar 20 13:28:54 2018 From: amber-spec-experts-bounces at openjdk.java.net (amber-spec-experts-bounces at openjdk.java.net) Date: Tue, 20 Mar 2018 13:28:54 +0000 Subject: Forward of moderated message Message-ID: An embedded message was scrubbed... From: Federico Peralta Schaffner Subject: Re: Records -- current status Date: Tue, 20 Mar 2018 00:56:48 +0000 Size: 15551 URL: From edharned at gmail.com Fri Mar 23 13:17:49 2018 From: edharned at gmail.com (Edward Harned) Date: Fri, 23 Mar 2018 09:17:49 -0400 Subject: Style Guide for Local Variable Type Inference ("var") In-Reply-To: <7da527aa-befa-8d25-1805-ab109d0274ae@oracle.com> References: <7da527aa-befa-8d25-1805-ab109d0274ae@oracle.com> Message-ID: Very nice, thank you. ed On Fri, Mar 23, 2018 at 2:44 AM, Stuart Marks wrote: > (replies to amber-dev, please) > > > -------- Forwarded Message -------- > Subject: LVTI Style Guide > Date: Thu, 22 Mar 2018 23:15:05 -0700 > From: Stuart Marks > To: amber-dev > > Hi all, > > I've posted a style guide for LVTI: > > http://openjdk.java.net/projects/amber/LVTIstyle.html > > There has been quite a bit of chatter about LVTI surrounding the Java 10 > release. This has tended to obscure thoughtful conversations that > programmers need to have regarding when and how to use `var` effectively on > their projects. This document is an attempt to provide a head start in > those conversations. > > Enjoy, > > s'marks > > From brian.goetz at oracle.com Fri Mar 23 14:05:28 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 23 Mar 2018 10:05:28 -0400 Subject: Raw String Literals In-Reply-To: References: <10B03F59-E391-4F94-B4FE-FD76537D72E8@oracle.com> <621126EE-AD85-4245-9AE0-9135F9BF17FD@oracle.com> Message-ID: These arguments all boil down to a subjective "I like my syntax better, and I like the anomalies in my version better."? Which is fine, you're welcome to that opinion, and some others might even share it, but please, let's be honest about it -- these are your subjective opinion about what you like better or worse.? Let's not present them as universal truth. Further, personal criticism ("it's just you"), overblown rhetoric ("design flaw in the language"), and aggressive demands for proof of opinions that disagree with yours are not a constructive form of discourse here -- and surely not an effective way to persuade people over to your opinion.? So please dial it down.? The people who worked on this feature worked hard to consider all the tradeoffs -- as evidenced by the fact that all the concerns you raised had in fact already been considered.? You may disagree with their conclusions, or wish to persuade them to reconsider, but if you're going to question their competence, please do it privately. On 3/23/2018 3:55 AM, Stephen Colebourne wrote: > On 22 March 2018 at 19:29, John Rose wrote: >> I didn't need to scroll down; it was obvious to me. Perhaps because >> I've spent a little more time gaining experience with Jim's proposal. >> But maybe that's just me? > Yes it is, sorry. I've spent time with the feature too, yet it still > got me on first showing. Rather than denying it, the correct question > to be asking is why? > > Two things combine - a deep held expectation about empty strings, and > the fact that in visual terms the comma grabs the eye as the argument > separator. I saw 4 arguments, carried on reading the email and a few > seconds later realised that I was being shown a puzzler, and had to go > back and figure it out. > >> The only downside you mention is the puzzler you found. > No. The puzzler is evidence of a design flaw in the language feature. > Something that language designers should be very wary of. The > problem/downside is breaking a fundamental expectation of users - that > empty strings exist. In the spec-experts emails and beyond discussing > this feature, I don't think the lack of empty raw string has been > widely tested. > >> Can you guarantee that your more complex proposal has no such >> puzzlers of its own? > Any puzzler that applies to 3+ backticks in Oracle's proposal also > applies to my proposal. > > Any puzzler that revolves around the empty string necessarily applies > only to Oracle's proposal because of the lack of empty string support. > > Any puzzler that applies to single backtick blocks will have a smaller > blast radius with my proposal because it cannot contain new lines. > > QED! > > But, just for fun, here is another one: > > public void foo() { > output(`/n`, ``); > var str = execute(); > output(str + `/r/n`, ``); > } > > OMG!!! > (IDE colouring would help with this one, but I refer you to Stuart > Marks' recent var style guide - "P3. Code readability shouldn't depend > on IDEs") > > > On 22 March 2018 at 19:23, Jim Laskey wrote: >> 1. The assumption that developers are unable to see double-backtick as anything other than an empty string is not valid. > You've done a survey to show this? Got some other proof? I'd argue > forcefully that it is deeply ingrained into most developers thinking > about strings that any delimiter repeated twice is the empty string. > (a char isn't a string and developers know chars can't be empty, so > its not comparable). > >> 3. This ?puzzler? isn?t really a puzzler; it?s just a speed bump on the way to learning the feature. > A puzzler is something that if you put up on a screen, most people get > wrong (or struggle with) potentially even if they know the rules. I > believe that is definitely the case here. > >> And, IDEs will help with syntax coloring > Not in the case of this puzzler - you are relying on users noticing > the colour of two small commas. > >> 4. Let?s look at the proposed alternate solution, which is to disallow double-backticks. This may look like an obvious win, because it rescues those that assume that double-backtick is the empty string. But it also creates a cost, which is borne by all users: an anomaly. Learning the rule "any number of ticks" is easier than "any number of ticks, except the second most convenient number (two) you?d probably want to use.? > Thats not how it would be described or explained though. Instead, just > like Markdown does, there would be two distinct descriptions in one > section, a simple single backtick variant and a complex multi-backtick > one. With the new line restriction on single backticks this would be > very obvious in code too. (IMO, a triple backtick a much better visual > delimiter for multi-line raw strings anyway). > >> 5. Any solution will introduce puzzlers: varargs(`Hello`, ````, ` World! `, ````);. > In that case the four backtick delimiter is of such a size that it > draws the eye away from the comma and doesn't build on the deeply held > empty string concept. And anyway, this example applies to both > proposals. > > Stephen From stuart.marks at oracle.com Fri Mar 23 16:56:07 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 23 Mar 2018 09:56:07 -0700 Subject: LVTI Style Guide In-Reply-To: References: Message-ID: Thank you! Of course, this wasn't an individual effort. There was lots of help, review, editing etc. from the usual suspects including Brian, Alex, Dan, and Maurizio. s'marks On 3/23/18 3:34 AM, Kevin Bourrillion wrote: > Stuart, > > This document is amazing and should emulated for every future language > feature. My confidence that our codebase will actually profit from this new > feature just shot up a hundredfold. > > Thanks so much for doing this! > > > On Thu, Mar 22, 2018 at 11:15 PM, Stuart Marks > wrote: > > Hi all, > > I've posted a style guide for LVTI: > > http://openjdk.java.net/projects/amber/LVTIstyle.html > > > There has been quite a bit of chatter about LVTI surrounding the Java 10 > release. This has tended to obscure thoughtful conversations that > programmers need to have regarding when and how to use `var` effectively > on their projects. This document is an attempt to provide a head start in > those conversations. > > Enjoy, > > s'marks > > > > > -- > Kevin Bourrillion?|?Java Librarian |?Google, Inc.?|kevinb at google.com > From forax at univ-mlv.fr Fri Mar 23 17:02:18 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 23 Mar 2018 18:02:18 +0100 (CET) Subject: LVTI Style Guide In-Reply-To: References: Message-ID: <636882417.2763567.1521824538202.JavaMail.zimbra@u-pem.fr> Hi Tagir, there is also a notion of block of code, alternating var definition and non-var definition make the code not really readable. R?mi ----- Mail original ----- > De: "Tagir Valeev" > ?: "Stuart Marks" > Cc: "amber-dev" > Envoy?: Vendredi 23 Mars 2018 09:51:25 > Objet: Re: LVTI Style Guide > Hello! > > Thank you for this post. I was thinking about tuning our IntelliJ IDEA > inspection "Local variable type can be omitted". Initially we enabled it, > so every local variable type which can be replaced with 'var' was grayed > out. Then Brian asked us not to do this, as many people just blindly apply > the quick-fixes the IDE suggests. Now by default we display no > highlighting, but quick-fix can be invoked from the drop-down menu. Still I > think that we may have some intermediate solution when 'var' is suggested > by default only in some places where it's applicable. My current thought is > to limit default suggestions to these cases: > > * when the variable initializer is a new expression. > * when the variable initializer is a static method, and its containing > class is the same as the return value erased type. > * when the variable initializer is an instance method and its name is > "getXyz" where "Xyz" is the simple class name of the return value erased > type. > * when the variable initializer is a String/char/boolean constant. > > More questionable: > * when the variable initializer is a String/boolean expression > * when the variable initializer is a conditional operator and both branches > are satisfied by any of these rules > > Even more questionable (too much magic): > * when the variable initializer is a method which name is "getFoos" where > "Foos" is the plural form (according to English grammar rules) of the > declared variable name "foo" and type is either "Foo[]" or inheritor of > "Iterable" > > Of course, this list is preliminary. If you have suggestions what to add or > what to remove here, feel free to comment. > > The main concern we have is that the default behavior might look confusing > for users: why var is suggested in one place, but not suggested in the > other. So we probably should not make this feature too magical. Instead it > should follow a clear list of rules which could be documented. Also there > are open questions, whether we should have three modes ("never suggest > var", "suggest var in some cases", "suggest var everywhere where possible") > or more fine-grained setting like additional levels to include questionable > cases or even set of checkboxes. > > Please share your opinions on this! Probably there should be an official > recommendation from JDK team to IDE authors (not only IntelliJ) on whether > the 'var' suggestion is considered a good style. > > Thank you, > With best regards, > Tagir Valeev. > > On Fri, Mar 23, 2018 at 1:15 PM, Stuart Marks > wrote: > >> Hi all, >> >> I've posted a style guide for LVTI: >> >> http://openjdk.java.net/projects/amber/LVTIstyle.html >> >> There has been quite a bit of chatter about LVTI surrounding the Java 10 >> release. This has tended to obscure thoughtful conversations that >> programmers need to have regarding when and how to use `var` effectively on >> their projects. This document is an attempt to provide a head start in >> those conversations. >> >> Enjoy, >> >> s'marks >> From john.r.rose at oracle.com Fri Mar 23 17:17:16 2018 From: john.r.rose at oracle.com (John Rose) Date: Fri, 23 Mar 2018 10:17:16 -0700 Subject: LVTI Style Guide In-Reply-To: References: Message-ID: <17314B5B-B7BB-4FA7-980E-0EABAA574180@oracle.com> I'm also very pleased by Stuarts writeup. Piling on about IDEs: On Mar 23, 2018, at 1:51 AM, Tagir Valeev wrote: > > Please share your opinions on this! Probably there should be an official > recommendation from JDK team to IDE authors (not only IntelliJ) on whether > the 'var' suggestion is considered a good style. As Stuart says, there is an unavoidable element of human discretion here. So it seems to me that, in Intellij terms, the decision shouldn't usually start with an inspection, a prompt from the IDE with a yes/no answer, but rather a refactoring, which starts with a user decision and elicits user input. Var-ifying a declaration should always involve rethinking whether to add more type information to the variable name. So perhaps the IDE should provide a check-box item (a sticky one) on the Rename Variable refactoring, rather than creating a separate "Toggle Var" refactoring with a name field. (By "sticky" I mean that if I just var-ified one line of code, it's likely I may want to var-ify the next, so I shouldn't have to tick the box twice. And similarly in the other direction.) To Remi's suggestion that the decision is sometimes blockwise, which Stuart also alludes to, perhaps the IDE's Rename Variable refactoring, or something like it, could apply to a whole block or selected set of declarations, collecting a whole roster of more type-ful names for a series of var-ified declarations. HTH ? John From brian.goetz at oracle.com Fri Mar 23 17:28:18 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 23 Mar 2018 13:28:18 -0400 Subject: LVTI Style Guide In-Reply-To: <636882417.2763567.1521824538202.JavaMail.zimbra@u-pem.fr> References: <636882417.2763567.1521824538202.JavaMail.zimbra@u-pem.fr> Message-ID: <861c5408-53c3-34c2-8473-748d5cc7a3d7@oracle.com> > there is also a notion of block of code, > alternating var definition and non-var definition make the code not really readable. Stuart takes pains to point out how much context matters; this is another bit of context that could nudge the right balance a bit one way or the other in certain cases.? If there's a big block of variables that are all-but-one var, then it exerts a pull towards all-var.? This is often OK, but needs to be subject to the other guidelines.? However, I think the IDE is best to not try and take this into account; as Tagir mentions, there is always the option of asking for the quick-fix.? I think the real IDE discussion is, under what circumstances should the IDE highlight something as "your code is dirty".? And I think erring on the side of minimal here is desirable. From stuart.marks at oracle.com Fri Mar 23 21:35:10 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 23 Mar 2018 14:35:10 -0700 Subject: LVTI Style Guide In-Reply-To: References: Message-ID: <264c391d-97fb-fdf9-26cf-ebc2a7e97144@oracle.com> On 3/23/18 3:34 AM, Kevin Bourrillion wrote: > This document is amazing and should emulated for every future language feature. You've picked up on the idea that we've been kicking around internally, which is to publish style information about a feature at more-or-less the same time as the feature is delivered. We didn't do this for Java 8, for example. Things eventually settled down, with most people preferring short expression lambdas, refactoring longer bits of code into methods and using method references, etc. But I still see people trying to replace abstract classes with interfaces and default methods, and I still see people misusing Optional. Publishing some style-related information on those topics earlier probably would have helped things to settle down sooner. Most design discussions of language features seems to involve dialog of the form, "Is it reasonable to use it this way?" "Well that depends." "What if you did it that other way?" "That's probably wrong. Don't do that." "How about this way instead?" "Yes, it works in that case, because...." Eventually this could lead to style guidelines, but usually it just bounces around the mailing list and is forgotten. Occasionally some enterprising soul will dig things out of the mailing list and post them on Stack Overflow, but that's pretty rare, and in any case such information is still relatively inaccessible. So yes, the idea is to try to get usage guidelines out there at the same as the feature is delivered. Can't promise anything, but it's certainly something I'm advocating. Same for design rationale, but that's a different topic. s'marks From vicente.romero at oracle.com Mon Mar 26 13:20:58 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Mon, 26 Mar 2018 09:20:58 -0400 Subject: deduplicating lambda methods In-Reply-To: References: Message-ID: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> Hi all, Although there are still areas under discussion I think that we should go for a 'final', pushable version of the patch, which I think should be based on the last iteration sent by Liam, plus code based in the changes I proposed to don't go deeper in the tree comparison as soon as a constant is found, plus regression tests. We can continue discussing about the no-op filtering proposed by Bernard and if we have data supporting that change, we can include it in a next iteration. I think that we all mostly agree on the consistency of the state I'm describing, so I think we should go for it, Thanks, Vicente On 03/02/2018 08:03 PM, Liam Miller-Cushon wrote: > Hello, > > I'm interested in adding support for deduplicating lambda methods to > javac. The idea is that if a compilation unit contains two lambdas > that are identical (including any captured state and the functional > interface they implement) we could re-use the same implementation > method for both. > > I understand there might have been some prior discussion about this. > Is there interest in investigating the feature? What sort of technical > considerations have been identified so far? > > I have been thinking about a couple of questions: > > 1) How to identifying duplicates: I have a prototype that runs during > lambda desugaring and identifies duplicates by diffing ASTs. Is that > the best place for deduplication, or it worth considering comparing > generated code instead of ASTs? > > 2) Debug info: the optimization is safe if line numbers are not being > emitted. If they are, is there a way to deduplicate the methods > without breaking debug info? > > Thanks, > Liam From vicente.romero at oracle.com Mon Mar 26 16:19:32 2018 From: vicente.romero at oracle.com (vicente.romero at oracle.com) Date: Mon, 26 Mar 2018 16:19:32 +0000 Subject: hg: amber/amber: jshell failing on launch with:No corresponding TK for TokenKind error Message-ID: <201803261619.w2QGJWYL001658@aojmv0008.oracle.com> Changeset: 1be3e11b4413 Author: vromero Date: 2018-03-26 12:11 -0400 URL: http://hg.openjdk.java.net/amber/amber/rev/1be3e11b4413 jshell failing on launch with:No corresponding TK for TokenKind error ! src/jdk.jshell/share/classes/jdk/jshell/CompletenessAnalyzer.java From cushon at google.com Mon Mar 26 23:14:34 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Mon, 26 Mar 2018 23:14:34 +0000 Subject: deduplicating lambda methods In-Reply-To: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> Message-ID: Hi Vicente, I pushed an updated webrev to: http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.04/ The main changes are: * I incorporated your patch for handling constant values. * The change now includes a test harness for lambda deduplication. * Deduplication is now skipped if debug info is enabled. The test asserts different groups of lambdas in the test input are deduplicated as expected, and has targeted coverage of diffing and hashing for all pairwise combinations of lambdas in the test input. I like Maurizio's idea of applying fuzzing here, but haven't yet made progress on that. As an aside, there might be a separate larger project here to investigate ways to fuzz javac APIs. I noticed there was some work done to fuzz the class reader [1]. [1] https://bugs.openjdk.java.net/browse/JDK-8054284?jql=labels%20%3D%20class_fuzzing Re: deduplicating lambdas that declare locals, e.g. : `() -> { int y = 42; return y; }`. I think that comparing locals by name is correct, assuming that the diff is not used on trees that can refer to locals declared in enclosing scopes (this is true for desugared lambda bodies, but not in general). Another option is to have the diff record local variable declarations it sees, and maintain a map of symbols that can be treated as equivalent. The current lambda parameter handling could be folded in to this--the lambda parameters would be initial values in the map. This could also be generalized to other types of declarations, but that becomes problematic for declarations whose identity could escape the lambda. E.g. deduplicating two instances of `x -> { class Foo {}; baz(Foo.class); }` could change behaviour. Finally, I agree with Brian and Louis that we should have stronger evidence that more sophisticated handling of no-ops would be a win before adding complexity to support that. Inter-file handling of very common lambdas might be a more beneficial future improvement. On Mon, Mar 26, 2018 at 6:21 AM Vicente Romero wrote: > Hi all, > > Although there are still areas under discussion I think that we should > go for a 'final', pushable version of the patch, which I think should be > based on the last iteration sent by Liam, plus code based in the changes > I proposed to don't go deeper in the tree comparison as soon as a > constant is found, plus regression tests. We can continue discussing > about the no-op filtering proposed by Bernard and if we have data > supporting that change, we can include it in a next iteration. I think > that we all mostly agree on the consistency of the state I'm describing, > so I think we should go for it, > > Thanks, > Vicente > > On 03/02/2018 08:03 PM, Liam Miller-Cushon wrote: > > Hello, > > > > I'm interested in adding support for deduplicating lambda methods to > > javac. The idea is that if a compilation unit contains two lambdas > > that are identical (including any captured state and the functional > > interface they implement) we could re-use the same implementation > > method for both. > > > > I understand there might have been some prior discussion about this. > > Is there interest in investigating the feature? What sort of technical > > considerations have been identified so far? > > > > I have been thinking about a couple of questions: > > > > 1) How to identifying duplicates: I have a prototype that runs during > > lambda desugaring and identifies duplicates by diffing ASTs. Is that > > the best place for deduplication, or it worth considering comparing > > generated code instead of ASTs? > > > > 2) Debug info: the optimization is safe if line numbers are not being > > emitted. If they are, is there a way to deduplicate the methods > > without breaking debug info? > > > > Thanks, > > Liam > > From vicente.romero at oracle.com Tue Mar 27 00:24:53 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Mon, 26 Mar 2018 20:24:53 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> Message-ID: Hi Liam, looks good! one minor comment: at TreeDiffer::visitIdent there is some code that was probably added for testing: ??????? Consumer c = x -> { ??????????? class Foo {} ??????????? foo(Foo.class); ??????? }; along with the empty method: `void foo(Class clazz) {}` that is defined above ::visitIdent. there is no need for another iteration for this. Regarding the fuzzing test, if Maurizio agrees, I'm OK with going with this version of the patch, and add that one and more tests later on. Thanks! Vicente On 03/26/2018 07:14 PM, Liam Miller-Cushon wrote: > Hi Vicente, > > I pushed an updated webrev to: > http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.04/ > > > The main changes are: > * I incorporated your patch for handling constant values. > * The change now includes a test harness for lambda deduplication. > * Deduplication is now skipped if debug info is enabled. > > The test asserts different groups of lambdas in the test input are > deduplicated as expected, and has targeted coverage of diffing > and hashing for all pairwise combinations of lambdas in the test input. > > I like Maurizio's idea of applying fuzzing here, but haven't yet > made progress on that. As an aside, there might be a separate larger > project here to investigate ways to fuzz javac APIs. I noticed there > was some work done to fuzz the class reader [1]. > > [1] > https://bugs.openjdk.java.net/browse/JDK-8054284?jql=labels%20%3D%20class_fuzzing > > Re: deduplicating lambdas that declare locals, > e.g. : `() -> { int y = 42; return y; }`. I think that comparing locals by > name is correct, assuming that the diff is not used on trees > that can refer to locals declared in enclosing scopes (this is true for > desugared lambda bodies, but not in general). Another option is to have > the diff record local variable declarations it sees, and maintain a map > of symbols that can be treated as equivalent. The current lambda parameter > handling could be folded in to this--the lambda parameters would be > initial > values in the map. This could also be generalized to other types of > declarations, but that becomes problematic for declarations whose identity > could escape the lambda. E.g. deduplicating two instances of > `x -> { class Foo {}; baz(Foo.class); }` could change behaviour. > > Finally, I agree with Brian and Louis that we should have stronger > evidence that more sophisticated handling of no-ops would be a win before > adding complexity to support that. Inter-file handling of very common > lambdas > might be a more beneficial future improvement. > > > On Mon, Mar 26, 2018 at 6:21 AM Vicente Romero > > wrote: > > Hi all, > > Although there are still areas under discussion I think that we should > go for a 'final', pushable version of the patch, which I think > should be > based on the last iteration sent by Liam, plus code based in the > changes > I proposed to don't go deeper in the tree comparison as soon as a > constant is found, plus regression tests. We can continue discussing > about the no-op filtering proposed by Bernard and if we have data > supporting that change, we can include it in a next iteration. I think > that we all mostly agree on the consistency of the state I'm > describing, > so I think we should go for it, > > Thanks, > Vicente > > On 03/02/2018 08:03 PM, Liam Miller-Cushon wrote: > > Hello, > > > > I'm interested in adding support for deduplicating lambda methods to > > javac. The idea is that if a compilation unit contains two lambdas > > that are identical (including any captured state and the functional > > interface they implement) we could re-use the same implementation > > method for both. > > > > I understand there might have been some prior discussion about this. > > Is there interest in investigating the feature? What sort of > technical > > considerations have been identified so far? > > > > I have been thinking about a couple of questions: > > > > 1) How to identifying duplicates: I have a prototype that runs > during > > lambda desugaring and identifies duplicates by diffing ASTs. Is that > > the best place for deduplication, or it worth considering comparing > > generated code instead of ASTs? > > > > 2) Debug info: the optimization is safe if line numbers are not > being > > emitted. If they are, is there a way to deduplicate the methods > > without breaking debug info? > > > > Thanks, > > Liam > From cushon at google.com Tue Mar 27 05:25:20 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Tue, 27 Mar 2018 05:25:20 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> Message-ID: On Mon, Mar 26, 2018 at 5:25 PM Vicente Romero wrote: > looks good! > thanks! > one minor comment: > oops, thanks for the catch. Fixed. I made a few more small changes since the last webrev: * There were some failing javac tests: annotations/typeAnnotations/classfile/InstanceInitializer, annotations/typeAnnotations/classfile/StaticInitializer, classfiles/attributes/Synthetic/BridgeMethodsForLambdaTest. The output looks correct, but the lambda deduplication broke some assertions about the expected bytecode. I added a flag to those tests to disable the feature for now. * I fixed CheckExamples to handle the 'note' diagnostic that was added to LambdaToMethod. * I fixed a bug in TreeDiffer involving incorrectly recursing into the target of break and continue statements, and added another test case. The tier1 tests are all passing now, and the changeset is attached. From maurizio.cimadamore at oracle.com Tue Mar 27 10:12:59 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 27 Mar 2018 11:12:59 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> Message-ID: <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> This looks good to me. Great job! Maurizio On 27/03/18 06:25, Liam Miller-Cushon wrote: > On Mon, Mar 26, 2018 at 5:25 PM Vicente Romero > wrote: > >> looks good! >> > thanks! > > >> one minor comment: >> > oops, thanks for the catch. Fixed. > > I made a few more small changes since the last webrev: > * There were some failing javac tests: > annotations/typeAnnotations/classfile/InstanceInitializer, > annotations/typeAnnotations/classfile/StaticInitializer, > classfiles/attributes/Synthetic/BridgeMethodsForLambdaTest. The output > looks correct, but the lambda deduplication broke some assertions about the > expected bytecode. I added a flag to those tests to disable the feature for > now. > * I fixed CheckExamples to handle the 'note' diagnostic that was added to > LambdaToMethod. > * I fixed a bug in TreeDiffer involving incorrectly recursing into the > target of break and continue statements, and added another test case. > > The tier1 tests are all passing now, and the changeset is attached. From maurizio.cimadamore at oracle.com Tue Mar 27 10:16:58 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 27 Mar 2018 11:16:58 +0100 Subject: deduplicating lambda methods In-Reply-To: <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> Message-ID: <05cca7f8-f886-15b4-825c-c4689cdf0c1a@oracle.com> Hit 'Send' too fast - I have only one small comment: I see that the dedup logic is disabled for serializable lambdas and also when debugging options are present. As you said, there are some tests that rely on certain bytecode shape, so in such case it might be handy to switch off the machinery entirely; rather than switching off the machinery by using '-g' (which could result in potentially other bytecode changes), I guess I'd prefer to have an escape hatch hidden flag we could use to shut down the dedup machinery entirely, without altering any other javac codegen behavior. This could be useful as a conservative prevention measures in build systems (e.g. JDK itself). Maurizio On 27/03/18 11:12, Maurizio Cimadamore wrote: > This looks good to me. > > Great job! > > Maurizio > > > On 27/03/18 06:25, Liam Miller-Cushon wrote: >> On Mon, Mar 26, 2018 at 5:25 PM Vicente Romero >> >> wrote: >> >>> looks good! >>> >> thanks! >> >> >>> one minor comment: >>> >> oops, thanks for the catch. Fixed. >> >> I made a few more small changes since the last webrev: >> * There were some failing javac tests: >> annotations/typeAnnotations/classfile/InstanceInitializer, >> annotations/typeAnnotations/classfile/StaticInitializer, >> classfiles/attributes/Synthetic/BridgeMethodsForLambdaTest. The output >> looks correct, but the lambda deduplication broke some assertions >> about the >> expected bytecode. I added a flag to those tests to disable the >> feature for >> now. >> * I fixed CheckExamples to handle the 'note' diagnostic that was >> added to >> LambdaToMethod. >> * I fixed a bug in TreeDiffer involving incorrectly recursing into the >> target of break and continue statements, and added another test case. >> >> The tier1 tests are all passing now, and the changeset is attached. > From cushon at google.com Tue Mar 27 10:53:59 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Tue, 27 Mar 2018 10:53:59 +0000 Subject: deduplicating lambda methods In-Reply-To: <05cca7f8-f886-15b4-825c-c4689cdf0c1a@oracle.com> References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <05cca7f8-f886-15b4-825c-c4689cdf0c1a@oracle.com> Message-ID: On Tue, Mar 27, 2018 at 3:17 AM Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > I see that the dedup logic is disabled for serializable lambdas and also > when debugging options are present. As you said, there are some tests > that rely on certain bytecode shape, so in such case it might be handy > to switch off the machinery entirely; rather than switching off the > machinery by using '-g' (which could result in potentially other > bytecode changes), I guess I'd prefer to have an escape hatch hidden > flag we could use to shut down the dedup machinery entirely, without > altering any other javac codegen behavior. This could be useful as a > conservative prevention measures in build systems (e.g. JDK itself). > I should have clarified that the flag I added in the last update to fix the tests is actually a new flag (--debug:deduplicateLambdas={true,false}), instead of using '-g' to disable deduplication as a side-effect. Here it is as a webrev: http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.05/ From brian.goetz at oracle.com Tue Mar 27 14:17:22 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 27 Mar 2018 10:17:22 -0400 Subject: deduplicating lambda methods In-Reply-To: <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> Message-ID: <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Yes, I think this is ready to say we're done with development on this and move towards testing and integration (modulo the improvements suggested in review.)? We'll take the patch and re-run the stats on the JDK to see where there was duplication; I don't expect a lot as most the JDK codebase has been slow to adopt lambdas for various reasons.? I'll post the results here. I'm more interested in the impact on the Google codebase, both the fraction of lambdas combined, as well as if there is any effect on compilation performance. On 3/27/2018 6:12 AM, Maurizio Cimadamore wrote: > This looks good to me. > > Great job! > > Maurizio > > > On 27/03/18 06:25, Liam Miller-Cushon wrote: >> On Mon, Mar 26, 2018 at 5:25 PM Vicente Romero >> >> wrote: >> >>> looks good! >>> >> thanks! >> >> >>> one minor comment: >>> >> oops, thanks for the catch. Fixed. >> >> I made a few more small changes since the last webrev: >> * There were some failing javac tests: >> annotations/typeAnnotations/classfile/InstanceInitializer, >> annotations/typeAnnotations/classfile/StaticInitializer, >> classfiles/attributes/Synthetic/BridgeMethodsForLambdaTest. The output >> looks correct, but the lambda deduplication broke some assertions >> about the >> expected bytecode. I added a flag to those tests to disable the >> feature for >> now. >> * I fixed CheckExamples to handle the 'note' diagnostic that was >> added to >> LambdaToMethod. >> * I fixed a bug in TreeDiffer involving incorrectly recursing into the >> target of break and continue statements, and added another test case. >> >> The tier1 tests are all passing now, and the changeset is attached. > From vicente.romero at oracle.com Tue Mar 27 14:42:26 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 27 Mar 2018 10:42:26 -0400 Subject: deduplicating lambda methods In-Reply-To: <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: On 03/27/2018 10:17 AM, Brian Goetz wrote: > Yes, I think this is ready to say we're done with development on this > and move towards testing and integration (modulo the improvements > suggested in review.) I'm running a mach5 job previous to pushing the patch, so that we cover closed repo tests too > ? We'll take the patch and re-run the stats on the JDK to see where > there was duplication; I don't expect a lot as most the JDK codebase > has been slow to adopt lambdas for various reasons.? I'll post the > results here. I'm running the stats, will send the results soon > > I'm more interested in the impact on the Google codebase, both the > fraction of lambdas combined, as well as if there is any effect on > compilation performance. yep, me too :) Vicente > > > > On 3/27/2018 6:12 AM, Maurizio Cimadamore wrote: >> This looks good to me. >> >> Great job! >> >> Maurizio >> >> >> On 27/03/18 06:25, Liam Miller-Cushon wrote: >>> On Mon, Mar 26, 2018 at 5:25 PM Vicente Romero >>> >>> wrote: >>> >>>> looks good! >>>> >>> thanks! >>> >>> >>>> one minor comment: >>>> >>> oops, thanks for the catch. Fixed. >>> >>> I made a few more small changes since the last webrev: >>> * There were some failing javac tests: >>> annotations/typeAnnotations/classfile/InstanceInitializer, >>> annotations/typeAnnotations/classfile/StaticInitializer, >>> classfiles/attributes/Synthetic/BridgeMethodsForLambdaTest. The output >>> looks correct, but the lambda deduplication broke some assertions >>> about the >>> expected bytecode. I added a flag to those tests to disable the >>> feature for >>> now. >>> * I fixed CheckExamples to handle the 'note' diagnostic that was >>> added to >>> LambdaToMethod. >>> * I fixed a bug in TreeDiffer involving incorrectly recursing into the >>> target of break and continue statements, and added another test case. >>> >>> The tier1 tests are all passing now, and the changeset is attached. >> > From maurizio.cimadamore at oracle.com Tue Mar 27 15:07:22 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 27 Mar 2018 16:07:22 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <05cca7f8-f886-15b4-825c-c4689cdf0c1a@oracle.com> Message-ID: <1cda8fef-3b52-bf9d-6da3-5847e3829520@oracle.com> Hi Liam, I believe --debug options are used to generate extra information out of javac, not to alter its behavior - it's probably better to use an hidden flag (e.g. -XDdeduplicateLambdas=...) - which is consistent to what we have done for indy string concat. Maurizio On 27/03/18 11:53, Liam Miller-Cushon wrote: > On Tue, Mar 27, 2018 at 3:17 AM Maurizio Cimadamore > > wrote: > > I see that the dedup logic is disabled for serializable lambdas > and also > when debugging options are present. As you said, there are some tests > that rely on certain bytecode shape, so in such case it might be handy > to switch off the machinery entirely; rather than switching off the > machinery by using '-g' (which could result in potentially other > bytecode changes), I guess I'd prefer to have an escape hatch hidden > flag we could use to shut down the dedup machinery entirely, without > altering any other javac codegen behavior. This could be useful as a > conservative prevention measures in build systems (e.g. JDK itself). > > > I should have clarified that the flag I added in the last update to > fix the tests > is actually a new flag (--debug:deduplicateLambdas={true,false}), > instead of > using '-g' to disable deduplication as a side-effect. > > Here it is as a webrev: > http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.05/ > From vicente.romero at oracle.com Tue Mar 27 15:19:46 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 27 Mar 2018 11:19:46 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: Hi, These are the stats after applying the lambda deduplication patch to the condy-folding repo. There are actually 2 results, first with no condy begin generated and the second with condy, for the sake of the comparison. I'm running an extra test with constant folding but I don't expect it to change the results. Thanks, Vicente On 03/27/2018 10:42 AM, Vicente Romero wrote: > > > On 03/27/2018 10:17 AM, Brian Goetz wrote: >> Yes, I think this is ready to say we're done with development on this >> and move towards testing and integration (modulo the improvements >> suggested in review.) > > I'm running a mach5 job previous to pushing the patch, so that we > cover closed repo tests too > >> ? We'll take the patch and re-run the stats on the JDK to see where >> there was duplication; I don't expect a lot as most the JDK codebase >> has been slow to adopt lambdas for various reasons.? I'll post the >> results here. > > I'm running the stats, will send the results soon > >> >> I'm more interested in the impact on the Google codebase, both the >> fraction of lambdas combined, as well as if there is any effect on >> compilation performance. > > yep, me too :) > > Vicente >> >> >> >> On 3/27/2018 6:12 AM, Maurizio Cimadamore wrote: >>> This looks good to me. >>> >>> Great job! >>> >>> Maurizio >>> >>> >>> On 27/03/18 06:25, Liam Miller-Cushon wrote: >>>> On Mon, Mar 26, 2018 at 5:25 PM Vicente Romero >>>> >>>> wrote: >>>> >>>>> looks good! >>>>> >>>> thanks! >>>> >>>> >>>>> one minor comment: >>>>> >>>> oops, thanks for the catch. Fixed. >>>> >>>> I made a few more small changes since the last webrev: >>>> * There were some failing javac tests: >>>> annotations/typeAnnotations/classfile/InstanceInitializer, >>>> annotations/typeAnnotations/classfile/StaticInitializer, >>>> classfiles/attributes/Synthetic/BridgeMethodsForLambdaTest. The output >>>> looks correct, but the lambda deduplication broke some assertions >>>> about the >>>> expected bytecode. I added a flag to those tests to disable the >>>> feature for >>>> now. >>>> * I fixed CheckExamples to handle the 'note' diagnostic that was >>>> added to >>>> LambdaToMethod. >>>> * I fixed a bug in TreeDiffer involving incorrectly recursing into the >>>> target of break and continue statements, and added another test case. >>>> >>>> The tier1 tests are all passing now, and the changeset is attached. >>> >> > -------------- next part -------------- no condy generation + deduplicate patch applied TOTAL: total number of sources 742 Lambda Serializable Non-Serializable stateless Lambda 7 953 stateful Lambda 10 1689 Reference Serializable Non-Serializable stateless Reference 23 711 stateful Reference 0 528 total Number Of Indy CP Entries Per Source 3683 total Number Of Indy Invocations Per Source 3966 total Number Of Condy CP Entries Per Source 0 total Number Of Condy Loads Per Source 0 condy for stateless lambdas + deduplicate lambdas ---------------------------------------------------------------------------- TOTAL: total number of sources 742 Lambda Serializable Non-Serializable stateless Lambda 7 953 stateful Lambda 10 1689 Reference Serializable Non-Serializable stateless Reference 23 711 stateful Reference 0 528 total Number Of Indy CP Entries Per Source 2157 total Number Of Indy Invocations Per Source 2282 total Number Of Condy CP Entries Per Source 1526 total Number Of Condy Loads Per Source 1684 From brian.goetz at oracle.com Tue Mar 27 16:58:03 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 27 Mar 2018 12:58:03 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: It looks like there were no changes in the outcome, perhaps because there were no within-file duplications in the JDK.? (Which I believe.)? A more streams/Rx/CompletableFuture-heavy codebase would likely see an improvement. On 3/27/2018 11:19 AM, Vicente Romero wrote: > Hi, > > These are the stats after applying the lambda deduplication patch to > the condy-folding repo. There are actually 2 results, first with no > condy begin generated and the second with condy, for the sake of the > comparison. I'm running an extra test with constant folding but I > don't expect it to change the results. > > Thanks, > Vicente > > On 03/27/2018 10:42 AM, Vicente Romero wrote: >> >> >> On 03/27/2018 10:17 AM, Brian Goetz wrote: >>> Yes, I think this is ready to say we're done with development on >>> this and move towards testing and integration (modulo the >>> improvements suggested in review.) >> >> I'm running a mach5 job previous to pushing the patch, so that we >> cover closed repo tests too >> >>> ? We'll take the patch and re-run the stats on the JDK to see where >>> there was duplication; I don't expect a lot as most the JDK codebase >>> has been slow to adopt lambdas for various reasons.? I'll post the >>> results here. >> >> I'm running the stats, will send the results soon >> >>> >>> I'm more interested in the impact on the Google codebase, both the >>> fraction of lambdas combined, as well as if there is any effect on >>> compilation performance. >> >> yep, me too :) >> >> Vicente >>> >>> >>> >>> On 3/27/2018 6:12 AM, Maurizio Cimadamore wrote: >>>> This looks good to me. >>>> >>>> Great job! >>>> >>>> Maurizio >>>> >>>> >>>> On 27/03/18 06:25, Liam Miller-Cushon wrote: >>>>> On Mon, Mar 26, 2018 at 5:25 PM Vicente Romero >>>>> >>>>> wrote: >>>>> >>>>>> looks good! >>>>>> >>>>> thanks! >>>>> >>>>> >>>>>> one minor comment: >>>>>> >>>>> oops, thanks for the catch. Fixed. >>>>> >>>>> I made a few more small changes since the last webrev: >>>>> * There were some failing javac tests: >>>>> annotations/typeAnnotations/classfile/InstanceInitializer, >>>>> annotations/typeAnnotations/classfile/StaticInitializer, >>>>> classfiles/attributes/Synthetic/BridgeMethodsForLambdaTest. The >>>>> output >>>>> looks correct, but the lambda deduplication broke some assertions >>>>> about the >>>>> expected bytecode. I added a flag to those tests to disable the >>>>> feature for >>>>> now. >>>>> * I fixed CheckExamples to handle the 'note' diagnostic that was >>>>> added to >>>>> LambdaToMethod. >>>>> * I fixed a bug in TreeDiffer involving incorrectly recursing into >>>>> the >>>>> target of break and continue statements, and added another test case. >>>>> >>>>> The tier1 tests are all passing now, and the changeset is attached. >>>> >>> >> > From vicente.romero at oracle.com Tue Mar 27 17:20:10 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 27 Mar 2018 13:20:10 -0400 Subject: deduplicating lambda methods In-Reply-To: <1cda8fef-3b52-bf9d-6da3-5847e3829520@oracle.com> References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <05cca7f8-f886-15b4-825c-c4689cdf0c1a@oracle.com> <1cda8fef-3b52-bf9d-6da3-5847e3829520@oracle.com> Message-ID: <6fd5de3c-e377-119d-4614-5e4d9226d32d@oracle.com> Hi, On 03/27/2018 11:07 AM, Maurizio Cimadamore wrote: > > Hi Liam, > I believe --debug options are used to generate extra information out > of javac, not to alter its behavior - it's probably better to use an > hidden flag (e.g. -XDdeduplicateLambdas=...) - which is consistent to > what we have done for indy string concat. > I will introduce this change before pushing the patch, > Maurizio > Vicente > > On 27/03/18 11:53, Liam Miller-Cushon wrote: >> On Tue, Mar 27, 2018 at 3:17 AM Maurizio Cimadamore >> > > wrote: >> >> I see that the dedup logic is disabled for serializable lambdas >> and also >> when debugging options are present. As you said, there are some tests >> that rely on certain bytecode shape, so in such case it might be >> handy >> to switch off the machinery entirely; rather than switching off the >> machinery by using '-g' (which could result in potentially other >> bytecode changes), I guess I'd prefer to have an escape hatch hidden >> flag we could use to shut down the dedup machinery entirely, without >> altering any other javac codegen behavior. This could be useful as a >> conservative prevention measures in build systems (e.g. JDK itself). >> >> >> I should have clarified that the flag I added in the last update to >> fix the tests >> is actually a new flag (--debug:deduplicateLambdas={true,false}), >> instead of >> using '-g' to disable deduplication as a side-effect. >> >> Here it is as a webrev: >> http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.05/ >> > From vicente.romero at oracle.com Tue Mar 27 17:33:40 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 27 Mar 2018 13:33:40 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: On 03/27/2018 12:58 PM, Brian Goetz wrote: > It looks like there were no changes in the outcome, perhaps because > there were no within-file duplications in the JDK.? (Which I > believe.)? A more streams/Rx/CompletableFuture-heavy codebase would > likely see an improvement. right, no difference :(, let's see what happens with Liam's numbers :) Vicente > > On 3/27/2018 11:19 AM, Vicente Romero wrote: >> Hi, >> >> These are the stats after applying the lambda deduplication patch to >> the condy-folding repo. There are actually 2 results, first with no >> condy begin generated and the second with condy, for the sake of the >> comparison. I'm running an extra test with constant folding but I >> don't expect it to change the results. >> >> Thanks, >> Vicente >> >> On 03/27/2018 10:42 AM, Vicente Romero wrote: >>> >>> >>> On 03/27/2018 10:17 AM, Brian Goetz wrote: >>>> Yes, I think this is ready to say we're done with development on >>>> this and move towards testing and integration (modulo the >>>> improvements suggested in review.) >>> >>> I'm running a mach5 job previous to pushing the patch, so that we >>> cover closed repo tests too >>> >>>> ? We'll take the patch and re-run the stats on the JDK to see where >>>> there was duplication; I don't expect a lot as most the JDK >>>> codebase has been slow to adopt lambdas for various reasons.? I'll >>>> post the results here. >>> >>> I'm running the stats, will send the results soon >>> >>>> >>>> I'm more interested in the impact on the Google codebase, both the >>>> fraction of lambdas combined, as well as if there is any effect on >>>> compilation performance. >>> >>> yep, me too :) >>> >>> Vicente >>>> >>>> >>>> >>>> On 3/27/2018 6:12 AM, Maurizio Cimadamore wrote: >>>>> This looks good to me. >>>>> >>>>> Great job! >>>>> >>>>> Maurizio >>>>> >>>>> >>>>> On 27/03/18 06:25, Liam Miller-Cushon wrote: >>>>>> On Mon, Mar 26, 2018 at 5:25 PM Vicente Romero >>>>>> >>>>>> wrote: >>>>>> >>>>>>> looks good! >>>>>>> >>>>>> thanks! >>>>>> >>>>>> >>>>>>> one minor comment: >>>>>>> >>>>>> oops, thanks for the catch. Fixed. >>>>>> >>>>>> I made a few more small changes since the last webrev: >>>>>> * There were some failing javac tests: >>>>>> annotations/typeAnnotations/classfile/InstanceInitializer, >>>>>> annotations/typeAnnotations/classfile/StaticInitializer, >>>>>> classfiles/attributes/Synthetic/BridgeMethodsForLambdaTest. The >>>>>> output >>>>>> looks correct, but the lambda deduplication broke some assertions >>>>>> about the >>>>>> expected bytecode. I added a flag to those tests to disable the >>>>>> feature for >>>>>> now. >>>>>> * I fixed CheckExamples to handle the 'note' diagnostic that was >>>>>> added to >>>>>> LambdaToMethod. >>>>>> * I fixed a bug in TreeDiffer involving incorrectly recursing >>>>>> into the >>>>>> target of break and continue statements, and added another test >>>>>> case. >>>>>> >>>>>> The tier1 tests are all passing now, and the changeset is attached. >>>>> >>>> >>> >> > From bsrbnd at gmail.com Tue Mar 27 18:22:14 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Tue, 27 Mar 2018 20:22:14 +0200 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: On 27 March 2018 at 19:33, Vicente Romero wrote: > > > On 03/27/2018 12:58 PM, Brian Goetz wrote: >> >> It looks like there were no changes in the outcome, perhaps because there >> were no within-file duplications in the JDK. (Which I believe.) A more >> streams/Rx/CompletableFuture-heavy codebase would likely see an improvement. > > > right, no difference :(, let's see what happens with Liam's numbers :) Did you include the improvement I suggested to compare local variables *by name* instead of * by symbol* ? If not, there's quite no chance to find duplicates excepted trivial ones ;-) Bernard > Vicente From bsrbnd at gmail.com Tue Mar 27 18:38:51 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Tue, 27 Mar 2018 20:38:51 +0200 Subject: deduplicating lambda methods In-Reply-To: <6fd5de3c-e377-119d-4614-5e4d9226d32d@oracle.com> References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <05cca7f8-f886-15b4-825c-c4689cdf0c1a@oracle.com> <1cda8fef-3b52-bf9d-6da3-5847e3829520@oracle.com> <6fd5de3c-e377-119d-4614-5e4d9226d32d@oracle.com> Message-ID: On 27 March 2018 at 19:20, Vicente Romero wrote: > Hi, > > On 03/27/2018 11:07 AM, Maurizio Cimadamore wrote: >> >> >> Hi Liam, >> I believe --debug options are used to generate extra information out of >> javac, not to alter its behavior - it's probably better to use an hidden >> flag (e.g. -XDdeduplicateLambdas=...) - which is consistent to what we have >> done for indy string concat. >> > > I will introduce this change before pushing the patch, Should this option really be hidden? It's currently necessary to disable de-duplication when debugging to preserve all LNT/LVT. Of course, this could be a '-X...' instead of '-g' but documented, I think. Note that de-duplication doesn't really alter javac behavior; let say 'in reverse' that lambda *duplication* would be an additional debug information... Bernard >> Maurizio >> > > Vicente From vicente.romero at oracle.com Tue Mar 27 18:43:42 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 27 Mar 2018 14:43:42 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <05cca7f8-f886-15b4-825c-c4689cdf0c1a@oracle.com> <1cda8fef-3b52-bf9d-6da3-5847e3829520@oracle.com> <6fd5de3c-e377-119d-4614-5e4d9226d32d@oracle.com> Message-ID: <0de362e7-1e78-865e-115b-d69bd3146ee8@oracle.com> On 03/27/2018 02:38 PM, B. Blaser wrote: > On 27 March 2018 at 19:20, Vicente Romero wrote: >> Hi, >> >> On 03/27/2018 11:07 AM, Maurizio Cimadamore wrote: >>> >>> Hi Liam, >>> I believe --debug options are used to generate extra information out of >>> javac, not to alter its behavior - it's probably better to use an hidden >>> flag (e.g. -XDdeduplicateLambdas=...) - which is consistent to what we have >>> done for indy string concat. >>> >> I will introduce this change before pushing the patch, > Should this option really be hidden? It's currently necessary to > disable de-duplication when debugging to preserve all LNT/LVT. > Of course, this could be a '-X...' instead of '-g' but documented, I think. > Note that de-duplication doesn't really alter javac behavior; let say > 'in reverse' that lambda *duplication* would be an additional debug > information... err, I think you have a good point. I have already pushed the patch to: http://hg.openjdk.java.net/jdk/jdk, but we can still rework the option > > Bernard Vicente > >>> Maurizio >>> >> Vicente From cushon at google.com Tue Mar 27 18:51:46 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Tue, 27 Mar 2018 18:51:46 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: On Tue, Mar 27, 2018 at 10:33 AM Vicente Romero wrote: > On 03/27/2018 12:58 PM, Brian Goetz wrote: > > It looks like there were no changes in the outcome, perhaps because > > there were no within-file duplications in the JDK. (Which I > > believe.) A more streams/Rx/CompletableFuture-heavy codebase would > > likely see an improvement. > > right, no difference :(, let's see what happens with Liam's numbers :) > I'm working on getting those numbers now. From cushon at google.com Tue Mar 27 18:54:02 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Tue, 27 Mar 2018 18:54:02 +0000 Subject: deduplicating lambda methods In-Reply-To: <0de362e7-1e78-865e-115b-d69bd3146ee8@oracle.com> References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <05cca7f8-f886-15b4-825c-c4689cdf0c1a@oracle.com> <1cda8fef-3b52-bf9d-6da3-5847e3829520@oracle.com> <6fd5de3c-e377-119d-4614-5e4d9226d32d@oracle.com> <0de362e7-1e78-865e-115b-d69bd3146ee8@oracle.com> Message-ID: On Tue, Mar 27, 2018 at 11:43 AM Vicente Romero wrote: > >>> I believe --debug options are used to generate extra information out of > >>> javac, not to alter its behavior - it's probably better to use an > hidden > >>> flag (e.g. -XDdeduplicateLambdas=...) - which is consistent to what we > have > >>> done for indy string concat. > >>> > >> I will introduce this change before pushing the patch, > > Should this option really be hidden? It's currently necessary to > > disable de-duplication when debugging to preserve all LNT/LVT. > > Of course, this could be a '-X...' instead of '-g' but documented, I > think. > > Note that de-duplication doesn't really alter javac behavior; let say > > 'in reverse' that lambda *duplication* would be an additional debug > > information... > > err, I think you have a good point. I have already pushed the patch to: > http://hg.openjdk.java.net/jdk/jdk, but we can still rework the option > To make sure we're all on the same page, the feature is unconditionally disabled if -g or -g:{lines,vars} is passed, to ensure the LNT/LVT info is correct. The feature can also be disabled with -XDdeduplicateLambdas, but it isn't necessary to explicitly set that flag to preserve LNT/LVT info. The logic for this is here: http://hg.openjdk.java.net/jdk/jdk/rev/752ecccb0b7f#l1.116 From bsrbnd at gmail.com Tue Mar 27 19:04:52 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Tue, 27 Mar 2018 21:04:52 +0200 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> Message-ID: On 27 March 2018 at 01:14, Liam Miller-Cushon wrote: > Hi Vicente, > > I pushed an updated webrev to: > http://cr.openjdk.java.net/~cushon/lambdadedup/webrev.04/ > > [...] > > Finally, I agree with Brian and Louis that we should have stronger > evidence that more sophisticated handling of no-ops would be a win before > adding complexity to support that. Inter-file handling of very common > lambdas > might be a more beneficial future improvement. I think we have to be careful with the stats' interpretations because they often express what we want them to. Real life code-bases are rarely the ones we choose carefully for our statistics. There's often a lack of coding conventions and many duplications (with different coding styles), that's why I suggest to activate this feature with an option depending of programmers' judgments. Of course, inter-file common trivial shapes would be rather straightforward and probably would also bring some good results on quite a lot of code-bases as discussed previously. Bernard From maurizio.cimadamore at oracle.com Tue Mar 27 22:46:13 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 27 Mar 2018 23:46:13 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <05cca7f8-f886-15b4-825c-c4689cdf0c1a@oracle.com> <1cda8fef-3b52-bf9d-6da3-5847e3829520@oracle.com> <6fd5de3c-e377-119d-4614-5e4d9226d32d@oracle.com> <0de362e7-1e78-865e-115b-d69bd3146ee8@oracle.com> Message-ID: <634f8e0a-6054-e330-bd87-7c613fdf2ec9@oracle.com> Right - I think the current mix of option is fine. If people want t use debugging info, they get that with -g, as the dedup machinery is disabled. But in extreme cases where dedup can't be used, and neither can -g (e.g. in tests that are sensitive to bytecode changes), there's an escape hatch. Since it's an escape hatch, I don't think it deserves more than an hidden flag. Maurizio On 27/03/18 19:54, Liam Miller-Cushon wrote: > On Tue, Mar 27, 2018 at 11:43 AM Vicente Romero > > wrote: > > >>> I believe --debug options are used to generate extra > information out of > >>> javac, not to alter its behavior - it's probably better to use > an hidden > >>> flag (e.g. -XDdeduplicateLambdas=...) - which is consistent to > what we have > >>> done for indy string concat. > >>> > >> I will introduce this change before pushing the patch, > > Should this option really be hidden? It's currently necessary to > > disable de-duplication when debugging to preserve all LNT/LVT. > > Of course, this could be a '-X...' instead of '-g' but > documented, I think. > > Note that de-duplication doesn't really alter javac behavior; > let say > > 'in reverse' that lambda *duplication* would be an additional debug > > information... > > err, I think you have a good point. I have already pushed the > patch to: > http://hg.openjdk.java.net/jdk/jdk, but we can still rework the option > > > To make sure we're all on the same page, the feature is unconditionally > disabled if -g or -g:{lines,vars} is passed, to ensure the LNT/LVT info is > correct. The feature can also be disabled with -XDdeduplicateLambdas, > but it isn't necessary to explicitly set that flag to preserve LNT/LVT > info. > > The logic for this is here: > http://hg.openjdk.java.net/jdk/jdk/rev/752ecccb0b7f#l1.116 From bsrbnd at gmail.com Wed Mar 28 15:44:49 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 28 Mar 2018 17:44:49 +0200 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: On 27 March 2018 at 20:22, B. Blaser wrote: > On 27 March 2018 at 19:33, Vicente Romero wrote: >> >> >> On 03/27/2018 12:58 PM, Brian Goetz wrote: >>> >>> It looks like there were no changes in the outcome, perhaps because there >>> were no within-file duplications in the JDK. (Which I believe.) A more >>> streams/Rx/CompletableFuture-heavy codebase would likely see an improvement. >> >> >> right, no difference :(, let's see what happens with Liam's numbers :) > > Did you include the improvement I suggested to compare local variables > *by name* instead of * by symbol* ? > If not, there's quite no chance to find duplicates excepted trivial ones ;-) Field access symbols don't seem to be '==' neither (is that right?). So, to begin with something simple, I would compare all identifiers by name: TreeDiffer.visitIdent: result = tree.sym.name == that.sym.name; TreeDiffer.visitSelect: result = scan(tree.selected, that.selected) && tree.sym.name == that.sym.name; TreeHasher.visitIdent: hash(sym.name); TreeHasher.visitSelect: hash(tree.sym.name); I'll try to write it a bit better, run some more tests and find a couple of examples like: Runnable r1 = () -> { Runnable r2 = () -> {}; }; r1 = () -> { Runnable r2 = () -> {}; }; r1 = () -> { Class c = Integer.class; }; r1 = () -> { Class c = Integer.class; }; What do you think? Thanks, Bernard > Bernard > >> Vicente From maurizio.cimadamore at oracle.com Wed Mar 28 17:44:17 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 28 Mar 2018 18:44:17 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> Uhm - seems to me that the currently implemented logic is more subtle than comparing symbols by name in the visitIdent case? E.g. it (correctly) uses information about the relative position of a lambda parameter in the parameter list and compares that instead of names. And visitSelect is correctly doing a == on the accessed symbol, so that class Foo { String x; } (Foo a) -> a.x (Foo b) -> b.x are deemed equal because: (i) - a and b have same positional info (ii) - a.x and b.x refer to the same (==) symbol (Foo::x) Maurizio On 28/03/18 16:44, B. Blaser wrote: > On 27 March 2018 at 20:22, B. Blaser wrote: >> On 27 March 2018 at 19:33, Vicente Romero wrote: >>> >>> On 03/27/2018 12:58 PM, Brian Goetz wrote: >>>> It looks like there were no changes in the outcome, perhaps because there >>>> were no within-file duplications in the JDK. (Which I believe.) A more >>>> streams/Rx/CompletableFuture-heavy codebase would likely see an improvement. >>> >>> right, no difference :(, let's see what happens with Liam's numbers :) >> Did you include the improvement I suggested to compare local variables >> *by name* instead of * by symbol* ? >> If not, there's quite no chance to find duplicates excepted trivial ones ;-) > Field access symbols don't seem to be '==' neither (is that right?). > So, to begin with something simple, I would compare all identifiers by name: > > TreeDiffer.visitIdent: > result = tree.sym.name == that.sym.name; > > TreeDiffer.visitSelect: > result = scan(tree.selected, that.selected) && tree.sym.name == > that.sym.name; > > TreeHasher.visitIdent: > hash(sym.name); > > TreeHasher.visitSelect: > hash(tree.sym.name); > > I'll try to write it a bit better, run some more tests and find a > couple of examples like: > > Runnable r1 = () -> { > Runnable r2 = () -> {}; > }; > r1 = () -> { > Runnable r2 = () -> {}; > }; > > r1 = () -> { > Class c = Integer.class; > }; > r1 = () -> { > Class c = Integer.class; > }; > > What do you think? > > Thanks, > Bernard > > >> Bernard >> >>> Vicente From bsrbnd at gmail.com Wed Mar 28 17:44:28 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 28 Mar 2018 19:44:28 +0200 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: On 28 March 2018 at 17:44, B. Blaser wrote: > On 27 March 2018 at 20:22, B. Blaser wrote: >> On 27 March 2018 at 19:33, Vicente Romero wrote: >>> >>> >>> On 03/27/2018 12:58 PM, Brian Goetz wrote: >>>> >>>> It looks like there were no changes in the outcome, perhaps because there >>>> were no within-file duplications in the JDK. (Which I believe.) A more >>>> streams/Rx/CompletableFuture-heavy codebase would likely see an improvement. >>> >>> >>> right, no difference :(, let's see what happens with Liam's numbers :) >> >> Did you include the improvement I suggested to compare local variables >> *by name* instead of * by symbol* ? >> If not, there's quite no chance to find duplicates excepted trivial ones ;-) > > Field access symbols don't seem to be '==' neither (is that right?). > So, to begin with something simple, I would compare all identifiers by name: > > TreeDiffer.visitIdent: > result = tree.sym.name == that.sym.name; > > TreeDiffer.visitSelect: > result = scan(tree.selected, that.selected) && tree.sym.name == > that.sym.name; > > TreeHasher.visitIdent: > hash(sym.name); > > TreeHasher.visitSelect: > hash(tree.sym.name); Here is the full suggested improvement. Bernard diff -r 9925be430918 src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java Wed Mar 28 14:24:17 2018 +0100 +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeDiffer.java Wed Mar 28 19:32:16 2018 +0200 @@ -178,13 +178,13 @@ return; } } - result = tree.sym == that.sym; + result = tree.sym.name == that.sym.name; } @Override public void visitSelect(JCFieldAccess tree) { JCFieldAccess that = (JCFieldAccess) parameter; - result = scan(tree.selected, that.selected) && tree.sym == that.sym; + result = scan(tree.selected, that.selected) && tree.sym.name == that.sym.name; } @Override diff -r 9925be430918 src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java Wed Mar 28 14:24:17 2018 +0100 +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TreeHasher.java Wed Mar 28 19:32:16 2018 +0200 @@ -91,12 +91,12 @@ return; } } - hash(sym); + hash(sym.name); } @Override public void visitSelect(JCFieldAccess tree) { - hash(tree.sym); + hash(tree.sym.name); super.visitSelect(tree); } } > I'll try to write it a bit better, run some more tests and find a > couple of examples like: > > Runnable r1 = () -> { > Runnable r2 = () -> {}; > }; > r1 = () -> { > Runnable r2 = () -> {}; > }; > > r1 = () -> { > Class c = Integer.class; > }; > r1 = () -> { > Class c = Integer.class; > }; > > What do you think? > > Thanks, > Bernard > > >> Bernard >> >>> Vicente From bsrbnd at gmail.com Wed Mar 28 17:58:10 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 28 Mar 2018 19:58:10 +0200 Subject: deduplicating lambda methods In-Reply-To: <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> Message-ID: On 28 March 2018 at 19:44, Maurizio Cimadamore wrote: > Uhm - seems to me that the currently implemented logic is more subtle than > comparing symbols by name in the visitIdent case? E.g. it (correctly) uses > information about the relative position of a lambda parameter in the > parameter list and compares that instead of names. And visitSelect is > correctly doing a == on the accessed symbol, so that > > class Foo { String x; } > > > (Foo a) -> a.x > > (Foo b) -> b.x > > are deemed equal because: > > (i) - a and b have same positional info > (ii) - a.x and b.x refer to the same (==) symbol (Foo::x) I missed this e-mail but I just sent 5 minutes ago the full patch which preserve relative positions of lambda parameters, of course. If you try the two examples I sent previously without the patch, you'll see that: r1 = () -> { Class c = Integer.class; }; isn't de-duplicated because occurrences of Integer.class aren't '==' and: r1 = () -> { Runnable r2 = () -> {}; }; isn't de-duplicated because calls to lambda meta-factory aren't '=='. Bernard > Maurizio > > > > On 28/03/18 16:44, B. Blaser wrote: >> >> On 27 March 2018 at 20:22, B. Blaser wrote: >>> >>> On 27 March 2018 at 19:33, Vicente Romero >>> wrote: >>>> >>>> >>>> On 03/27/2018 12:58 PM, Brian Goetz wrote: >>>>> >>>>> It looks like there were no changes in the outcome, perhaps because >>>>> there >>>>> were no within-file duplications in the JDK. (Which I believe.) A >>>>> more >>>>> streams/Rx/CompletableFuture-heavy codebase would likely see an >>>>> improvement. >>>> >>>> >>>> right, no difference :(, let's see what happens with Liam's numbers :) >>> >>> Did you include the improvement I suggested to compare local variables >>> *by name* instead of * by symbol* ? >>> If not, there's quite no chance to find duplicates excepted trivial ones >>> ;-) >> >> Field access symbols don't seem to be '==' neither (is that right?). >> So, to begin with something simple, I would compare all identifiers by >> name: >> >> TreeDiffer.visitIdent: >> result = tree.sym.name == that.sym.name; >> >> TreeDiffer.visitSelect: >> result = scan(tree.selected, that.selected) && tree.sym.name == >> that.sym.name; >> >> TreeHasher.visitIdent: >> hash(sym.name); >> >> TreeHasher.visitSelect: >> hash(tree.sym.name); >> >> I'll try to write it a bit better, run some more tests and find a >> couple of examples like: >> >> Runnable r1 = () -> { >> Runnable r2 = () -> {}; >> }; >> r1 = () -> { >> Runnable r2 = () -> {}; >> }; >> >> r1 = () -> { >> Class c = Integer.class; >> }; >> r1 = () -> { >> Class c = Integer.class; >> }; >> >> What do you think? >> >> Thanks, >> Bernard >> >> >>> Bernard >>> >>>> Vicente > > From maurizio.cimadamore at oracle.com Wed Mar 28 18:09:06 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 28 Mar 2018 19:09:06 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> Message-ID: On 28/03/18 18:58, B. Blaser wrote: > On 28 March 2018 at 19:44, Maurizio Cimadamore > wrote: >> Uhm - seems to me that the currently implemented logic is more subtle than >> comparing symbols by name in the visitIdent case? E.g. it (correctly) uses >> information about the relative position of a lambda parameter in the >> parameter list and compares that instead of names. And visitSelect is >> correctly doing a == on the accessed symbol, so that >> >> class Foo { String x; } >> >> >> (Foo a) -> a.x >> >> (Foo b) -> b.x >> >> are deemed equal because: >> >> (i) - a and b have same positional info >> (ii) - a.x and b.x refer to the same (==) symbol (Foo::x) > I missed this e-mail but I just sent 5 minutes ago the full patch > which preserve relative positions of lambda parameters, of course. > If you try the two examples I sent previously without the patch, > you'll see that: > r1 = () -> { > Class c = Integer.class; > }; > > isn't de-duplicated because occurrences of Integer.class aren't '==' and: > > r1 = () -> { > Runnable r2 = () -> {}; > }; > > isn't de-duplicated because calls to lambda meta-factory aren't '=='. To me this seems to suggest that, if we want to get better results out of the dedup machinery, we have to up our lowering game, so that we don't do excessive generation of fresh symbols. Right now we don't cache .class symbols and also metafactory calls with same static args and BSM - and that's the problem you are seeing, I believe. Maurizio > > Bernard > >> Maurizio >> >> >> >> On 28/03/18 16:44, B. Blaser wrote: >>> On 27 March 2018 at 20:22, B. Blaser wrote: >>>> On 27 March 2018 at 19:33, Vicente Romero >>>> wrote: >>>>> >>>>> On 03/27/2018 12:58 PM, Brian Goetz wrote: >>>>>> It looks like there were no changes in the outcome, perhaps because >>>>>> there >>>>>> were no within-file duplications in the JDK. (Which I believe.) A >>>>>> more >>>>>> streams/Rx/CompletableFuture-heavy codebase would likely see an >>>>>> improvement. >>>>> >>>>> right, no difference :(, let's see what happens with Liam's numbers :) >>>> Did you include the improvement I suggested to compare local variables >>>> *by name* instead of * by symbol* ? >>>> If not, there's quite no chance to find duplicates excepted trivial ones >>>> ;-) >>> Field access symbols don't seem to be '==' neither (is that right?). >>> So, to begin with something simple, I would compare all identifiers by >>> name: >>> >>> TreeDiffer.visitIdent: >>> result = tree.sym.name == that.sym.name; >>> >>> TreeDiffer.visitSelect: >>> result = scan(tree.selected, that.selected) && tree.sym.name == >>> that.sym.name; >>> >>> TreeHasher.visitIdent: >>> hash(sym.name); >>> >>> TreeHasher.visitSelect: >>> hash(tree.sym.name); >>> >>> I'll try to write it a bit better, run some more tests and find a >>> couple of examples like: >>> >>> Runnable r1 = () -> { >>> Runnable r2 = () -> {}; >>> }; >>> r1 = () -> { >>> Runnable r2 = () -> {}; >>> }; >>> >>> r1 = () -> { >>> Class c = Integer.class; >>> }; >>> r1 = () -> { >>> Class c = Integer.class; >>> }; >>> >>> What do you think? >>> >>> Thanks, >>> Bernard >>> >>> >>>> Bernard >>>> >>>>> Vicente >> From bsrbnd at gmail.com Wed Mar 28 18:32:43 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 28 Mar 2018 20:32:43 +0200 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> Message-ID: On 28 March 2018 at 20:09, Maurizio Cimadamore wrote: > > > On 28/03/18 18:58, B. Blaser wrote: >> >> On 28 March 2018 at 19:44, Maurizio Cimadamore >> wrote: >>> >>> Uhm - seems to me that the currently implemented logic is more subtle >>> than >>> comparing symbols by name in the visitIdent case? E.g. it (correctly) >>> uses >>> information about the relative position of a lambda parameter in the >>> parameter list and compares that instead of names. And visitSelect is >>> correctly doing a == on the accessed symbol, so that >>> >>> class Foo { String x; } >>> >>> >>> (Foo a) -> a.x >>> >>> (Foo b) -> b.x >>> >>> are deemed equal because: >>> >>> (i) - a and b have same positional info >>> (ii) - a.x and b.x refer to the same (==) symbol (Foo::x) >> >> I missed this e-mail but I just sent 5 minutes ago the full patch >> which preserve relative positions of lambda parameters, of course. >> If you try the two examples I sent previously without the patch, >> you'll see that: >> r1 = () -> { >> Class c = Integer.class; >> }; >> >> isn't de-duplicated because occurrences of Integer.class aren't '==' and: >> >> r1 = () -> { >> Runnable r2 = () -> {}; >> }; >> >> isn't de-duplicated because calls to lambda meta-factory aren't '=='. > > To me this seems to suggest that, if we want to get better results out of > the dedup machinery, we have to up our lowering game, so that we don't do > excessive generation of fresh symbols. Right now we don't cache .class > symbols and also metafactory calls with same static args and BSM - and > that's the problem you are seeing, I believe. Note that the fix I suggest also addresses local variable symbols which aren't '=='. Bernard > Maurizio > >> >> Bernard >> >>> Maurizio From bsrbnd at gmail.com Wed Mar 28 21:51:39 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 28 Mar 2018 23:51:39 +0200 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> Message-ID: On 28 March 2018 at 20:09, Maurizio Cimadamore wrote: > > > On 28/03/18 18:58, B. Blaser wrote: >> >> On 28 March 2018 at 19:44, Maurizio Cimadamore >> wrote: >>> >>> Uhm - seems to me that the currently implemented logic is more subtle >>> than >>> comparing symbols by name in the visitIdent case? E.g. it (correctly) >>> uses >>> information about the relative position of a lambda parameter in the >>> parameter list and compares that instead of names. And visitSelect is >>> correctly doing a == on the accessed symbol, so that >>> >>> class Foo { String x; } >>> >>> >>> (Foo a) -> a.x >>> >>> (Foo b) -> b.x >>> >>> are deemed equal because: >>> >>> (i) - a and b have same positional info >>> (ii) - a.x and b.x refer to the same (==) symbol (Foo::x) >> >> I missed this e-mail but I just sent 5 minutes ago the full patch >> which preserve relative positions of lambda parameters, of course. >> If you try the two examples I sent previously without the patch, >> you'll see that: >> r1 = () -> { >> Class c = Integer.class; >> }; >> >> isn't de-duplicated because occurrences of Integer.class aren't '==' and: >> >> r1 = () -> { >> Runnable r2 = () -> {}; >> }; >> >> isn't de-duplicated because calls to lambda meta-factory aren't '=='. > > To me this seems to suggest that, if we want to get better results out of > the dedup machinery, we have to up our lowering game, so that we don't do > excessive generation of fresh symbols. Right now we don't cache .class > symbols and also metafactory calls with same static args and BSM - and > that's the problem you are seeing, I believe. You're right. The fix below tries to cache meta-factory calls, solving the following de-duplication problem: r1 = () -> { Runnable r2 = () -> {}; }; Bernard diff -r 9925be430918 src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java Wed Mar 28 14:24:17 2018 +0100 +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java Wed Mar 28 23:24:59 2018 +0200 @@ -226,6 +226,8 @@ private Map dedupedLambdas; + private Map dynMethSyms = new HashMap<>(); + /** * list of deserialization cases */ @@ -1220,7 +1222,8 @@ staticArgs.toArray()); JCFieldAccess qualifier = make.Select(make.QualIdent(site.tsym), bsmName); - qualifier.sym = dynSym; + DynamicMethodSymbol existing = kInfo.dynMethSyms.putIfAbsent(methName, dynSym); + qualifier.sym = existing != null ? existing : dynSym; qualifier.type = indyType.getReturnType(); JCMethodInvocation proxyCall = make.Apply(List.nil(), qualifier, indyArgs); > Maurizio > >> >> Bernard >> >>> Maurizio From bsrbnd at gmail.com Thu Mar 29 10:32:16 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Thu, 29 Mar 2018 12:32:16 +0200 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> Message-ID: On 28 March 2018 at 23:51, B. Blaser wrote: > On 28 March 2018 at 20:09, Maurizio Cimadamore > wrote: >> >> To me this seems to suggest that, if we want to get better results out of >> the dedup machinery, we have to up our lowering game, so that we don't do >> excessive generation of fresh symbols. Right now we don't cache .class >> symbols and also metafactory calls with same static args and BSM - and >> that's the problem you are seeing, I believe. > > You're right. The fix below tries to cache meta-factory calls, solving > the following de-duplication problem: > r1 = () -> { > Runnable r2 = () -> {}; > }; Running javac tests, I note that the cache key isn't precise enough. Any idea of what a good key would be? Bernard > Bernard > > diff -r 9925be430918 > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > Wed Mar 28 14:24:17 2018 +0100 > +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > Wed Mar 28 23:24:59 2018 +0200 > @@ -226,6 +226,8 @@ > > private Map dedupedLambdas; > > + private Map dynMethSyms = new HashMap<>(); > + > /** > * list of deserialization cases > */ > @@ -1220,7 +1222,8 @@ > staticArgs.toArray()); > > JCFieldAccess qualifier = > make.Select(make.QualIdent(site.tsym), bsmName); > - qualifier.sym = dynSym; > + DynamicMethodSymbol existing = > kInfo.dynMethSyms.putIfAbsent(methName, dynSym); > + qualifier.sym = existing != null ? existing : dynSym; > qualifier.type = indyType.getReturnType(); > > JCMethodInvocation proxyCall = make.Apply(List.nil(), > qualifier, indyArgs); > >> Maurizio >> >>> >>> Bernard >>> >>>> Maurizio From maurizio.cimadamore at oracle.com Thu Mar 29 14:00:12 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 29 Mar 2018 15:00:12 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> Message-ID: <6daf4c8e-1af5-2d76-cd93-82038894bbac@oracle.com> On 29/03/18 11:32, B. Blaser wrote: > On 28 March 2018 at 23:51, B. Blaser wrote: >> On 28 March 2018 at 20:09, Maurizio Cimadamore >> wrote: >>> To me this seems to suggest that, if we want to get better results out of >>> the dedup machinery, we have to up our lowering game, so that we don't do >>> excessive generation of fresh symbols. Right now we don't cache .class >>> symbols and also metafactory calls with same static args and BSM - and >>> that's the problem you are seeing, I believe. >> You're right. The fix below tries to cache meta-factory calls, solving >> the following de-duplication problem: >> r1 = () -> { >> Runnable r2 = () -> {}; >> }; > Running javac tests, I note that the cache key isn't precise enough. > Any idea of what a good key would be? I think the key has to take into account: * name and type of BSM method * dynamic type of the indy call * values of all static args just using the lambda impl name would not cut it, esp. if deduplication is turned on (as the same lambda impl will be reused e.g. for a Function and a Function - of the underlying lambda is the identity x -> x ). Maurizio > > Bernard > > >> Bernard >> >> diff -r 9925be430918 >> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >> --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >> Wed Mar 28 14:24:17 2018 +0100 >> +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >> Wed Mar 28 23:24:59 2018 +0200 >> @@ -226,6 +226,8 @@ >> >> private Map dedupedLambdas; >> >> + private Map dynMethSyms = new HashMap<>(); >> + >> /** >> * list of deserialization cases >> */ >> @@ -1220,7 +1222,8 @@ >> staticArgs.toArray()); >> >> JCFieldAccess qualifier = >> make.Select(make.QualIdent(site.tsym), bsmName); >> - qualifier.sym = dynSym; >> + DynamicMethodSymbol existing = >> kInfo.dynMethSyms.putIfAbsent(methName, dynSym); >> + qualifier.sym = existing != null ? existing : dynSym; >> qualifier.type = indyType.getReturnType(); >> >> JCMethodInvocation proxyCall = make.Apply(List.nil(), >> qualifier, indyArgs); >> >>> Maurizio >>> >>>> Bernard >>>> >>>>> Maurizio From maurizio.cimadamore at oracle.com Thu Mar 29 14:04:15 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 29 Mar 2018 15:04:15 +0100 Subject: deduplicating lambda methods In-Reply-To: <6daf4c8e-1af5-2d76-cd93-82038894bbac@oracle.com> References: <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> <6daf4c8e-1af5-2d76-cd93-82038894bbac@oracle.com> Message-ID: In other words, what I'm saying is that the key should be DynamicMethodSymbol, and that we should avoid creating multiple DMS for callsites which have the same properties. That reflects also invariant in javac: symbols are shared (where possible). A similar point holds for class literals, which are modeled as field access to a fictional symbol called 'class' - we should no creating that fictional symbol multiple times, there should be some shared Scope or similar structure somewhere which, given a class C, gives you back the .class symbol associated with C. Maurizio On 29/03/18 15:00, Maurizio Cimadamore wrote: > > > On 29/03/18 11:32, B. Blaser wrote: >> On 28 March 2018 at 23:51, B. Blaser wrote: >>> On 28 March 2018 at 20:09, Maurizio Cimadamore >>> wrote: >>>> To me this seems to suggest that, if we want to get better results >>>> out of >>>> the dedup machinery, we have to up our lowering game, so that we >>>> don't do >>>> excessive generation of fresh symbols. Right now we don't cache .class >>>> symbols and also metafactory calls with same static args and BSM - and >>>> that's the problem you are seeing, I believe. >>> You're right. The fix below tries to cache meta-factory calls, solving >>> the following de-duplication problem: >>> ???? r1 = () -> { >>> ???????? Runnable r2 = () -> {}; >>> ???? }; >> Running javac tests, I note that the cache key isn't precise enough. >> Any idea of what a good key would be? > I think the key has to take into account: > > * name and type of BSM method > * dynamic type of the indy call > * values of all static args > > just using the lambda impl name would not cut it, esp. if > deduplication is turned on (as the same lambda impl will be reused > e.g. for a Function and a Function - > of the underlying lambda is the identity x -> x ). > > Maurizio >> >> Bernard >> >> >>> Bernard >>> >>> diff -r 9925be430918 >>> src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >>> >>> --- >>> a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >>> ??? Wed Mar 28 14:24:17 2018 +0100 >>> +++ >>> b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java >>> ??? Wed Mar 28 23:24:59 2018 +0200 >>> @@ -226,6 +226,8 @@ >>> >>> ????????? private Map dedupedLambdas; >>> >>> +??????? private Map dynMethSyms = new >>> HashMap<>(); >>> + >>> ????????? /** >>> ?????????? * list of deserialization cases >>> ?????????? */ >>> @@ -1220,7 +1222,8 @@ >>> staticArgs.toArray()); >>> >>> ????????????? JCFieldAccess qualifier = >>> make.Select(make.QualIdent(site.tsym), bsmName); >>> -??????????? qualifier.sym = dynSym; >>> +??????????? DynamicMethodSymbol existing = >>> kInfo.dynMethSyms.putIfAbsent(methName, dynSym); >>> +??????????? qualifier.sym = existing != null ? existing : dynSym; >>> ????????????? qualifier.type = indyType.getReturnType(); >>> >>> ????????????? JCMethodInvocation proxyCall = make.Apply(List.nil(), >>> qualifier, indyArgs); >>> >>>> Maurizio >>>> >>>>> Bernard >>>>> >>>>>> Maurizio > From bsrbnd at gmail.com Thu Mar 29 18:37:50 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Thu, 29 Mar 2018 20:37:50 +0200 Subject: deduplicating lambda methods In-Reply-To: References: <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> <6daf4c8e-1af5-2d76-cd93-82038894bbac@oracle.com> Message-ID: On 29 March 2018 at 16:04, Maurizio Cimadamore wrote: > In other words, what I'm saying is that the key should be > DynamicMethodSymbol, and that we should avoid creating multiple DMS for > callsites which have the same properties. That reflects also invariant in > javac: symbols are shared (where possible). A similar point holds for class > literals, which are modeled as field access to a fictional symbol called > 'class' - we should no creating that fictional symbol multiple times, there > should be some shared Scope or similar structure somewhere which, given a > class C, gives you back the .class symbol associated with C. > > Maurizio I was expecting a simpler key but I think you're right. Here is a patch with the full DMS as key. All 'javac/lambda' tests are passing successfully, I'll run the others soon... Bernard diff -r 9925be430918 src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java Wed Mar 28 14:24:17 2018 +0100 +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java Thu Mar 29 20:09:45 2018 +0200 @@ -226,6 +226,8 @@ private Map dedupedLambdas; + private Map dynMethSyms = new HashMap<>(); + /** * list of deserialization cases */ @@ -1218,9 +1220,10 @@ (MethodSymbol)bsm, indyType, staticArgs.toArray()); - JCFieldAccess qualifier = make.Select(make.QualIdent(site.tsym), bsmName); - qualifier.sym = dynSym; + DynamicMethodSymbol existing = kInfo.dynMethSyms.putIfAbsent( + new DynMethSymKey(dynSym), dynSym); + qualifier.sym = existing != null ? existing : dynSym; qualifier.type = indyType.getReturnType(); JCMethodInvocation proxyCall = make.Apply(List.nil(), qualifier, indyArgs); @@ -1231,6 +1234,63 @@ } } //where + private class DynMethSymKey { + private DynamicMethodSymbol sym; + + public DynMethSymKey(DynamicMethodSymbol sym) { + this.sym = sym; + } + + @Override + public int hashCode() { + return sym.name.hashCode() | + sym.owner.hashCode() << 8 | + ((Integer)sym.bsmKind).hashCode() << 16 | + sym.bsm.hashCode() << 24; + } + + @Override + public boolean equals(Object o) { + if ( !(o instanceof DynMethSymKey) ) + return false; + + DynMethSymKey that = (DynMethSymKey) o; + boolean equals = true; + + // name, type, owner + equals &= sym.name == that.sym.name; + equals &= types.isSameType(sym.type, that.sym.type); + equals &= sym.owner == that.sym.owner; + + // bsmKind, bsm + equals &= sym.bsmKind == that.sym.bsmKind; + equals &= sym.bsm == that.sym.bsm; + + // staticArgs + if (sym.staticArgs.length != that.sym.staticArgs.length) + return false; + + for (int i=0; i bsmStaticArgToTypes(List args) { ListBuffer argtypes = new ListBuffer<>(); for (Object arg : args) { From maurizio.cimadamore at oracle.com Thu Mar 29 20:02:21 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 29 Mar 2018 21:02:21 +0100 Subject: deduplicating lambda methods In-Reply-To: References: <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> <6daf4c8e-1af5-2d76-cd93-82038894bbac@oracle.com> Message-ID: <441e0be7-6ef0-3782-139b-73916b1f6c01@oracle.com> I wonder if there could be a way to reuse the logic in Pool::DynamicMethod - that code is essentially doing the same thing! Maurizio On 29/03/18 19:37, B. Blaser wrote: > On 29 March 2018 at 16:04, Maurizio Cimadamore > wrote: >> In other words, what I'm saying is that the key should be >> DynamicMethodSymbol, and that we should avoid creating multiple DMS for >> callsites which have the same properties. That reflects also invariant in >> javac: symbols are shared (where possible). A similar point holds for class >> literals, which are modeled as field access to a fictional symbol called >> 'class' - we should no creating that fictional symbol multiple times, there >> should be some shared Scope or similar structure somewhere which, given a >> class C, gives you back the .class symbol associated with C. >> >> Maurizio > I was expecting a simpler key but I think you're right. > Here is a patch with the full DMS as key. > All 'javac/lambda' tests are passing successfully, I'll run the others soon... > > Bernard > > > diff -r 9925be430918 > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > Wed Mar 28 14:24:17 2018 +0100 > +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > Thu Mar 29 20:09:45 2018 +0200 > @@ -226,6 +226,8 @@ > > private Map dedupedLambdas; > > + private Map dynMethSyms = > new HashMap<>(); > + > /** > * list of deserialization cases > */ > @@ -1218,9 +1220,10 @@ > (MethodSymbol)bsm, > indyType, > staticArgs.toArray()); > - > JCFieldAccess qualifier = > make.Select(make.QualIdent(site.tsym), bsmName); > - qualifier.sym = dynSym; > + DynamicMethodSymbol existing = kInfo.dynMethSyms.putIfAbsent( > + new DynMethSymKey(dynSym), dynSym); > + qualifier.sym = existing != null ? existing : dynSym; > qualifier.type = indyType.getReturnType(); > > JCMethodInvocation proxyCall = make.Apply(List.nil(), > qualifier, indyArgs); > @@ -1231,6 +1234,63 @@ > } > } > //where > + private class DynMethSymKey { > + private DynamicMethodSymbol sym; > + > + public DynMethSymKey(DynamicMethodSymbol sym) { > + this.sym = sym; > + } > + > + @Override > + public int hashCode() { > + return sym.name.hashCode() | > + sym.owner.hashCode() << 8 | > + ((Integer)sym.bsmKind).hashCode() << 16 | > + sym.bsm.hashCode() << 24; > + } > + > + @Override > + public boolean equals(Object o) { > + if ( !(o instanceof DynMethSymKey) ) > + return false; > + > + DynMethSymKey that = (DynMethSymKey) o; > + boolean equals = true; > + > + // name, type, owner > + equals &= sym.name == that.sym.name; > + equals &= types.isSameType(sym.type, that.sym.type); > + equals &= sym.owner == that.sym.owner; > + > + // bsmKind, bsm > + equals &= sym.bsmKind == that.sym.bsmKind; > + equals &= sym.bsm == that.sym.bsm; > + > + // staticArgs > + if (sym.staticArgs.length != that.sym.staticArgs.length) > + return false; > + > + for (int i=0; i + equals &= equalsStaticArg(sym.staticArgs[i], > that.sym.staticArgs[i]); > + > + return equals; > + } > + > + private boolean equalsStaticArg(Object arg1, Object arg2) { > + Assert.checkNonNull(arg1); > + Assert.checkNonNull(arg2); > + if (arg1 instanceof Number && arg2 instanceof Number > + || arg1 instanceof String && arg2 instanceof String > + || arg1 instanceof Pool.MethodHandle && arg2 > instanceof Pool.MethodHandle) { > + return arg1.equals(arg2); > + } else if (arg1 instanceof MethodType && arg2 instanceof > MethodType) { > + return types.isSameType((MethodType)arg1, (MethodType)arg2); > + } else { > + return false; > + } > + } > + } > + > private List bsmStaticArgToTypes(List args) { > ListBuffer argtypes = new ListBuffer<>(); > for (Object arg : args) { From bsrbnd at gmail.com Fri Mar 30 13:12:01 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Fri, 30 Mar 2018 15:12:01 +0200 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: On 27 March 2018 at 19:33, Vicente Romero wrote: > > > On 03/27/2018 12:58 PM, Brian Goetz wrote: >> >> It looks like there were no changes in the outcome, perhaps because there >> were no within-file duplications in the JDK. (Which I believe.) A more >> streams/Rx/CompletableFuture-heavy codebase would likely see an improvement. > > > right, no difference :(, let's see what happens with Liam's numbers :) > > Vicente I perhaps found at least one trivial lambda duplicate in the JDK (javac) here: http://hg.openjdk.java.net/jdk/jdk/file/814bd31f8da0/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java#l450 I tried to reproduce it like this (is that right?): import java.util.List; import java.util.Set; public class Test { class CaputureScanner extends SimpleVisitor { public Void visitClassType(Type.ClassType t, Set seen) { if ( t.isCompound() ) { directSupertypes(t).forEach(s -> visit(s, seen)); } else { t.allparams().forEach(ta -> visit(ta, seen)); } return null; } } public static class SimpleVisitor { public Void visit(Type t, Set seen) { return null; } } public static class Type { public boolean isCompound() { return false; } public static class ClassType extends Type { public List allparams() { return null; } } } public List directSupertypes(Type t) { return null; } } which is correctly de-duplicated. But stats don't show any difference, are they missing something? Bernard From vicente.romero at oracle.com Fri Mar 30 13:27:26 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 30 Mar 2018 09:27:26 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: <76838b11-28b0-af6c-40fa-4e92b4dbb3c1@oracle.com> Hi Bernard, Thanks for the report I will take a look, Vicente On 03/30/2018 09:12 AM, B. Blaser wrote: > On 27 March 2018 at 19:33, Vicente Romero wrote: >> >> On 03/27/2018 12:58 PM, Brian Goetz wrote: >>> It looks like there were no changes in the outcome, perhaps because there >>> were no within-file duplications in the JDK. (Which I believe.) A more >>> streams/Rx/CompletableFuture-heavy codebase would likely see an improvement. >> >> right, no difference :(, let's see what happens with Liam's numbers :) >> >> Vicente > I perhaps found at least one trivial lambda duplicate in the JDK (javac) here: > > http://hg.openjdk.java.net/jdk/jdk/file/814bd31f8da0/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java#l450 > > I tried to reproduce it like this (is that right?): > > import java.util.List; > import java.util.Set; > > public class Test { > class CaputureScanner extends SimpleVisitor { > public Void visitClassType(Type.ClassType t, Set seen) { > if ( t.isCompound() ) { > directSupertypes(t).forEach(s -> visit(s, seen)); > } else { > t.allparams().forEach(ta -> visit(ta, seen)); > } > return null; > } > } > public static class SimpleVisitor { > public Void visit(Type t, Set seen) { return null; } > } > public static class Type { > public boolean isCompound() { return false; } > > public static class ClassType extends Type { > public List allparams() { return null; } > } > } > public List directSupertypes(Type t) { return null; } > } > > which is correctly de-duplicated. > > But stats don't show any difference, are they missing something? > > Bernard From bsrbnd at gmail.com Fri Mar 30 16:29:26 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Fri, 30 Mar 2018 18:29:26 +0200 Subject: deduplicating lambda methods In-Reply-To: <441e0be7-6ef0-3782-139b-73916b1f6c01@oracle.com> References: <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> <6daf4c8e-1af5-2d76-cd93-82038894bbac@oracle.com> <441e0be7-6ef0-3782-139b-73916b1f6c01@oracle.com> Message-ID: On 29 March 2018 at 22:02, Maurizio Cimadamore wrote: > I wonder if there could be a way to reuse the logic in Pool::DynamicMethod - > that code is essentially doing the same thing! > > Maurizio Yes (as next), great! All lambda tests are passing successfully but we'll have to re-run the others which seem OK with the previous key. Bernard diff -r 9925be430918 src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java Wed Mar 28 14:24:17 2018 +0100 +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java Fri Mar 30 18:11:49 2018 +0200 @@ -67,6 +67,7 @@ import static com.sun.tools.javac.code.Kinds.Kind.*; import static com.sun.tools.javac.code.TypeTag.*; import static com.sun.tools.javac.tree.JCTree.Tag.*; +import static com.sun.tools.javac.jvm.Pool.DynamicMethod; import javax.lang.model.element.ElementKind; import javax.lang.model.type.TypeKind; @@ -226,6 +227,8 @@ private Map dedupedLambdas; + private Map dynMethSyms = new HashMap<>(); + /** * list of deserialization cases */ @@ -1218,9 +1221,10 @@ (MethodSymbol)bsm, indyType, staticArgs.toArray()); - JCFieldAccess qualifier = make.Select(make.QualIdent(site.tsym), bsmName); - qualifier.sym = dynSym; + DynamicMethodSymbol existing = kInfo.dynMethSyms.putIfAbsent( + new DynamicMethod(dynSym, types), dynSym); + qualifier.sym = existing != null ? existing : dynSym; qualifier.type = indyType.getReturnType(); JCMethodInvocation proxyCall = make.Apply(List.nil(), qualifier, indyArgs); diff -r 9925be430918 src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java Wed Mar 28 14:24:17 2018 +0100 +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java Fri Mar 30 18:11:49 2018 +0200 @@ -180,10 +180,10 @@ } } - static class DynamicMethod extends Method { + public static class DynamicMethod extends Method { public Object[] uniqueStaticArgs; - DynamicMethod(DynamicMethodSymbol m, Types types) { + public DynamicMethod(DynamicMethodSymbol m, Types types) { super(m, types); uniqueStaticArgs = getUniqueTypeArray(m.staticArgs, types); } From vicente.romero at oracle.com Fri Mar 30 17:03:12 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 30 Mar 2018 13:03:12 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: The stats application is able to see, and report, the difference in the class files with the reproductor you sent, but it seems like there is no lambda deduplication when Types is compiled. I have been checking the class file obtained for Types.CaptureScanner and there is no deduplication there, two lambda methods are generated. This will need further research, Vicente On 03/30/2018 09:12 AM, B. Blaser wrote: > On 27 March 2018 at 19:33, Vicente Romero wrote: >> >> On 03/27/2018 12:58 PM, Brian Goetz wrote: >>> It looks like there were no changes in the outcome, perhaps because there >>> were no within-file duplications in the JDK. (Which I believe.) A more >>> streams/Rx/CompletableFuture-heavy codebase would likely see an improvement. >> >> right, no difference :(, let's see what happens with Liam's numbers :) >> >> Vicente > I perhaps found at least one trivial lambda duplicate in the JDK (javac) here: > > http://hg.openjdk.java.net/jdk/jdk/file/814bd31f8da0/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java#l450 > > I tried to reproduce it like this (is that right?): > > import java.util.List; > import java.util.Set; > > public class Test { > class CaputureScanner extends SimpleVisitor { > public Void visitClassType(Type.ClassType t, Set seen) { > if ( t.isCompound() ) { > directSupertypes(t).forEach(s -> visit(s, seen)); > } else { > t.allparams().forEach(ta -> visit(ta, seen)); > } > return null; > } > } > public static class SimpleVisitor { > public Void visit(Type t, Set seen) { return null; } > } > public static class Type { > public boolean isCompound() { return false; } > > public static class ClassType extends Type { > public List allparams() { return null; } > } > } > public List directSupertypes(Type t) { return null; } > } > > which is correctly de-duplicated. > > But stats don't show any difference, are they missing something? > > Bernard From brian.goetz at oracle.com Fri Mar 30 19:34:06 2018 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Fri, 30 Mar 2018 19:34:06 +0000 Subject: hg: amber/amber: Add java.lang.compiler package; add Extractor support Message-ID: <201803301934.w2UJY6bn011563@aojmv0008.oracle.com> Changeset: 868580e46733 Author: briangoetz Date: 2018-03-30 15:33 -0400 URL: http://hg.openjdk.java.net/amber/amber/rev/868580e46733 Add java.lang.compiler package; add Extractor support + src/java.base/share/classes/java/lang/compiler/Extractor.java + src/java.base/share/classes/java/lang/compiler/ExtractorImpl.java + src/java.base/share/classes/java/lang/compiler/PatternCarriers.java ! src/java.base/share/classes/module-info.java + test/jdk/java/lang/compiler/ExtractorTest.java From cushon at google.com Sat Mar 31 03:03:22 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Sat, 31 Mar 2018 03:03:22 +0000 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: Hmm, if I compile Types.java by itself I'm seeing deduplication occur. I collected some statistics from Google's codebase. In compilation units with >1 lambda, 12.8% of lambdas were deduplicated. That number doesn't tell the whole story about whether deduplication is useful in practice, but if nothing else it lets us measure the relative improvement from changes to the deduplication logic. I also investigated deduplicating lambdas that contain local variable declarations. The implementation uses the approach I mentioned earlier: it keeps a record of variable declarations visited during diffing and hashing, recognizes subsequent uses of those variables, and treats them as equal / hashes them to a stable value. Lambda parameters are now handled using the same logic as locals, instead of as a special case. Handling locals increased the number of deduplicated lambdas by 1.3%. I don't think that's enough to justify adding a lot of complexity here, but the patch actually simplifies some of the existing logic. Here's the webrev: http://cr.openjdk.java.net/~cushon/lambdadedup2/webrev.00/ On Fri, Mar 30, 2018 at 10:03 AM Vicente Romero wrote: > The stats application is able to see, and report, the difference in the > class files with the reproductor you sent, but it seems like there is no > lambda deduplication when Types is compiled. I have been checking the > class file obtained for Types.CaptureScanner and there is no > deduplication there, two lambda methods are generated. This will need > further research, > > Vicente > > On 03/30/2018 09:12 AM, B. Blaser wrote: > > On 27 March 2018 at 19:33, Vicente Romero > wrote: > >> > >> On 03/27/2018 12:58 PM, Brian Goetz wrote: > >>> It looks like there were no changes in the outcome, perhaps because > there > >>> were no within-file duplications in the JDK. (Which I believe.) A > more > >>> streams/Rx/CompletableFuture-heavy codebase would likely see an > improvement. > >> > >> right, no difference :(, let's see what happens with Liam's numbers :) > >> > >> Vicente > > I perhaps found at least one trivial lambda duplicate in the JDK (javac) > here: > > > > > http://hg.openjdk.java.net/jdk/jdk/file/814bd31f8da0/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java#l450 > > > > I tried to reproduce it like this (is that right?): > > > > import java.util.List; > > import java.util.Set; > > > > public class Test { > > class CaputureScanner extends SimpleVisitor { > > public Void visitClassType(Type.ClassType t, Set seen) { > > if ( t.isCompound() ) { > > directSupertypes(t).forEach(s -> visit(s, seen)); > > } else { > > t.allparams().forEach(ta -> visit(ta, seen)); > > } > > return null; > > } > > } > > public static class SimpleVisitor { > > public Void visit(Type t, Set seen) { return null; } > > } > > public static class Type { > > public boolean isCompound() { return false; } > > > > public static class ClassType extends Type { > > public List allparams() { return null; } > > } > > } > > public List directSupertypes(Type t) { return null; } > > } > > > > which is correctly de-duplicated. > > > > But stats don't show any difference, are they missing something? > > > > Bernard > > From brian.goetz at oracle.com Sat Mar 31 14:24:47 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 31 Mar 2018 10:24:47 -0400 Subject: deduplicating lambda methods In-Reply-To: References: <60518a1f-9afe-857b-751f-2285cf4c33d2@oracle.com> <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> Message-ID: > That number doesn't tell the whole story about?whether deduplication > is useful > in practice, but if nothing else it lets us measure the relative > improvement from > changes to the deduplication logic. One reason I encouraged this project is that it is synergistic with other optimizations in the pipeline, namely using ConstantDynamic (condy) to translate non-capturing lambdas and method refs.? This has a direct performance benefit (lighter than indy) and offers more sharing.? Further, a little farther down the road is more aggresive constant propagation, which may turn some capturing lambdas into non-capturing ones (big win already), and further expose them to condy-ization and therefore deduplication. From bsrbnd at gmail.com Sat Mar 31 15:15:24 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sat, 31 Mar 2018 17:15:24 +0200 Subject: deduplicating lambda methods In-Reply-To: References: <7d3471bb-4512-981e-d4c3-d609a9b9e899@oracle.com> <68ca9412-e453-2a58-3153-14c1e12b0bee@oracle.com> <3fa7810b-0d08-3a0e-ee2c-97e1b80ec58e@oracle.com> <6daf4c8e-1af5-2d76-cd93-82038894bbac@oracle.com> <441e0be7-6ef0-3782-139b-73916b1f6c01@oracle.com> Message-ID: On 30 March 2018 at 18:29, B. Blaser wrote: > On 29 March 2018 at 22:02, Maurizio Cimadamore > wrote: >> I wonder if there could be a way to reuse the logic in Pool::DynamicMethod - >> that code is essentially doing the same thing! >> >> Maurizio > > Yes (as next), great! > All lambda tests are passing successfully but we'll have to re-run the > others which seem OK with the previous key. > > Bernard I've also updated the de-duplication test with meta-factory calls inside lambdas, as here under. Bernard diff -r 9925be430918 test/langtools/tools/javac/lambda/deduplication/Deduplication.java --- a/test/langtools/tools/javac/lambda/deduplication/Deduplication.java Wed Mar 28 14:24:17 2018 +0100 +++ b/test/langtools/tools/javac/lambda/deduplication/Deduplication.java Sat Mar 31 16:49:48 2018 +0200 @@ -32,6 +32,12 @@ void group(Object... xs) {} void test() { + + group( + (Runnable) () -> { ( (Runnable) () -> {} ).run(); }, + (Runnable) () -> { ( (Runnable) () -> {} ).run(); } + ); + group((Function) x -> x.hashCode()); group((Function) x -> x.hashCode()); > diff -r 9925be430918 > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > Wed Mar 28 14:24:17 2018 +0100 > +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java > Fri Mar 30 18:11:49 2018 +0200 > @@ -67,6 +67,7 @@ > import static com.sun.tools.javac.code.Kinds.Kind.*; > import static com.sun.tools.javac.code.TypeTag.*; > import static com.sun.tools.javac.tree.JCTree.Tag.*; > +import static com.sun.tools.javac.jvm.Pool.DynamicMethod; > > import javax.lang.model.element.ElementKind; > import javax.lang.model.type.TypeKind; > @@ -226,6 +227,8 @@ > > private Map dedupedLambdas; > > + private Map dynMethSyms = > new HashMap<>(); > + > /** > * list of deserialization cases > */ > @@ -1218,9 +1221,10 @@ > (MethodSymbol)bsm, > indyType, > staticArgs.toArray()); > - > JCFieldAccess qualifier = > make.Select(make.QualIdent(site.tsym), bsmName); > - qualifier.sym = dynSym; > + DynamicMethodSymbol existing = kInfo.dynMethSyms.putIfAbsent( > + new DynamicMethod(dynSym, types), dynSym); > + qualifier.sym = existing != null ? existing : dynSym; > qualifier.type = indyType.getReturnType(); > > JCMethodInvocation proxyCall = make.Apply(List.nil(), > qualifier, indyArgs); > diff -r 9925be430918 > src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java > --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java > Wed Mar 28 14:24:17 2018 +0100 > +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java > Fri Mar 30 18:11:49 2018 +0200 > @@ -180,10 +180,10 @@ > } > } > > - static class DynamicMethod extends Method { > + public static class DynamicMethod extends Method { > public Object[] uniqueStaticArgs; > > - DynamicMethod(DynamicMethodSymbol m, Types types) { > + public DynamicMethod(DynamicMethodSymbol m, Types types) { > super(m, types); > uniqueStaticArgs = getUniqueTypeArray(m.staticArgs, types); > }