From sarma.swaranga at gmail.com Fri Mar 1 00:02:56 2024 From: sarma.swaranga at gmail.com (Swaranga Sarma) Date: Thu, 29 Feb 2024 16:02:56 -0800 Subject: Scope for JEP 468: Derived record creation In-Reply-To: <1e81bd77-6417-4018-8c23-1cef01f2e1f7@oracle.com> References: <1e81bd77-6417-4018-8c23-1cef01f2e1f7@oracle.com> Message-ID: Thank you. >it's a pretty verbose syntax Agreed. I was trying to stay close to the current syntax but even more concise could be: R r = {a: 1, b: 2}. But the question was not about the syntax. > it makes it harder to refactor from records to classes Wouldn't the current JEP also have the same limitations? Although I do see the JEP mention that it may be possible in the future for regular classes. Regards Swaranga On Thu, Feb 29, 2024 at 3:11?PM Brian Goetz wrote: > While such a feature is possible, I am not particularly excited about it > for several reasons. If the goal is "I want to construct using named > instead of position arguments" (which I think is your goal), it's a pretty > verbose syntax; a better one would be > > new R(a: 1, b: 2) > > But that's a much smaller concern. The bigger one is that I don't think > the language is improved by having a named parameter mechanism for records, > but for nothing else (not for instantiating class instances, not for > invoking methods.) While it might seem "better than nothing", having two > different ways to do something, but one of them only works in narrow > situations, is as likely to be frustrating than beneficial. If the payoff > is big enough, then it might be a possibility, but "invocation by parameter > name" is nowhere near that bar. > > Finally, if it works for records but not for classes, it makes it harder > to refactor from records to classes, since there will be use sites that > have to be adjusted. > > So I think for the time being, the answer is "no", though if we run out of > things to work on, we might reconsider. > > > On 2/29/2024 4:34 PM, Swaranga Sarma wrote: > > The JEP looks really promising. However I am wondering if there will be a > separate JEP for creating new records from scratch with similar syntax. > > The current JEP states that its goals are to enable creating records from > an existing record and for that it seems sufficient. But I would also love > to be able to create new records from scratch using the same syntax. > Something like: > > var circle = new Circle with { > radius = 0.5f; > center = new Center with { > x = 0; > y = -1; > z = 8; > }; > }; > > Originally I had asked Brian Goetz about record literals and they seemed > like a possibility but withers seem like a more general feature so I am > hoping something like this would be supported in the future. > > Regards > Swaranga Sarma > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at upliftinglemma.net Fri Mar 1 00:39:02 2024 From: chris at upliftinglemma.net (Chris Bouchard) Date: Thu, 29 Feb 2024 19:39:02 -0500 Subject: Scope for JEP 468: Derived record creation Message-ID: Hi there. I'm new to this mailing list, but I had a similar thought to Swaranga recently when this JEP was posted on Reddit. Personally, my primary concern isn't having nice syntax for constructing records. It's that if we *don't* provide nice syntax for constructing records, people will be incentivized to hack it in by overloading the new "with" syntax. For example, consider something like record MyRecord(String foo, String bar) { MyRecord { // We'd prefer to reject nulls entirely, but we'll allow all nulls // to have a "blank" object. if (allNull(foo, bar)) return; // With that out of the way, validate our *actual* constraints. if (anyNull(foo, bar) || foo.length < 1 || bar.length < 1) throw MyDomainException(); } public static MyRecord blank() { return MyRecord(null, null); } } Now I can say var value = MyRecord.blank() with { bar = "World"; foo = greeting(bar); }; Except that now, because my model didn't have a natural "blank" value, I've added an *un*natural one in the interest of ergonomics. This "blank" object has to be a valid record value. And since the with block's variables are all initialized to null, users can accidentally run into problems with partial initialization. var value = MyRecord.blank() with { // Whoops, forgot to initialize bar so it's still null. // bar = "World"; foo = greeting(bar); // NPE } On the other hand, if we provided an actual way to initialize a fresh record using this block syntax, we could say that all variables in the block are unitialized to start and must be assigned before use. And this initial state wouldn't have to be accepted by the constructor. People have been asking for keyword parameters in Java for years. Whether or not this block syntax is what the language designers would choose for keyword parameters, I think that if we introduce this feature without some way to construct new instances it will become the de facto syntax. It's just too tempting. Thanks for your time and attention, Chris Bouchard On Thu Feb 29 23:11:32 UTC 2024, Brian Goetz wrote: > While such a feature is possible, I am not particularly excited about it > for several reasons. If the goal is "I want to construct using named > instead of position arguments" (which I think is your goal), it's a > pretty verbose syntax; a better one would be > > new R(a: 1, b: 2) > > But that's a much smaller concern. The bigger one is that I don't think > the language is improved by having a named parameter mechanism for > records, but for nothing else (not for instantiating class instances, > not for invoking methods.) While it might seem "better than nothing", > having two different ways to do something, but one of them only works in > narrow situations, is as likely to be frustrating than beneficial. If > the payoff is big enough, then it might be a possibility, but > "invocation by parameter name" is nowhere near that bar. > > Finally, if it works for records but not for classes, it makes it harder > to refactor from records to classes, since there will be use sites that > have to be adjusted. > > So I think for the time being, the answer is "no", though if we run out > of things to work on, we might reconsider. > > On 2/29/2024 4:34 PM, Swaranga Sarma wrote: > > The JEP looks really promising. However I am wondering if there will > > be a separate JEP for creating new records from scratch with similar > > syntax. > > > > The current JEP states that its goals are to enable creating records > > from an existing record and for that it seems sufficient. But I would > > also love to be able to create new records from scratch using the same > > syntax. Something like: > > > > var circle = new Circle with { > > radius = 0.5f; > > center = new Center with { > > x = 0; > > y = -1; > > z = 8; > > }; > > }; > > > > Originally I had asked Brian Goetz about record literals and they > > seemed like a possibility but withers seem like a more general feature > > so I am hoping something like this would be supported in the future. > > > > Regards > > Swaranga Sarma From scolebourne at joda.org Fri Mar 1 09:14:20 2024 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 1 Mar 2024 09:14:20 +0000 Subject: Could JEP 468 support a single expression? Message-ID: I believe that many, if not most, uses of derived record creation expressions will have a single assignment. Has consideration been given to allowing a single assignment expression instead of a block? eg this: record Complex(double re, double im) { Complex conjugate() { return this with { im = -im; }; } Complex realOnly() { return this with { im = 0; }; } Complex imOnly() { return this with { re = 0; }; } } becomes: record Complex(double re, double im) { Complex conjugate() { return this with im = -im; } Complex realOnly() { return this with im = 0; } Complex imOnly() { return this with re = 0; } } In general, I think this JEP is OK, but it doesn't move the needle much without a mechanism to create the record in a similar way. Stephen From brian.goetz at oracle.com Fri Mar 1 14:24:02 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 1 Mar 2024 09:24:02 -0500 Subject: Scope for JEP 468: Derived record creation In-Reply-To: References: Message-ID: I do understand that there are some people who want by-keyword invocation *so badly* that they are willing to write bad code to gain the illusion of doing so.? Your example is the canonical example: MyRecord has no good default, but some people will be tempted to expose (null, null) as the default anyway (instead of rejecting those in the constructor) just so they can "stick it to the compiler."? (Note that this is fine if MyRecord actually has a reasonable default, which some records do, or if you create a constructor that accepts the required components and fills in sane defaults for the rest.) Put more bluntly, some programmers overvalue superficial syntactic preferences so badly that they are willing to compromise the safety and correctness of their code -- and will pat themselves on the back for their cleverness while they do it. > People have been asking for keyword parameters in Java for years. Whether or not > this block syntax is what the language designers would choose for keyword > parameters, I think that if we introduce this feature without some way to > construct new instances it will become the de facto syntax. It's just too > tempting. Oh, I get people want this.? And that bad programmers will surely do this.? (And then they'll complain that they can't do it with classes, or with factories, or that it interferes with compatible migration from records to classes.)? But I don't think we should let their threatened bad behavior drive language design decisions. People think they want keyword parameters, but that's not really what they want -- they just don't realize it yet.? What they really want is keyword parameters *plus being able to omit parameters that have a default*.? These two may sound like almost the same thing, but the difference in reality is huge.? And as someone who has spent more time thinking about this problem in Java than probably anyone else, I promise you this is not the triviality it may seem.? So our antipathy to named invocation is not just that it is a "meh" feature; it is that its cost and benefit are way out of line with each other. On 2/29/2024 7:39 PM, Chris Bouchard wrote: > Hi there. I'm new to this mailing list, but I had a similar thought to Swaranga > recently when this JEP was posted on Reddit. > > Personally, my primary concern isn't having nice syntax for constructing > records. It's that if we *don't* provide nice syntax for constructing records, > people will be incentivized to hack it in by overloading the new "with" syntax. > For example, consider something like > > record MyRecord(String foo, String bar) { > MyRecord { > // We'd prefer to reject nulls entirely, but we'll allow all nulls > // to have a "blank" object. > if (allNull(foo, bar)) > return; > > // With that out of the way, validate our *actual* constraints. > if (anyNull(foo, bar) || foo.length < 1 || bar.length < 1) > throw MyDomainException(); > } > > public static MyRecord blank() { > return MyRecord(null, null); > } > } > > Now I can say > > var value = MyRecord.blank() with { > bar = "World"; > foo = greeting(bar); > }; > > Except that now, because my model didn't have a natural "blank" value, I've > added an *un*natural one in the interest of ergonomics. This "blank" object has > to be a valid record value. And since the with block's variables are all > initialized to null, users can accidentally run into problems with partial > initialization. > > var value = MyRecord.blank() with { > // Whoops, forgot to initialize bar so it's still null. > // bar = "World"; > foo = greeting(bar); // NPE > } > > On the other hand, if we provided an actual way to initialize a fresh record > using this block syntax, we could say that all variables in the block are > unitialized to start and must be assigned before use. And this initial state > wouldn't have to be accepted by the constructor. > > People have been asking for keyword parameters in Java for years. Whether or not > this block syntax is what the language designers would choose for keyword > parameters, I think that if we introduce this feature without some way to > construct new instances it will become the de facto syntax. It's just too > tempting. > > Thanks for your time and attention, > Chris Bouchard > > On Thu Feb 29 23:11:32 UTC 2024, Brian Goetz wrote: >> While such a feature is possible, I am not particularly excited about it >> for several reasons. If the goal is "I want to construct using named >> instead of position arguments" (which I think is your goal), it's a >> pretty verbose syntax; a better one would be >> >> new R(a: 1, b: 2) >> >> But that's a much smaller concern. The bigger one is that I don't think >> the language is improved by having a named parameter mechanism for >> records, but for nothing else (not for instantiating class instances, >> not for invoking methods.) While it might seem "better than nothing", >> having two different ways to do something, but one of them only works in >> narrow situations, is as likely to be frustrating than beneficial. If >> the payoff is big enough, then it might be a possibility, but >> "invocation by parameter name" is nowhere near that bar. >> >> Finally, if it works for records but not for classes, it makes it harder >> to refactor from records to classes, since there will be use sites that >> have to be adjusted. >> >> So I think for the time being, the answer is "no", though if we run out >> of things to work on, we might reconsider. >> >> On 2/29/2024 4:34 PM, Swaranga Sarma wrote: >>> The JEP looks really promising. However I am wondering if there will >>> be a separate JEP for creating new records from scratch with similar >>> syntax. >>> >>> The current JEP states that its goals are to enable creating records >>> from an existing record and for that it seems sufficient. But I would >>> also love to be able to create new records from scratch using the same >>> syntax. Something like: >>> >>> var circle = new Circle with { >>> radius = 0.5f; >>> center = new Center with { >>> x = 0; >>> y = -1; >>> z = 8; >>> }; >>> }; >>> >>> Originally I had asked Brian Goetz about record literals and they >>> seemed like a possibility but withers seem like a more general feature >>> so I am hoping something like this would be supported in the future. >>> >>> Regards >>> Swaranga Sarma From brian.goetz at oracle.com Fri Mar 1 14:28:24 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 1 Mar 2024 09:28:24 -0500 Subject: Could JEP 468 support a single expression? In-Reply-To: References: Message-ID: Yes, considerable consideration has been given to this.? It turns out it is trickier than it looks, but we understand that it is desirable, and will try to make it happen if we can. On 3/1/2024 4:14 AM, Stephen Colebourne wrote: > I believe that many, if not most, uses of derived record creation > expressions will have a single assignment. Has consideration been > given to allowing a single assignment expression instead of a block? > > eg this: > record Complex(double re, double im) { > Complex conjugate() { return this with { im = -im; }; } > Complex realOnly() { return this with { im = 0; }; } > Complex imOnly() { return this with { re = 0; }; } > } > > becomes: > record Complex(double re, double im) { > Complex conjugate() { return this with im = -im; } > Complex realOnly() { return this with im = 0; } > Complex imOnly() { return this with re = 0; } > } > > In general, I think this JEP is OK, but it doesn't move the needle > much without a mechanism to create the record in a similar way. > > Stephen -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Fri Mar 1 15:39:55 2024 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 1 Mar 2024 16:39:55 +0100 (CET) Subject: Scope for JEP 468: Derived record creation In-Reply-To: References: Message-ID: <846883428.18660307.1709307595144.JavaMail.zimbra@univ-eiffel.fr> ----- Original Message ----- > From: "Brian Goetz" > To: "Chris Bouchard" , "amber-dev" > Sent: Friday, March 1, 2024 3:24:02 PM > Subject: Re: Scope for JEP 468: Derived record creation > I do understand that there are some people who want by-keyword > invocation *so badly* that they are willing to write bad code to gain > the illusion of doing so.? Your example is the canonical example: > MyRecord has no good default, but some people will be tempted to expose > (null, null) as the default anyway (instead of rejecting those in the > constructor) just so they can "stick it to the compiler."? (Note that > this is fine if MyRecord actually has a reasonable default, which some > records do, or if you create a constructor that accepts the required > components and fills in sane defaults for the rest.) > > Put more bluntly, some programmers overvalue superficial syntactic > preferences so badly that they are willing to compromise the safety and > correctness of their code -- and will pat themselves on the back for > their cleverness while they do it. > >> People have been asking for keyword parameters in Java for years. Whether or not >> this block syntax is what the language designers would choose for keyword >> parameters, I think that if we introduce this feature without some way to >> construct new instances it will become the de facto syntax. It's just too >> tempting. > > Oh, I get people want this.? And that bad programmers will surely do > this.? (And then they'll complain that they can't do it with classes, or > with factories, or that it interferes with compatible migration from > records to classes.)? But I don't think we should let their threatened > bad behavior drive language design decisions. > > People think they want keyword parameters, but that's not really what > they want -- they just don't realize it yet.? What they really want is > keyword parameters *plus being able to omit parameters that have a > default*.? These two may sound like almost the same thing, but the > difference in reality is huge.? And as someone who has spent more time > thinking about this problem in Java than probably anyone else, I promise > you this is not the triviality it may seem.? So our antipathy to named > invocation is not just that it is a "meh" feature; it is that its cost > and benefit are way out of line with each other. Both can be unified if we introduced a weird syntax to declare and use a value record as parameter or return type. This is a little bit similar to an anonymous class because the value record is scoped to the method but it has a name so javac error messages and binary compatibility are not an issue record Circle(int x, int y, int radius) { Circle(Point(int x, int y)) { this(x, y, 1); } } which is translated to: record Circle(int x, int y, int radius) { Circle(Circle$$Point $p) { int x = $p.x; int y = $p.y; // or Circle$$Point(int x, int y) = $p; this(x, y, 1); } } // like an anonymous class, the EnclosingMethod is Circle(Circle$$Point) value record Circle$$Point(int x, int y) {} and at use site new Circle({ x = 3; y = 4; }) we need inference so { x = 3; y = 4; } is equivalent to new Circle$$Point(3, 4). The exact details are fuzy but as you know, this also solve - how to write a de-constructor Point(int x, int y) deconstructor() { return { x = this.x; y = this.y; }; } and it makes the syntax looks like the inverse of the constructor syntax - how to returns multiple values Div(int quotient, int remainder) div(int value, int divisor) { return { quotient = value / divisor; remainder = value % divisor; }; } - and even how to declare tuples, if we allow (3, 4) to be inferred as new Circle$$Point(3, 4) too. The idea is that a value record is so lighweight at runtime that having a syntax that mix the declaration of the value record and its use make sense. I also found this idea stupid at first but it keep popping in a lot of use cases. R?mi > > > > > > On 2/29/2024 7:39 PM, Chris Bouchard wrote: >> Hi there. I'm new to this mailing list, but I had a similar thought to Swaranga >> recently when this JEP was posted on Reddit. >> >> Personally, my primary concern isn't having nice syntax for constructing >> records. It's that if we *don't* provide nice syntax for constructing records, >> people will be incentivized to hack it in by overloading the new "with" syntax. >> For example, consider something like >> >> record MyRecord(String foo, String bar) { >> MyRecord { >> // We'd prefer to reject nulls entirely, but we'll allow all nulls >> // to have a "blank" object. >> if (allNull(foo, bar)) >> return; >> >> // With that out of the way, validate our *actual* constraints. >> if (anyNull(foo, bar) || foo.length < 1 || bar.length < 1) >> throw MyDomainException(); >> } >> >> public static MyRecord blank() { >> return MyRecord(null, null); >> } >> } >> >> Now I can say >> >> var value = MyRecord.blank() with { >> bar = "World"; >> foo = greeting(bar); >> }; >> >> Except that now, because my model didn't have a natural "blank" value, I've >> added an *un*natural one in the interest of ergonomics. This "blank" object has >> to be a valid record value. And since the with block's variables are all >> initialized to null, users can accidentally run into problems with partial >> initialization. >> >> var value = MyRecord.blank() with { >> // Whoops, forgot to initialize bar so it's still null. >> // bar = "World"; >> foo = greeting(bar); // NPE >> } >> >> On the other hand, if we provided an actual way to initialize a fresh record >> using this block syntax, we could say that all variables in the block are >> unitialized to start and must be assigned before use. And this initial state >> wouldn't have to be accepted by the constructor. >> >> People have been asking for keyword parameters in Java for years. Whether or not >> this block syntax is what the language designers would choose for keyword >> parameters, I think that if we introduce this feature without some way to >> construct new instances it will become the de facto syntax. It's just too >> tempting. >> >> Thanks for your time and attention, >> Chris Bouchard >> >> On Thu Feb 29 23:11:32 UTC 2024, Brian Goetz wrote: >>> While such a feature is possible, I am not particularly excited about it >>> for several reasons. If the goal is "I want to construct using named >>> instead of position arguments" (which I think is your goal), it's a >>> pretty verbose syntax; a better one would be >>> >>> new R(a: 1, b: 2) >>> >>> But that's a much smaller concern. The bigger one is that I don't think >>> the language is improved by having a named parameter mechanism for >>> records, but for nothing else (not for instantiating class instances, >>> not for invoking methods.) While it might seem "better than nothing", >>> having two different ways to do something, but one of them only works in >>> narrow situations, is as likely to be frustrating than beneficial. If >>> the payoff is big enough, then it might be a possibility, but >>> "invocation by parameter name" is nowhere near that bar. >>> >>> Finally, if it works for records but not for classes, it makes it harder >>> to refactor from records to classes, since there will be use sites that >>> have to be adjusted. >>> >>> So I think for the time being, the answer is "no", though if we run out >>> of things to work on, we might reconsider. >>> >>> On 2/29/2024 4:34 PM, Swaranga Sarma wrote: >>>> The JEP looks really promising. However I am wondering if there will >>>> be a separate JEP for creating new records from scratch with similar >>>> syntax. >>>> >>>> The current JEP states that its goals are to enable creating records >>>> from an existing record and for that it seems sufficient. But I would >>>> also love to be able to create new records from scratch using the same >>>> syntax. Something like: >>>> >>>> var circle = new Circle with { >>>> radius = 0.5f; >>>> center = new Center with { >>>> x = 0; >>>> y = -1; >>>> z = 8; >>>> }; >>>> }; >>>> >>>> Originally I had asked Brian Goetz about record literals and they >>>> seemed like a possibility but withers seem like a more general feature >>>> so I am hoping something like this would be supported in the future. >>>> >>>> Regards > >>> Swaranga Sarma From brian.goetz at oracle.com Fri Mar 1 15:45:58 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 1 Mar 2024 10:45:58 -0500 Subject: Scope for JEP 468: Derived record creation In-Reply-To: <846883428.18660307.1709307595144.JavaMail.zimbra@univ-eiffel.fr> References: <846883428.18660307.1709307595144.JavaMail.zimbra@univ-eiffel.fr> Message-ID: <97f72d76-067a-425f-b3af-882f0050b540@oracle.com> I would rather revisit "record literals" when we look more carefully and holistically at collection literals, which will come later when some other foundations are in place.? The current discussion is mostly one of those "While you've got the patient under sedation, can we do a nose job too?", and, just as in medicine, such questions usually bely some wrong assumptions about where the costs and benefits lie. On 3/1/2024 10:39 AM, Remi Forax wrote: > ----- Original Message ----- >> From: "Brian Goetz" >> To: "Chris Bouchard", "amber-dev" >> Sent: Friday, March 1, 2024 3:24:02 PM >> Subject: Re: Scope for JEP 468: Derived record creation >> I do understand that there are some people who want by-keyword >> invocation *so badly* that they are willing to write bad code to gain >> the illusion of doing so.? Your example is the canonical example: >> MyRecord has no good default, but some people will be tempted to expose >> (null, null) as the default anyway (instead of rejecting those in the >> constructor) just so they can "stick it to the compiler."? (Note that >> this is fine if MyRecord actually has a reasonable default, which some >> records do, or if you create a constructor that accepts the required >> components and fills in sane defaults for the rest.) >> >> Put more bluntly, some programmers overvalue superficial syntactic >> preferences so badly that they are willing to compromise the safety and >> correctness of their code -- and will pat themselves on the back for >> their cleverness while they do it. >> >>> People have been asking for keyword parameters in Java for years. Whether or not >>> this block syntax is what the language designers would choose for keyword >>> parameters, I think that if we introduce this feature without some way to >>> construct new instances it will become the de facto syntax. It's just too >>> tempting. >> Oh, I get people want this.? And that bad programmers will surely do >> this.? (And then they'll complain that they can't do it with classes, or >> with factories, or that it interferes with compatible migration from >> records to classes.)? But I don't think we should let their threatened >> bad behavior drive language design decisions. >> >> People think they want keyword parameters, but that's not really what >> they want -- they just don't realize it yet.? What they really want is >> keyword parameters *plus being able to omit parameters that have a >> default*.? These two may sound like almost the same thing, but the >> difference in reality is huge.? And as someone who has spent more time >> thinking about this problem in Java than probably anyone else, I promise >> you this is not the triviality it may seem.? So our antipathy to named >> invocation is not just that it is a "meh" feature; it is that its cost >> and benefit are way out of line with each other. > Both can be unified if we introduced a weird syntax to declare and use a value record as parameter or return type. > This is a little bit similar to an anonymous class because the value record is scoped to the method but it has a name so javac error messages and binary compatibility are not an issue > > record Circle(int x, int y, int radius) { > Circle(Point(int x, int y)) { > this(x, y, 1); > } > } > > which is translated to: > > record Circle(int x, int y, int radius) { > Circle(Circle$$Point $p) { > int x = $p.x; > int y = $p.y; // or Circle$$Point(int x, int y) = $p; > this(x, y, 1); > } > } > > // like an anonymous class, the EnclosingMethod is Circle(Circle$$Point) > value record Circle$$Point(int x, int y) {} > > > and at use site > new Circle({ x = 3; y = 4; }) > > we need inference so { x = 3; y = 4; } is equivalent to new Circle$$Point(3, 4). > > > The exact details are fuzy but as you know, this also solve > - how to write a de-constructor > Point(int x, int y) deconstructor() { > return { x = this.x; y = this.y; }; > } > and it makes the syntax looks like the inverse of the constructor syntax > > - how to returns multiple values > Div(int quotient, int remainder) div(int value, int divisor) { > return { quotient = value / divisor; remainder = value % divisor; }; > } > > - and even how to declare tuples, if we allow (3, 4) to be inferred as new Circle$$Point(3, 4) too. > > The idea is that a value record is so lighweight at runtime that having a syntax that mix the declaration of the value record and its use make sense. > > I also found this idea stupid at first but it keep popping in a lot of use cases. > > R?mi > >> >> >> >> >> On 2/29/2024 7:39 PM, Chris Bouchard wrote: >>> Hi there. I'm new to this mailing list, but I had a similar thought to Swaranga >>> recently when this JEP was posted on Reddit. >>> >>> Personally, my primary concern isn't having nice syntax for constructing >>> records. It's that if we *don't* provide nice syntax for constructing records, >>> people will be incentivized to hack it in by overloading the new "with" syntax. >>> For example, consider something like >>> >>> record MyRecord(String foo, String bar) { >>> MyRecord { >>> // We'd prefer to reject nulls entirely, but we'll allow all nulls >>> // to have a "blank" object. >>> if (allNull(foo, bar)) >>> return; >>> >>> // With that out of the way, validate our *actual* constraints. >>> if (anyNull(foo, bar) || foo.length < 1 || bar.length < 1) >>> throw MyDomainException(); >>> } >>> >>> public static MyRecord blank() { >>> return MyRecord(null, null); >>> } >>> } >>> >>> Now I can say >>> >>> var value = MyRecord.blank() with { >>> bar = "World"; >>> foo = greeting(bar); >>> }; >>> >>> Except that now, because my model didn't have a natural "blank" value, I've >>> added an *un*natural one in the interest of ergonomics. This "blank" object has >>> to be a valid record value. And since the with block's variables are all >>> initialized to null, users can accidentally run into problems with partial >>> initialization. >>> >>> var value = MyRecord.blank() with { >>> // Whoops, forgot to initialize bar so it's still null. >>> // bar = "World"; >>> foo = greeting(bar); // NPE >>> } >>> >>> On the other hand, if we provided an actual way to initialize a fresh record >>> using this block syntax, we could say that all variables in the block are >>> unitialized to start and must be assigned before use. And this initial state >>> wouldn't have to be accepted by the constructor. >>> >>> People have been asking for keyword parameters in Java for years. Whether or not >>> this block syntax is what the language designers would choose for keyword >>> parameters, I think that if we introduce this feature without some way to >>> construct new instances it will become the de facto syntax. It's just too >>> tempting. >>> >>> Thanks for your time and attention, >>> Chris Bouchard >>> >>> On Thu Feb 29 23:11:32 UTC 2024, Brian Goetz wrote: >>>> While such a feature is possible, I am not particularly excited about it >>>> for several reasons. If the goal is "I want to construct using named >>>> instead of position arguments" (which I think is your goal), it's a >>>> pretty verbose syntax; a better one would be >>>> >>>> new R(a: 1, b: 2) >>>> >>>> But that's a much smaller concern. The bigger one is that I don't think >>>> the language is improved by having a named parameter mechanism for >>>> records, but for nothing else (not for instantiating class instances, >>>> not for invoking methods.) While it might seem "better than nothing", >>>> having two different ways to do something, but one of them only works in >>>> narrow situations, is as likely to be frustrating than beneficial. If >>>> the payoff is big enough, then it might be a possibility, but >>>> "invocation by parameter name" is nowhere near that bar. >>>> >>>> Finally, if it works for records but not for classes, it makes it harder >>>> to refactor from records to classes, since there will be use sites that >>>> have to be adjusted. >>>> >>>> So I think for the time being, the answer is "no", though if we run out >>>> of things to work on, we might reconsider. >>>> >>>> On 2/29/2024 4:34 PM, Swaranga Sarma wrote: >>>>> The JEP looks really promising. However I am wondering if there will >>>>> be a separate JEP for creating new records from scratch with similar >>>>> syntax. >>>>> >>>>> The current JEP states that its goals are to enable creating records >>>>> from an existing record and for that it seems sufficient. But I would >>>>> also love to be able to create new records from scratch using the same >>>>> syntax. Something like: >>>>> >>>>> var circle = new Circle with { >>>>> radius = 0.5f; >>>>> center = new Center with { >>>>> x = 0; >>>>> y = -1; >>>>> z = 8; >>>>> }; >>>>> }; >>>>> >>>>> Originally I had asked Brian Goetz about record literals and they >>>>> seemed like a possibility but withers seem like a more general feature >>>>> so I am hoping something like this would be supported in the future. >>>>> >>>>> Regards >>>>> Swaranga Sarma -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Fri Mar 1 15:52:43 2024 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 1 Mar 2024 16:52:43 +0100 (CET) Subject: Scope for JEP 468: Derived record creation In-Reply-To: <97f72d76-067a-425f-b3af-882f0050b540@oracle.com> References: <846883428.18660307.1709307595144.JavaMail.zimbra@univ-eiffel.fr> <97f72d76-067a-425f-b3af-882f0050b540@oracle.com> Message-ID: <278886703.18668382.1709308363788.JavaMail.zimbra@univ-eiffel.fr> > From: "Brian Goetz" > To: "Remi Forax" > Cc: "Chris Bouchard" , "amber-dev" > > Sent: Friday, March 1, 2024 4:45:58 PM > Subject: Re: Scope for JEP 468: Derived record creation > I would rather revisit "record literals" when we look more carefully and > holistically at collection literals, which will come later when some other > foundations are in place. The current discussion is mostly one of those "While > you've got the patient under sedation, can we do a nose job too?", and, just as > in medicine, such questions usually bely some wrong assumptions about where the > costs and benefits lie. I agree that it's a step after record literals. Does it mean that deconstructor, at least the exact syntax, should also be discussed after record literals ? R?mi > On 3/1/2024 10:39 AM, Remi Forax wrote: >> ----- Original Message ----- >>> From: "Brian Goetz" [ mailto:brian.goetz at oracle.com | ] >>> To: "Chris Bouchard" [ mailto:chris at upliftinglemma.net | >>> ] , "amber-dev" [ mailto:amber-dev at openjdk.org | >>> ] Sent: Friday, March 1, 2024 3:24:02 PM >>> Subject: Re: Scope for JEP 468: Derived record creation >>> I do understand that there are some people who want by-keyword >>> invocation *so badly* that they are willing to write bad code to gain >>> the illusion of doing so.? Your example is the canonical example: >>> MyRecord has no good default, but some people will be tempted to expose >>> (null, null) as the default anyway (instead of rejecting those in the >>> constructor) just so they can "stick it to the compiler."? (Note that >>> this is fine if MyRecord actually has a reasonable default, which some >>> records do, or if you create a constructor that accepts the required >>> components and fills in sane defaults for the rest.) >>> Put more bluntly, some programmers overvalue superficial syntactic >>> preferences so badly that they are willing to compromise the safety and >>> correctness of their code -- and will pat themselves on the back for >>> their cleverness while they do it. >>>> People have been asking for keyword parameters in Java for years. Whether or not >>>> this block syntax is what the language designers would choose for keyword >>>> parameters, I think that if we introduce this feature without some way to >>>> construct new instances it will become the de facto syntax. It's just too >>>> tempting. >>> Oh, I get people want this.? And that bad programmers will surely do >>> this.? (And then they'll complain that they can't do it with classes, or >>> with factories, or that it interferes with compatible migration from >>> records to classes.)? But I don't think we should let their threatened >>> bad behavior drive language design decisions. >>> People think they want keyword parameters, but that's not really what >>> they want -- they just don't realize it yet.? What they really want is >>> keyword parameters *plus being able to omit parameters that have a >>> default*.? These two may sound like almost the same thing, but the >>> difference in reality is huge.? And as someone who has spent more time >>> thinking about this problem in Java than probably anyone else, I promise >>> you this is not the triviality it may seem.? So our antipathy to named >>> invocation is not just that it is a "meh" feature; it is that its cost >>> and benefit are way out of line with each other. >> Both can be unified if we introduced a weird syntax to declare and use a value >> record as parameter or return type. >> This is a little bit similar to an anonymous class because the value record is >> scoped to the method but it has a name so javac error messages and binary >> compatibility are not an issue >> record Circle(int x, int y, int radius) { >> Circle(Point(int x, int y)) { >> this(x, y, 1); >> } >> } >> which is translated to: >> record Circle(int x, int y, int radius) { >> Circle(Circle$$Point $p) { >> int x = $p.x; >> int y = $p.y; // or Circle$$Point(int x, int y) = $p; >> this(x, y, 1); >> } >> } >> // like an anonymous class, the EnclosingMethod is Circle(Circle$$Point) >> value record Circle$$Point(int x, int y) {} >> and at use site >> new Circle({ x = 3; y = 4; }) >> we need inference so { x = 3; y = 4; } is equivalent to new Circle$$Point(3, 4). >> The exact details are fuzy but as you know, this also solve >> - how to write a de-constructor >> Point(int x, int y) deconstructor() { >> return { x = this.x; y = this.y; }; >> } >> and it makes the syntax looks like the inverse of the constructor syntax >> - how to returns multiple values >> Div(int quotient, int remainder) div(int value, int divisor) { >> return { quotient = value / divisor; remainder = value % divisor; }; >> } >> - and even how to declare tuples, if we allow (3, 4) to be inferred as new >> Circle$$Point(3, 4) too. >> The idea is that a value record is so lighweight at runtime that having a syntax >> that mix the declaration of the value record and its use make sense. >> I also found this idea stupid at first but it keep popping in a lot of use >> cases. >> R?mi >>> On 2/29/2024 7:39 PM, Chris Bouchard wrote: >>>> Hi there. I'm new to this mailing list, but I had a similar thought to Swaranga >>>> recently when this JEP was posted on Reddit. >>>> Personally, my primary concern isn't having nice syntax for constructing >>>> records. It's that if we *don't* provide nice syntax for constructing records, >>>> people will be incentivized to hack it in by overloading the new "with" syntax. >>>> For example, consider something like >>>> record MyRecord(String foo, String bar) { >>>> MyRecord { >>>> // We'd prefer to reject nulls entirely, but we'll allow all nulls >>>> // to have a "blank" object. >>>> if (allNull(foo, bar)) >>>> return; >>>> // With that out of the way, validate our *actual* constraints. >>>> if (anyNull(foo, bar) || foo.length < 1 || bar.length < 1) >>>> throw MyDomainException(); >>>> } >>>> public static MyRecord blank() { >>>> return MyRecord(null, null); >>>> } >>>> } >>>> Now I can say >>>> var value = MyRecord.blank() with { >>>> bar = "World"; >>>> foo = greeting(bar); >>>> }; >>>> Except that now, because my model didn't have a natural "blank" value, I've >>>> added an *un*natural one in the interest of ergonomics. This "blank" object has >>>> to be a valid record value. And since the with block's variables are all >>>> initialized to null, users can accidentally run into problems with partial >>>> initialization. >>>> var value = MyRecord.blank() with { >>>> // Whoops, forgot to initialize bar so it's still null. >>>> // bar = "World"; >>>> foo = greeting(bar); // NPE >>>> } >>>> On the other hand, if we provided an actual way to initialize a fresh record >>>> using this block syntax, we could say that all variables in the block are >>>> unitialized to start and must be assigned before use. And this initial state >>>> wouldn't have to be accepted by the constructor. >>>> People have been asking for keyword parameters in Java for years. Whether or not >>>> this block syntax is what the language designers would choose for keyword >>>> parameters, I think that if we introduce this feature without some way to >>>> construct new instances it will become the de facto syntax. It's just too >>>> tempting. >>>> Thanks for your time and attention, >>>> Chris Bouchard >>>> On Thu Feb 29 23:11:32 UTC 2024, Brian Goetz wrote: >>>>> While such a feature is possible, I am not particularly excited about it >>>>> for several reasons. If the goal is "I want to construct using named >>>>> instead of position arguments" (which I think is your goal), it's a >>>>> pretty verbose syntax; a better one would be >>>>> new R(a: 1, b: 2) >>>>> But that's a much smaller concern. The bigger one is that I don't think >>>>> the language is improved by having a named parameter mechanism for >>>>> records, but for nothing else (not for instantiating class instances, >>>>> not for invoking methods.) While it might seem "better than nothing", >>>>> having two different ways to do something, but one of them only works in >>>>> narrow situations, is as likely to be frustrating than beneficial. If >>>>> the payoff is big enough, then it might be a possibility, but >>>>> "invocation by parameter name" is nowhere near that bar. >>>>> Finally, if it works for records but not for classes, it makes it harder >>>>> to refactor from records to classes, since there will be use sites that >>>>> have to be adjusted. >>>>> So I think for the time being, the answer is "no", though if we run out >>>>> of things to work on, we might reconsider. >>>>> On 2/29/2024 4:34 PM, Swaranga Sarma wrote: >>>>>> The JEP looks really promising. However I am wondering if there will >>>>>> be a separate JEP for creating new records from scratch with similar >>>>>> syntax. >>>>>> The current JEP states that its goals are to enable creating records >>>>>> from an existing record and for that it seems sufficient. But I would >>>>>> also love to be able to create new records from scratch using the same >>>>>> syntax. Something like: >>>>>> var circle = new Circle with { >>>>>> radius = 0.5f; >>>>>> center = new Center with { >>>>>> x = 0; >>>>>> y = -1; >>>>>> z = 8; >>>>>> }; >>>>>> }; >>>>>> Originally I had asked Brian Goetz about record literals and they >>>>>> seemed like a possibility but withers seem like a more general feature >>>>>> so I am hoping something like this would be supported in the future. >>>>>> Regards >>>>>> Swaranga Sarma -------------- next part -------------- An HTML attachment was scrubbed... URL: From gavin.bierman at oracle.com Fri Mar 1 17:37:51 2024 From: gavin.bierman at oracle.com (Gavin Bierman) Date: Fri, 1 Mar 2024 17:37:51 +0000 Subject: Scope for JEP 468: Derived record creation In-Reply-To: References: <1e81bd77-6417-4018-8c23-1cef01f2e1f7@oracle.com> Message-ID: On 1 Mar 2024, at 00:02, Swaranga Sarma wrote: Thank you. >it's a pretty verbose syntax Agreed. I was trying to stay close to the current syntax but even more concise could be: R r = {a: 1, b: 2}. But the question was not about the syntax. Right, but that?s a record literal. As Brian says on another thread, we have plans to look at this when we come to look at other new literal forms. (So many things in the pipeline!) Thanks, Gavin > it makes it harder to refactor from records to classes Wouldn't the current JEP also have the same limitations? Although I do see the JEP mention that it may be possible in the future for regular classes. Regards Swaranga On Thu, Feb 29, 2024 at 3:11?PM Brian Goetz > wrote: While such a feature is possible, I am not particularly excited about it for several reasons. If the goal is "I want to construct using named instead of position arguments" (which I think is your goal), it's a pretty verbose syntax; a better one would be new R(a: 1, b: 2) But that's a much smaller concern. The bigger one is that I don't think the language is improved by having a named parameter mechanism for records, but for nothing else (not for instantiating class instances, not for invoking methods.) While it might seem "better than nothing", having two different ways to do something, but one of them only works in narrow situations, is as likely to be frustrating than beneficial. If the payoff is big enough, then it might be a possibility, but "invocation by parameter name" is nowhere near that bar. Finally, if it works for records but not for classes, it makes it harder to refactor from records to classes, since there will be use sites that have to be adjusted. So I think for the time being, the answer is "no", though if we run out of things to work on, we might reconsider. On 2/29/2024 4:34 PM, Swaranga Sarma wrote: The JEP looks really promising. However I am wondering if there will be a separate JEP for creating new records from scratch with similar syntax. The current JEP states that its goals are to enable creating records from an existing record and for that it seems sufficient. But I would also love to be able to create new records from scratch using the same syntax. Something like: var circle = new Circle with { radius = 0.5f; center = new Center with { x = 0; y = -1; z = 8; }; }; Originally I had asked Brian Goetz about record literals and they seemed like a possibility but withers seem like a more general feature so I am hoping something like this would be supported in the future. Regards Swaranga Sarma -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Mon Mar 4 03:00:36 2024 From: davidalayachew at gmail.com (David Alayachew) Date: Sun, 3 Mar 2024 22:00:36 -0500 Subject: Any plans to make resource leaks easier to detect? Message-ID: Hello Amber Dev Team and Core Libs Dev Team, I am making my own implementation of java.util.stream.Stream that reads data from the internet lazily. It's basically java.nio.file.Files.lines(...), but the file is on the internet instead. Here is a StackOverflow post I made that basically gave birth to idea. Just for context. [1] I used composition -- I held my resource and the stream inside of my container data type. However, I also had this container data type implement java.util.stream.Stream too. This way, the end user can use this object exactly like a Stream. In my implementation, I delegated all method calls to the underlying stream. But, for each terminal operation, I would wrap the delegated calls with try-with-resources for my resources, to prevent resource leak upon exception. Now, I know that this is taking an uphill path to a solution. We could easily tell the user that they have to close the resources themself. Or we could let the user provide their own resources and thus, leave the responsibility on their plate. Or we could use the escape-hatch provided by stream lol, and tell the user to use try-with-resources on my custom stream. As you all intended us to do, I'm sure. I know the reason might seem poorly justified (StackOverflow told me as much -- [1]), but I did this so that none of the users of my stream would have to concern themselves doing try-with-resources. Try-with-resources is an excellent utility, but I have no way to know if I left a resource unclosed because there are no compiler warnings or errors if I do so. Are there any plans to make it easier to detect potential resource leaks? Ideally with a compiler warning or error? Thank you for your time and help! David Alayachew [1] = https://stackoverflow.com/questions/77959436 -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Mon Mar 4 03:02:47 2024 From: davidalayachew at gmail.com (David Alayachew) Date: Sun, 3 Mar 2024 22:02:47 -0500 Subject: Any plans to make resource leaks easier to detect? In-Reply-To: References: Message-ID: And as a side note, I did some pretty in-depth research on the topic, and stumbled on this post on the lambda mailing list during Java 8's creation. I am adding it, as it seems to be considering many of the same concerns I have now. https://mail.openjdk.org/pipermail/lambda-libs-spec-experts/2013-August/002195.html On Sun, Mar 3, 2024 at 10:00?PM David Alayachew wrote: > > Hello Amber Dev Team and Core Libs Dev Team, > > I am making my own implementation of java.util.stream.Stream that reads > data from the internet lazily. It's basically > java.nio.file.Files.lines(...), but the file is on the internet instead. > Here is a StackOverflow post I made that basically gave birth to idea. Just > for context. [1] > > I used composition -- I held my resource and the stream inside of my > container data type. However, I also had this container data type implement > java.util.stream.Stream too. This way, the end user can use this object > exactly like a Stream. > > In my implementation, I delegated all method calls to the underlying > stream. But, for each terminal operation, I would wrap the delegated calls > with try-with-resources for my resources, to prevent resource leak upon > exception. > > Now, I know that this is taking an uphill path to a solution. We could > easily tell the user that they have to close the resources themself. Or we > could let the user provide their own resources and thus, leave the > responsibility on their plate. Or we could use the escape-hatch provided by > stream lol, and tell the user to use try-with-resources on my custom > stream. As you all intended us to do, I'm sure. > > I know the reason might seem poorly justified (StackOverflow told me as > much -- [1]), but I did this so that none of the users of my stream would > have to concern themselves doing try-with-resources. Try-with-resources is > an excellent utility, but I have no way to know if I left a resource > unclosed because there are no compiler warnings or errors if I do so. > > Are there any plans to make it easier to detect potential resource leaks? > Ideally with a compiler warning or error? > > Thank you for your time and help! > David Alayachew > > [1] = https://stackoverflow.com/questions/77959436 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Thu Mar 7 14:26:51 2024 From: davidalayachew at gmail.com (David Alayachew) Date: Thu, 7 Mar 2024 09:26:51 -0500 Subject: Any plans to make resource leaks easier to detect? In-Reply-To: References: Message-ID: Further adding on, here is my final implementation. The folks on Code Review also informed me, in no uncertain terms, that it was a very problematic solution. And I agree with them. However, for all of its flaws, at least it cannot leak due to user's forgetting a TWR. https://codereview.stackexchange.com/questions/290912/implementation-of-java-util-stream-stream-and-friends-that-reads-lines-from-th But of course, this is a terrible solution to deal with a non-trivial problem -- forgetting TWR is not a compiler error/warning. Is there any chance that we could get better compile time support for detecting resource leaks? On Sun, Mar 3, 2024 at 10:02?PM David Alayachew wrote: > And as a side note, I did some pretty in-depth research on the topic, and > stumbled on this post on the lambda mailing list during Java 8's creation. > I am adding it, as it seems to be considering many of the same concerns I > have now. > > > https://mail.openjdk.org/pipermail/lambda-libs-spec-experts/2013-August/002195.html > > On Sun, Mar 3, 2024 at 10:00?PM David Alayachew > wrote: > >> >> Hello Amber Dev Team and Core Libs Dev Team, >> >> I am making my own implementation of java.util.stream.Stream that reads >> data from the internet lazily. It's basically >> java.nio.file.Files.lines(...), but the file is on the internet instead. >> Here is a StackOverflow post I made that basically gave birth to idea. Just >> for context. [1] >> >> I used composition -- I held my resource and the stream inside of my >> container data type. However, I also had this container data type implement >> java.util.stream.Stream too. This way, the end user can use this object >> exactly like a Stream. >> >> In my implementation, I delegated all method calls to the underlying >> stream. But, for each terminal operation, I would wrap the delegated calls >> with try-with-resources for my resources, to prevent resource leak upon >> exception. >> >> Now, I know that this is taking an uphill path to a solution. We could >> easily tell the user that they have to close the resources themself. Or we >> could let the user provide their own resources and thus, leave the >> responsibility on their plate. Or we could use the escape-hatch provided by >> stream lol, and tell the user to use try-with-resources on my custom >> stream. As you all intended us to do, I'm sure. >> >> I know the reason might seem poorly justified (StackOverflow told me as >> much -- [1]), but I did this so that none of the users of my stream would >> have to concern themselves doing try-with-resources. Try-with-resources is >> an excellent utility, but I have no way to know if I left a resource >> unclosed because there are no compiler warnings or errors if I do so. >> >> Are there any plans to make it easier to detect potential resource leaks? >> Ideally with a compiler warning or error? >> >> Thank you for your time and help! >> David Alayachew >> >> [1] = https://stackoverflow.com/questions/77959436 >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From duncan.macgregor at servicenow.com Mon Mar 11 12:17:39 2024 From: duncan.macgregor at servicenow.com (Duncan MacGregor) Date: Mon, 11 Mar 2024 12:17:39 +0000 Subject: New string template approach. Message-ID: So over Christmas I tried kicking the tires on string templates I by creating a method handle combinators library that worked with a small lisp DSL, because I had always described the method handles API to new developers as a lisp with really annoying syntax, so I thought I?d try and remove that syntax hurdle. Overall I think it worked well, except that there was some friction in composing fragments (should I use RAW templates, provide an intermediate form, or make sure I could always decompose the MH structures in all the ways my library might need) and that if I expected a call site to be used frequently I lacked the sealed linkage mechanism that FMT could use. Removing the processor from template literal declaration definitely steers things in a particular direction and should place all users on an even field in terms of implementation optimisations, and I?ll be very interested to see this when there is a branch available. I think the downside to this is that we will no longer see a clean correspondence of template literals to call sites that process them so I will be interested to see what performance boost we get on string formatting that couldn?t already be done with the existing format string. The other thing I?d be interested to know in this new approach is how StringTemplates inserted as values into other templates will be treated. Simply as StringTemplate objects in the values list, or will they produce a string template that has been flattened? The new approach feels like it will encourage this approach of combining fragments and is likely to be something that many processors are going to need to handle. Anyway, looking forward to seeing this new approach and trying this MH combinators prototype a second time. Duncan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Mon Mar 11 16:53:52 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 11 Mar 2024 16:53:52 +0000 Subject: New string template approach. In-Reply-To: References: Message-ID: <0269581B-8C4D-47FB-89D5-A55BE070DD76@oracle.com> On Mar 11, 2024, at 5:17 AM, Duncan MacGregor > wrote: The other thing I?d be interested to know in this new approach is how StringTemplates inserted as values into other templates will be treated. Simply as StringTemplate objects in the values list, or will they produce a string template that has been flattened? The new approach feels like it will encourage this approach of combining fragments and is likely to be something that many processors are going to need to handle. This is a significant point. In the previous design iteration, this never came up, because templates were processed by processors so early they didn?t have much of a chance to be combined with other templates. Intuitively, an unprocessed string template is in some DSL ?language?; a string template suitable for processing by FMT / String::format (which expects embedded format specifiers like %d) is in a different ?language? than one that is destined to be a SQL query (requires balanced quotes, etc.) The natural interpretation is that when we embed an ST fragment in another ST, the embedded ST fragment should be in the same ?language?, and that ?processors? should flatten embedded STs. If you are using string templates in a code generator, where you build up a class out of smaller fragments (possibly recursively), this works quite naturally; we can start with a class template ?class \{name} { \{body} }? and build the body separately, and the body may be built from method templates whose implementation is further embedded. Similarly, SQL queries are often built from pieces (which columns, which tables, join criteria, filter criteria) are often built separately and combined; this works cleanly with embedding ?sub-templates? as well. Similar with recursive formats like JSON. What this means is that ?processors? need to be prepared to see embedded STs, and flatten accordingly. There may be some library helpers for this; TBD. -------------- next part -------------- An HTML attachment was scrubbed... URL: From redio.development at gmail.com Wed Mar 13 12:16:21 2024 From: redio.development at gmail.com (Red IO) Date: Wed, 13 Mar 2024 13:16:21 +0100 Subject: Comparing ranges with switch Message-ID: The switch statement saw a huge transformation over the past few releases. So I was quite surprised to realize that the current switch construct can't check the range of an value. Example case x is between y and z. I'm most likely not the first one to notice that. Is there any discussion about adding some sort of range pattern? Would fit in the notion of the switch checking patterns quite well. Great regards RedIODev -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Wed Mar 13 13:09:57 2024 From: davidalayachew at gmail.com (David Alayachew) Date: Wed, 13 Mar 2024 09:09:57 -0400 Subject: Comparing ranges with switch In-Reply-To: References: Message-ID: Brian himself told me it is on the way, there are just higher priority amber features (for example, member patterns) that need to be worked on first. Which makes sense, range patterns are incredibly useful and feel like they are filling a hole in the language, but they only apply to numbers currently. Maybe to more after Valhalla releases some stuff. On Wed, Mar 13, 2024, 8:17?AM Red IO wrote: > The switch statement saw a huge transformation over the past few releases. > So I was quite surprised to realize that the current switch construct can't > check the range of an value. > Example case x is between y and z. > I'm most likely not the first one to notice that. Is there any discussion > about adding some sort of range pattern? Would fit in the notion of the > switch checking patterns quite well. > > Great regards > RedIODev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed Mar 13 14:09:32 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 13 Mar 2024 10:09:32 -0400 Subject: Comparing ranges with switch In-Reply-To: References: Message-ID: <80eacecc-876d-48f6-b1db-0c491a08d7c8@oracle.com> As you observe, this is an entirely reasonable fit for switch.? However, if we were to just do labels like: ??? case 1..10: this would fall into the category of "ad-hoc syntax-oriented feature", and our budget for such things is limited (and arguably overspent.)? To make such a feature worthwhile, we'd want to address ranges more holistically, such as support in loops, arrays, ranges of other ordered types (such as long or enums), etc.? Which is a totally reasonable feature to consider, but is also more significant in scope. On 3/13/2024 8:16 AM, Red IO wrote: > The switch statement saw a huge transformation over the past few > releases. So I was quite surprised to realize that the current switch > construct can't check the range of an value. > Example case x is between y and z. > I'm most likely not the first one to notice that. Is there any > discussion about adding some sort of range pattern? Would fit in the > notion of the switch checking patterns quite well. > > Great regards > RedIODev -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Wed Mar 13 15:40:11 2024 From: davidalayachew at gmail.com (David Alayachew) Date: Wed, 13 Mar 2024 11:40:11 -0400 Subject: Comparing ranges with switch In-Reply-To: <80eacecc-876d-48f6-b1db-0c491a08d7c8@oracle.com> References: <80eacecc-876d-48f6-b1db-0c491a08d7c8@oracle.com> Message-ID: Hijacking for a second -- Where might one say we have overspent our ad-hoc budget? Purely out of curiosity. But very excited to hear that this might also extend to loops and whatnot. That would simplify a lot of code out there. On Wed, Mar 13, 2024 at 10:10?AM Brian Goetz wrote: > As you observe, this is an entirely reasonable fit for switch. However, > if we were to just do labels like: > > case 1..10: > > this would fall into the category of "ad-hoc syntax-oriented feature", and > our budget for such things is limited (and arguably overspent.) To make > such a feature worthwhile, we'd want to address ranges more holistically, > such as support in loops, arrays, ranges of other ordered types (such as > long or enums), etc. Which is a totally reasonable feature to consider, > but is also more significant in scope. > > > > On 3/13/2024 8:16 AM, Red IO wrote: > > The switch statement saw a huge transformation over the past few releases. > So I was quite surprised to realize that the current switch construct can't > check the range of an value. > Example case x is between y and z. > I'm most likely not the first one to notice that. Is there any discussion > about adding some sort of range pattern? Would fit in the notion of the > switch checking patterns quite well. > > Great regards > RedIODev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From josiahnoel at gmail.com Wed Mar 13 16:10:39 2024 From: josiahnoel at gmail.com (Josiah Noel) Date: Wed, 13 Mar 2024 12:10:39 -0400 Subject: Comparing ranges with switch In-Reply-To: References: Message-ID: we sorta have this with switch guards switch (intyMcIntFace) { case Integer i when i > 1 && i < 10 -> {} case 42 -> {} default -> throw new IllegalArgumentException("Unexpected value: "); } On Wed, Mar 13, 2024 at 8:16?AM Red IO wrote: > The switch statement saw a huge transformation over the past few releases. > So I was quite surprised to realize that the current switch construct can't > check the range of an value. > Example case x is between y and z. > I'm most likely not the first one to notice that. Is there any discussion > about adding some sort of range pattern? Would fit in the notion of the > switch checking patterns quite well. > > Great regards > RedIODev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Wed Mar 13 16:33:11 2024 From: davidalayachew at gmail.com (David Alayachew) Date: Wed, 13 Mar 2024 12:33:11 -0400 Subject: Comparing ranges with switch In-Reply-To: References: Message-ID: Well, what Brian and RedIO are suggesting would be more like this. byte b = 123; System.out.println ( switch (b) { case -128..-1 -> "It's negative!"; case 0 -> "It's zero!"; case 1..127 -> "It's positive!"; } ) ; The difference is, in this example, it would be EXHAUSTIVE, NO DEFAULT CLAUSE NECESSARY. Your example requires either a default or a type pattern to be exhaustive, as do basically all switches that use when clauses. That exhaustiveness is powerful because -- patterns compose. For example, if you have a complex set of number checks, this switch will tell you if you left a value out. That is the power of exhaustiveness. When you combine it with composition, you can take an arbitrarily complex domain, and the compiler will let you know if you missed a case. On Wed, Mar 13, 2024 at 12:13?PM Josiah Noel wrote: > we sorta have this with switch guards > > switch (intyMcIntFace) { > > case Integer i when i > 1 && i < 10 -> {} > > case 42 -> {} > > > default -> throw new IllegalArgumentException("Unexpected value: "); > > } > > On Wed, Mar 13, 2024 at 8:16?AM Red IO > wrote: > >> The switch statement saw a huge transformation over the past few >> releases. So I was quite surprised to realize that the current switch >> construct can't check the range of an value. >> Example case x is between y and z. >> I'm most likely not the first one to notice that. Is there any discussion >> about adding some sort of range pattern? Would fit in the notion of the >> switch checking patterns quite well. >> >> Great regards >> RedIODev >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed Mar 13 17:40:01 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 13 Mar 2024 13:40:01 -0400 Subject: Comparing ranges with switch In-Reply-To: References: Message-ID: As an added bonus, it becomes practical to get useful exhaustiveness checking for integral types larger than boolean, as this example with byte shows. On 3/13/2024 12:33 PM, David Alayachew wrote: > Well, what Brian and RedIO are suggesting would be more like this. > > byte b = 123; > > System.out.println > ( > switch (b) > ?? { > > case -128..-1? -> "It's negative!"; > case 0???????? -> "It's zero!"; > case 1..127??? -> "It's positive!"; > > ?? } > ) > ; > > The difference is, in this example, it would be EXHAUSTIVE, NO DEFAULT > CLAUSE NECESSARY. Your example requires either a default or a type > pattern to be exhaustive, as do basically all switches that use when > clauses. > > That exhaustiveness is powerful because -- patterns compose. For > example, if you have a complex set of number checks, this switch will > tell you if you left a value out. That is the power of exhaustiveness. > When you combine it with composition, you can take an arbitrarily > complex domain, and the compiler will let you know if you missed a case. > > On Wed, Mar 13, 2024 at 12:13?PM Josiah Noel wrote: > > we sorta have this with switch guards > > switch(intyMcIntFace){ > > caseIntegeri wheni >1&&i <10->{} > > case42->{} > > default->thrownewIllegalArgumentException("Unexpected value: "); > > } > > > On Wed, Mar 13, 2024 at 8:16?AM Red IO > wrote: > > The switch statement saw a huge transformation over the past > few releases. So I was quite surprised to realize that the > current switch construct can't check the range of an value. > Example case x is between y and z. > I'm most likely not the first one to notice that. Is there any > discussion about adding some sort of range pattern? Would fit > in the notion of the switch checking patterns quite well. > > Great regards > RedIODev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.r.rose at oracle.com Wed Mar 13 22:59:31 2024 From: john.r.rose at oracle.com (John Rose) Date: Wed, 13 Mar 2024 15:59:31 -0700 Subject: Comparing ranges with switch In-Reply-To: References: Message-ID: <54D03C1B-9E37-46EC-A381-396DA75454A6@oracle.com> If we make range-comparison a non-primitive, via some sort of user-programmable pattern mechanism, we lose exhaustiveness. But we?d simplify the language, since we?d be ?growing a language? (Guy?s YouTube talk!) via library code, not via adding to the spec. We?d get a switch like this: > switch (b) > ?? { > > case Range.of(-128, -1)? -> "It's negative!"; > case 0???????? -> "It's zero!"; > case Range.of(1, 127)??? -> "It's positive!"; > > ?? } With that approach, the coder needs a concluding case to force exhaustiveness: > ? default -> "It's positive!"; or even > ? default -> throw AssertionError(); That makes the patterns a little less compositional than a hardwired feature, but otherwise it?s a solid tradeoff. Hmm, in principle, it would also be possible to encode ranges in Java?s type system, enough to make some switches like this be exhaustive. (With the right library definitions.) The C2 JIT type system, used for optimizations, is fully aware of integer and long subrange types. (C2 doesn?t do types representing multiple unconnected ranges, as of today.) That range-based type system, in fact, is how C2 would (after inlining) conclude that the above switch was in fact exhaustive. C2 would not bother to compile an unreached path to throw the impossible exception. That is, the JIT would properly optimize the switch as exhaustive, even if the static analysis of javac couldn?t quite get the job done. That?s a normal state of affairs: The JIT always knows more online than javac does offline. To be clear: I would never want to see ranges in the JLS (Java) type system. (IMO the volume of details would be overwhelming, with tricky rules for each operator, and all that for not enough gain.) Hence, if range patterns are user-defined, then exhaustiveness over ranged cases is probably something we have to leave off the table. The JIT would fix it under the covers, but there would be some warts in the source code (the default cases above). On 13 Mar 2024, at 10:40, Brian Goetz wrote: > As an added bonus, it becomes practical to get useful exhaustiveness > checking for integral types larger than boolean, as this example with > byte shows. > > On 3/13/2024 12:33 PM, David Alayachew wrote: >> Well, what Brian and RedIO are suggesting would be more like this. >> >> byte b = 123; >> >> System.out.println >> ( >> switch (b) >> ?? { >> >> case -128..-1? -> "It's negative!"; >> case 0???????? -> "It's zero!"; >> case 1..127??? -> "It's positive!"; >> >> ?? } >> ) >> ; >> >> The difference is, in this example, it would be EXHAUSTIVE, NO >> DEFAULT CLAUSE NECESSARY. Your example requires either a default or a >> type pattern to be exhaustive, as do basically all switches that use >> when clauses. >> >> That exhaustiveness is powerful because -- >> patterns compose. For example, if you have a complex set of number >> checks, this switch will tell you if you left a value out. That is >> the power of exhaustiveness. When you combine it with composition, >> you can take an arbitrarily complex domain, and the compiler will let >> you know if you missed a case. >> >> On Wed, Mar 13, 2024 at 12:13?PM Josiah Noel >> wrote: >> >> we sorta have this with switch guards >> >> switch(intyMcIntFace){ >> >> caseIntegeri wheni >1&&i <10->{} >> >> case42->{} >> >> default->thrownewIllegalArgumentException("Unexpected value: "); >> >> } >> >> >> On Wed, Mar 13, 2024 at 8:16?AM Red IO >> wrote: >> >> The switch statement saw a huge transformation over the past >> few releases. So I was quite surprised to realize that the >> current switch construct can't check the range of an value. >> Example case x is between y and z. >> I'm most likely not the first one to notice that. Is there >> any >> discussion about adding some sort of range pattern? Would fit >> in the notion of the switch checking patterns quite well. >> >> Great regards >> RedIODev >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Thu Mar 14 02:00:51 2024 From: davidalayachew at gmail.com (David Alayachew) Date: Wed, 13 Mar 2024 22:00:51 -0400 Subject: Comparing ranges with switch In-Reply-To: <54D03C1B-9E37-46EC-A381-396DA75454A6@oracle.com> References: <54D03C1B-9E37-46EC-A381-396DA75454A6@oracle.com> Message-ID: Hey John, Thanks for speaking on this. Very disappointing to hear since javac being able to tell you the switch is exhaustive without a default would be incredibly useful, but understood. I would like clarification on one part -- you said "with tricky rules for each operator". I had assumed that range patterns were only for >= and <= operators. Effectively the BETWEEN operator from ORACLE SQL. Do range patterns carry more operators in mind? On Wed, Mar 13, 2024 at 6:59?PM John Rose wrote: > If we make range-comparison a non-primitive, via some sort of > user-programmable pattern mechanism, we lose exhaustiveness. > > But we?d simplify the language, since we?d be ?growing a language? (Guy?s > YouTube talk!) via library code, not via adding to the spec. We?d get a > switch like this: > > switch (b) > { > > case Range.of(-128, -1) -> "It's negative!"; > case 0 -> "It's zero!"; > case Range.of(1, 127) -> "It's positive!"; > > } > > With that approach, the coder needs a concluding case to force > exhaustiveness: > > ? default -> "It's positive!"; > > or even > > ? default -> throw AssertionError(); > > That makes the patterns a little less compositional than a hardwired > feature, but otherwise it?s a solid tradeoff. > > Hmm, in principle, it would also be possible to encode ranges in Java?s > type system, enough to make some switches like this be exhaustive. (With > the right library definitions.) The C2 JIT type system, used for > optimizations, is fully aware of integer and long subrange types. (C2 > doesn?t do types representing multiple unconnected ranges, as of today.) > That range-based type system, in fact, is how C2 would (after inlining) > conclude that the above switch was in fact exhaustive. C2 would not bother > to compile an unreached path to throw the impossible exception. > > That is, the JIT would properly optimize the switch as exhaustive, even if > the static analysis of javac couldn?t quite get the job done. That?s a > normal state of affairs: The JIT always knows more online than javac does > offline. > > To be clear: I would never want to see ranges in the JLS (Java) type > system. (IMO the volume of details would be overwhelming, with tricky rules > for each operator, and all that for not enough gain.) Hence, if range > patterns are user-defined, then exhaustiveness over ranged cases is > probably something we have to leave off the table. The JIT would fix it > under the covers, but there would be some warts in the source code (the > default cases above). > > On 13 Mar 2024, at 10:40, Brian Goetz wrote: > > As an added bonus, it becomes practical to get useful exhaustiveness > checking for integral types larger than boolean, as this example with byte > shows. > > On 3/13/2024 12:33 PM, David Alayachew wrote: > > Well, what Brian and RedIO are suggesting would be more like this. > > byte b = 123; > > System.out.println > ( > switch (b) > { > > case -128..-1 -> "It's negative!"; > case 0 -> "It's zero!"; > case 1..127 -> "It's positive!"; > > } > ) > ; > > The difference is, in this example, it would be EXHAUSTIVE, NO DEFAULT > CLAUSE NECESSARY. Your example requires either a default or a type pattern > to be exhaustive, as do basically all switches that use when clauses. > > That exhaustiveness is powerful because -- patterns compose. For example, > if you have a complex set of number checks, this switch will tell you if > you left a value out. That is the power of exhaustiveness. When you combine > it with composition, you can take an arbitrarily complex domain, and the > compiler will let you know if you missed a case. > > On Wed, Mar 13, 2024 at 12:13?PM Josiah Noel wrote: > >> we sorta have this with switch guards >> >> switch (intyMcIntFace) { >> >> case Integer i when i > 1 && i < 10 -> {} >> >> case 42 -> {} >> >> default -> throw new IllegalArgumentException("Unexpected value: "); >> >> } >> >> On Wed, Mar 13, 2024 at 8:16?AM Red IO >> wrote: >> >>> The switch statement saw a huge transformation over the past few >>> releases. So I was quite surprised to realize that the current switch >>> construct can't check the range of an value. >>> Example case x is between y and z. >>> I'm most likely not the first one to notice that. Is there any >>> discussion about adding some sort of range pattern? Would fit in the notion >>> of the switch checking patterns quite well. >>> >>> Great regards >>> RedIODev >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From redio.development at gmail.com Fri Mar 15 10:00:36 2024 From: redio.development at gmail.com (Red IO) Date: Fri, 15 Mar 2024 11:00:36 +0100 Subject: Comparing ranges with switch In-Reply-To: <54D03C1B-9E37-46EC-A381-396DA75454A6@oracle.com> References: <54D03C1B-9E37-46EC-A381-396DA75454A6@oracle.com> Message-ID: I misclicked and only replied to John. I don't quite understand why we need to leave exhaustiveness on the table, just because it's difficult to implement for custom types. Also I don't think it's that complicated. Custom types just need open ranges in the exhaustive patterns. case foo.. -> case ..bar -> Sure you can't check if foo is correctly larger than than bar. But this would need to be checked at runtime and throw an exception if the case. You could easily implement custom ranges by requiring the type to be a specialized iterable (that takes a starting point) and comparable. That way you can iterate the range and choose the correct branch for range comparisons. Great regards RedIODev On Wed, Mar 13, 2024, 23:59 John Rose wrote: > If we make range-comparison a non-primitive, via some sort of > user-programmable pattern mechanism, we lose exhaustiveness. > > But we?d simplify the language, since we?d be ?growing a language? (Guy?s > YouTube talk!) via library code, not via adding to the spec. We?d get a > switch like this: > > switch (b) > { > > case Range.of(-128, -1) -> "It's negative!"; > case 0 -> "It's zero!"; > case Range.of(1, 127) -> "It's positive!"; > > } > > With that approach, the coder needs a concluding case to force > exhaustiveness: > > ? default -> "It's positive!"; > > or even > > ? default -> throw AssertionError(); > > That makes the patterns a little less compositional than a hardwired > feature, but otherwise it?s a solid tradeoff. > > Hmm, in principle, it would also be possible to encode ranges in Java?s > type system, enough to make some switches like this be exhaustive. (With > the right library definitions.) The C2 JIT type system, used for > optimizations, is fully aware of integer and long subrange types. (C2 > doesn?t do types representing multiple unconnected ranges, as of today.) > That range-based type system, in fact, is how C2 would (after inlining) > conclude that the above switch was in fact exhaustive. C2 would not bother > to compile an unreached path to throw the impossible exception. > > That is, the JIT would properly optimize the switch as exhaustive, even if > the static analysis of javac couldn?t quite get the job done. That?s a > normal state of affairs: The JIT always knows more online than javac does > offline. > > To be clear: I would never want to see ranges in the JLS (Java) type > system. (IMO the volume of details would be overwhelming, with tricky rules > for each operator, and all that for not enough gain.) Hence, if range > patterns are user-defined, then exhaustiveness over ranged cases is > probably something we have to leave off the table. The JIT would fix it > under the covers, but there would be some warts in the source code (the > default cases above). > > On 13 Mar 2024, at 10:40, Brian Goetz wrote: > > As an added bonus, it becomes practical to get useful exhaustiveness > checking for integral types larger than boolean, as this example with byte > shows. > > On 3/13/2024 12:33 PM, David Alayachew wrote: > > Well, what Brian and RedIO are suggesting would be more like this. > > byte b = 123; > > System.out.println > ( > switch (b) > { > > case -128..-1 -> "It's negative!"; > case 0 -> "It's zero!"; > case 1..127 -> "It's positive!"; > > } > ) > ; > > The difference is, in this example, it would be EXHAUSTIVE, NO DEFAULT > CLAUSE NECESSARY. Your example requires either a default or a type pattern > to be exhaustive, as do basically all switches that use when clauses. > > That exhaustiveness is powerful because -- patterns compose. For example, > if you have a complex set of number checks, this switch will tell you if > you left a value out. That is the power of exhaustiveness. When you combine > it with composition, you can take an arbitrarily complex domain, and the > compiler will let you know if you missed a case. > > On Wed, Mar 13, 2024 at 12:13?PM Josiah Noel wrote: > >> we sorta have this with switch guards >> >> switch (intyMcIntFace) { >> >> case Integer i when i > 1 && i < 10 -> {} >> >> case 42 -> {} >> >> default -> throw new IllegalArgumentException("Unexpected value: "); >> >> } >> >> On Wed, Mar 13, 2024 at 8:16?AM Red IO >> wrote: >> >>> The switch statement saw a huge transformation over the past few >>> releases. So I was quite surprised to realize that the current switch >>> construct can't check the range of an value. >>> Example case x is between y and z. >>> I'm most likely not the first one to notice that. Is there any >>> discussion about adding some sort of range pattern? Would fit in the notion >>> of the switch checking patterns quite well. >>> >>> Great regards >>> RedIODev >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From justin.spindler at gmail.com Sun Mar 17 15:09:06 2024 From: justin.spindler at gmail.com (Justin Spindler) Date: Sun, 17 Mar 2024 11:09:06 -0400 Subject: StringTemplates deferred evaluation Message-ID: I was toying around with the second preview of StringTemplates and I had a question regarding their design. I was wondering if it had been considered for the embedded expressions to be evaluated lazily? One of the first use cases that came to mind when I was exploring StringTemplates is logging. That is an extremely common case where we want to produce a form of interpolated value, and the current syntax generally has the same concerns that a formatted string would, in that the inputs are removed from where they are defined within the message format. However, if the log message is below the log level threshold you generally don't want to incur the cost of building the message, including evaluating those embedded expressions. Log libraries typically offer a mechanism to defer evaluation via Suppliers, but that feels like it would be a challenge to use within StringTemplate. C# is an example of a language that offers this functionality via FormattableString, which gives a method the ability to choose whether or not to interpret the template or evaluate the expressions. That allows logging below threshold to be more or less a no-op. -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Sun Mar 17 16:16:59 2024 From: davidalayachew at gmail.com (David Alayachew) Date: Sun, 17 Mar 2024 12:16:59 -0400 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: I feel like that would be on the logging libraries to provide, not so much the language. Let's say that your problem (as I understand it) is like this. log.debug("metrics = \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}"); Sounds to me like the solution is this instead. log.debug(() -> "metrics = \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}" ) So, your logging library just needs to add a new overload for a Supplier, and then this problem is solved entirely outside of the language. Log4J and friends are pretty good about keeping up with new additions, so it should not take long. I know it's a little less convenient, but doing it this way helps keep out complexity from the feature, and only introduces it where necessary (and it's only necessary at use-site). Would this meet your needs? On Sun, Mar 17, 2024 at 11:15?AM Justin Spindler wrote: > I was toying around with the second preview of StringTemplates and I had a > question regarding their design. I was wondering if it had been considered > for the embedded expressions to be evaluated lazily? > > One of the first use cases that came to mind when I was exploring > StringTemplates is logging. That is an extremely common case where we want > to produce a form of interpolated value, and the current syntax generally > has the same concerns that a formatted string would, in that the inputs are > removed from where they are defined within the message format. However, if > the log message is below the log level threshold you generally don't want > to incur the cost of building the message, including evaluating those > embedded expressions. Log libraries typically offer a mechanism to defer > evaluation via Suppliers, but that feels like it would be a challenge to > use within StringTemplate. > > C# is an example of a language that offers this functionality via > FormattableString, which gives a method the ability to choose whether or > not to interpret the template or evaluate the expressions. That allows > logging below threshold to be more or less a no-op. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kan.izh at gmail.com Sun Mar 17 16:28:24 2024 From: kan.izh at gmail.com (Anatoly Kupriyanov) Date: Sun, 17 Mar 2024 16:28:24 +0000 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: But the "() ->" thing is still a runtime object to be allocated on the heap... or is the JIT smart enough to always optimize it out? On Sun, 17 Mar 2024 at 16:17, David Alayachew wrote: > I feel like that would be on the logging libraries to provide, not so much > the language. > > Let's say that your problem (as I understand it) is like this. > > log.debug("metrics = > \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}"); > > Sounds to me like the solution is this instead. > > log.debug(() -> "metrics = > \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}" > ) > > So, your logging library just needs to add a new overload for a > Supplier, and then this problem is solved entirely outside > of the language. Log4J and friends are pretty good about keeping up with > new additions, so it should not take long. > > I know it's a little less convenient, but doing it this way helps keep out > complexity from the feature, and only introduces it where necessary (and > it's only necessary at use-site). > > Would this meet your needs? > > On Sun, Mar 17, 2024 at 11:15?AM Justin Spindler < > justin.spindler at gmail.com> wrote: > >> I was toying around with the second preview of StringTemplates and I had >> a question regarding their design. I was wondering if it had been >> considered for the embedded expressions to be evaluated lazily? >> >> One of the first use cases that came to mind when I was exploring >> StringTemplates is logging. That is an extremely common case where we want >> to produce a form of interpolated value, and the current syntax generally >> has the same concerns that a formatted string would, in that the inputs are >> removed from where they are defined within the message format. However, if >> the log message is below the log level threshold you generally don't want >> to incur the cost of building the message, including evaluating those >> embedded expressions. Log libraries typically offer a mechanism to defer >> evaluation via Suppliers, but that feels like it would be a challenge to >> use within StringTemplate. >> >> C# is an example of a language that offers this functionality via >> FormattableString, which gives a method the ability to choose whether or >> not to interpret the template or evaluate the expressions. That allows >> logging below threshold to be more or less a no-op. >> >> >> -- WBR, Anatoly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Sun Mar 17 16:40:51 2024 From: davidalayachew at gmail.com (David Alayachew) Date: Sun, 17 Mar 2024 12:40:51 -0400 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: The cost would be no different than evaluating a StringTemplate lazily. Utlimately, log.debug(blah) is still a method call, and therefore, some object is going into the method. Whether we evaluate the StringTemplate lazily or the Supplier lazily is largely the same thing. Or am I misunderstanding you? On Sun, Mar 17, 2024 at 12:28?PM Anatoly Kupriyanov wrote: > But the "() ->" thing is still a runtime object to be allocated on the > heap... or is the JIT smart enough to always optimize it out? > > On Sun, 17 Mar 2024 at 16:17, David Alayachew > wrote: > >> I feel like that would be on the logging libraries to provide, not so >> much the language. >> >> Let's say that your problem (as I understand it) is like this. >> >> log.debug("metrics = >> \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}"); >> >> Sounds to me like the solution is this instead. >> >> log.debug(() -> "metrics = >> \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}" >> ) >> >> So, your logging library just needs to add a new overload for a >> Supplier, and then this problem is solved entirely outside >> of the language. Log4J and friends are pretty good about keeping up with >> new additions, so it should not take long. >> >> I know it's a little less convenient, but doing it this way helps keep >> out complexity from the feature, and only introduces it where necessary >> (and it's only necessary at use-site). >> >> Would this meet your needs? >> >> On Sun, Mar 17, 2024 at 11:15?AM Justin Spindler < >> justin.spindler at gmail.com> wrote: >> >>> I was toying around with the second preview of StringTemplates and I had >>> a question regarding their design. I was wondering if it had been >>> considered for the embedded expressions to be evaluated lazily? >>> >>> One of the first use cases that came to mind when I was exploring >>> StringTemplates is logging. That is an extremely common case where we want >>> to produce a form of interpolated value, and the current syntax generally >>> has the same concerns that a formatted string would, in that the inputs are >>> removed from where they are defined within the message format. However, if >>> the log message is below the log level threshold you generally don't want >>> to incur the cost of building the message, including evaluating those >>> embedded expressions. Log libraries typically offer a mechanism to defer >>> evaluation via Suppliers, but that feels like it would be a challenge to >>> use within StringTemplate. >>> >>> C# is an example of a language that offers this functionality via >>> FormattableString, which gives a method the ability to choose whether or >>> not to interpret the template or evaluate the expressions. That allows >>> logging below threshold to be more or less a no-op. >>> >>> >>> > > -- > WBR, Anatoly. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kan.izh at gmail.com Sun Mar 17 17:07:19 2024 From: kan.izh at gmail.com (Anatoly Kupriyanov) Date: Sun, 17 Mar 2024 17:07:19 +0000 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: Classically (how it was done a decade or so ago) the logger method would look like: void debug(String template, Object arg1, Object arg2) { if(!enabled) return; writeLog(interpolate(template, arg1.toString(), arg2.toString())); } That allows that if enabled==false, then it will be zero-gc, guaranteed. It even used a lot of overrides with zero args, one arg, two args to avoid ...-ellipsis array allocation. E.g. see https://www.slf4j.org/api/org/slf4j/Logger.html There is a C# hacky approach the String Interpolation Handler ( https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/interpolated-string-handler ) to use struct (allocated on stack) and magical isEnabled flags to cover logging effectively. I don't see how it would be possible to use the StringTemplate to achieve the similar effectiveness, unless there are some good and reliable optimizations in JIT to avoid allocations. On Sun, 17 Mar 2024 at 16:41, David Alayachew wrote: > The cost would be no different than evaluating a StringTemplate lazily. > Utlimately, log.debug(blah) is still a method call, and therefore, some > object is going into the method. Whether we evaluate the StringTemplate > lazily or the Supplier lazily is largely the same thing. > > Or am I misunderstanding you? > > On Sun, Mar 17, 2024 at 12:28?PM Anatoly Kupriyanov > wrote: > >> But the "() ->" thing is still a runtime object to be allocated on the >> heap... or is the JIT smart enough to always optimize it out? >> >> On Sun, 17 Mar 2024 at 16:17, David Alayachew >> wrote: >> >>> I feel like that would be on the logging libraries to provide, not so >>> much the language. >>> >>> Let's say that your problem (as I understand it) is like this. >>> >>> log.debug("metrics = >>> \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}"); >>> >>> Sounds to me like the solution is this instead. >>> >>> log.debug(() -> "metrics = >>> \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}" >>> ) >>> >>> So, your logging library just needs to add a new overload for a >>> Supplier, and then this problem is solved entirely outside >>> of the language. Log4J and friends are pretty good about keeping up with >>> new additions, so it should not take long. >>> >>> I know it's a little less convenient, but doing it this way helps keep >>> out complexity from the feature, and only introduces it where necessary >>> (and it's only necessary at use-site). >>> >>> Would this meet your needs? >>> >>> On Sun, Mar 17, 2024 at 11:15?AM Justin Spindler < >>> justin.spindler at gmail.com> wrote: >>> >>>> I was toying around with the second preview of StringTemplates and I >>>> had a question regarding their design. I was wondering if it had been >>>> considered for the embedded expressions to be evaluated lazily? >>>> >>>> One of the first use cases that came to mind when I was exploring >>>> StringTemplates is logging. That is an extremely common case where we want >>>> to produce a form of interpolated value, and the current syntax generally >>>> has the same concerns that a formatted string would, in that the inputs are >>>> removed from where they are defined within the message format. However, if >>>> the log message is below the log level threshold you generally don't want >>>> to incur the cost of building the message, including evaluating those >>>> embedded expressions. Log libraries typically offer a mechanism to defer >>>> evaluation via Suppliers, but that feels like it would be a challenge to >>>> use within StringTemplate. >>>> >>>> C# is an example of a language that offers this functionality via >>>> FormattableString, which gives a method the ability to choose whether or >>>> not to interpret the template or evaluate the expressions. That allows >>>> logging below threshold to be more or less a no-op. >>>> >>>> >>>> >> >> -- >> WBR, Anatoly. >> > -- WBR, Anatoly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.laskey at oracle.com Sun Mar 17 17:28:07 2024 From: james.laskey at oracle.com (Jim Laskey) Date: Sun, 17 Mar 2024 17:28:07 +0000 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: There was a discussion about this on this list a while back. The issue is that deferred evaluation could provide a gapping vulnerability. You can get the effect you want by using Suppler objects as embedded expressions. I posted a logging example using a timestamp Supplier. The Suppler was evaluated by the processor. In the new world we are working on, the same thing can done. The plan is to provide a mapValues method as in st.mapValues(v -> v instanceof Supplier S ? S.get() : v). ? Jim ? > On Mar 17, 2024, at 12:14?PM, Justin Spindler wrote: > > ? > I was toying around with the second preview of StringTemplates and I had a question regarding their design. I was wondering if it had been considered for the embedded expressions to be evaluated lazily? > > One of the first use cases that came to mind when I was exploring StringTemplates is logging. That is an extremely common case where we want to produce a form of interpolated value, and the current syntax generally has the same concerns that a formatted string would, in that the inputs are removed from where they are defined within the message format. However, if the log message is below the log level threshold you generally don't want to incur the cost of building the message, including evaluating those embedded expressions. Log libraries typically offer a mechanism to defer evaluation via Suppliers, but that feels like it would be a challenge to use within StringTemplate. > > C# is an example of a language that offers this functionality via FormattableString, which gives a method the ability to choose whether or not to interpret the template or evaluate the expressions. That allows logging below threshold to be more or less a no-op. > > From davidalayachew at gmail.com Sun Mar 17 17:35:38 2024 From: davidalayachew at gmail.com (David Alayachew) Date: Sun, 17 Mar 2024 13:35:38 -0400 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: Thank you for the code example. That helps clarify your desired behaviour. Ok, if your intent is to have absolutely no garbage collection whatsoever, then yes, my solution does not guarantee that. How much the JIT will help or optimize, I cannot say. That said, a String is a "blessed type" in Java. It has a handful of optimizations that are specific to it and it alone. I am actually expecting that StringTemplates will receive at least a small taste of this sort of optimization. Furthermore, the cost of creating a StringTemplate is rarely, if ever, the expensive part of the operation. In fact, I am certain it is trivial, if not optimized away like you were asking. I won't presume to call this premature optimization -- perhaps your application needs require it. But if so, I would first wait and see (or better yet, calculate!) the performance metrics of these StringTemplates. If they fail to meet your performance needs, then communicate your performance needs to this mailing list, with a hard example (and ideally metrics!) in tow. On Sun, Mar 17, 2024 at 1:07?PM Anatoly Kupriyanov wrote: > Classically (how it was done a decade or so ago) the logger method would > look like: > void debug(String template, Object arg1, Object arg2) { > if(!enabled) return; > writeLog(interpolate(template, arg1.toString(), arg2.toString())); > } > > That allows that if enabled==false, then it will be zero-gc, guaranteed. > It even used a lot of overrides with zero args, one arg, two args to avoid > ...-ellipsis array allocation. E.g. see > https://www.slf4j.org/api/org/slf4j/Logger.html > > There is a C# hacky approach the String Interpolation Handler ( > https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/interpolated-string-handler > ) to use struct (allocated on stack) and magical isEnabled flags to cover > logging effectively. > > I don't see how it would be possible to use the StringTemplate to achieve > the similar effectiveness, unless there are some good and reliable > optimizations in JIT to avoid allocations. > > On Sun, 17 Mar 2024 at 16:41, David Alayachew > wrote: > >> The cost would be no different than evaluating a StringTemplate lazily. >> Utlimately, log.debug(blah) is still a method call, and therefore, some >> object is going into the method. Whether we evaluate the StringTemplate >> lazily or the Supplier lazily is largely the same thing. >> >> Or am I misunderstanding you? >> >> On Sun, Mar 17, 2024 at 12:28?PM Anatoly Kupriyanov >> wrote: >> >>> But the "() ->" thing is still a runtime object to be allocated on the >>> heap... or is the JIT smart enough to always optimize it out? >>> >>> On Sun, 17 Mar 2024 at 16:17, David Alayachew >>> wrote: >>> >>>> I feel like that would be on the logging libraries to provide, not so >>>> much the language. >>>> >>>> Let's say that your problem (as I understand it) is like this. >>>> >>>> log.debug("metrics = >>>> \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}"); >>>> >>>> Sounds to me like the solution is this instead. >>>> >>>> log.debug(() -> "metrics = >>>> \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}" >>>> ) >>>> >>>> So, your logging library just needs to add a new overload for a >>>> Supplier, and then this problem is solved entirely outside >>>> of the language. Log4J and friends are pretty good about keeping up with >>>> new additions, so it should not take long. >>>> >>>> I know it's a little less convenient, but doing it this way helps keep >>>> out complexity from the feature, and only introduces it where necessary >>>> (and it's only necessary at use-site). >>>> >>>> Would this meet your needs? >>>> >>>> On Sun, Mar 17, 2024 at 11:15?AM Justin Spindler < >>>> justin.spindler at gmail.com> wrote: >>>> >>>>> I was toying around with the second preview of StringTemplates and I >>>>> had a question regarding their design. I was wondering if it had been >>>>> considered for the embedded expressions to be evaluated lazily? >>>>> >>>>> One of the first use cases that came to mind when I was exploring >>>>> StringTemplates is logging. That is an extremely common case where we want >>>>> to produce a form of interpolated value, and the current syntax generally >>>>> has the same concerns that a formatted string would, in that the inputs are >>>>> removed from where they are defined within the message format. However, if >>>>> the log message is below the log level threshold you generally don't want >>>>> to incur the cost of building the message, including evaluating those >>>>> embedded expressions. Log libraries typically offer a mechanism to defer >>>>> evaluation via Suppliers, but that feels like it would be a challenge to >>>>> use within StringTemplate. >>>>> >>>>> C# is an example of a language that offers this functionality via >>>>> FormattableString, which gives a method the ability to choose whether or >>>>> not to interpret the template or evaluate the expressions. That allows >>>>> logging below threshold to be more or less a no-op. >>>>> >>>>> >>>>> >>> >>> -- >>> WBR, Anatoly. >>> >> > > -- > WBR, Anatoly. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From justin.spindler at gmail.com Sun Mar 17 17:39:24 2024 From: justin.spindler at gmail.com (Justin Spindler) Date: Sun, 17 Mar 2024 13:39:24 -0400 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: > The issue is that deferred evaluation could provide a gapping vulnerability. I'd be curious about this vulnerability. > You can get the effect you want by using Suppler objects as embedded expressions. Sure, but that increases the amount of code that the developer needs to write pretty substantially, especially since there are no natural types for lambdas so the developer would have to assign that to a typed variable ahead of time or cast explicitly to a Supplier in the embedded expression. FOO."This is a test: \{(Supplier) () -> calc(x, y, z)}" It's fine if this isn't a scenario that the team hasn't considered, or believes that there are sufficient workarounds. I will say that it's not that uncommon that I find logging code at trace or debug levels that ends up calculating complex strings (even stacktraces) which is noticeable enough to cause performance regressions. We do try to emphasize using the Supplier overloads of log4j methods to avoid that calculation, but it's certainly not a pit of success. On Sun, Mar 17, 2024 at 1:28?PM Jim Laskey wrote: > There was a discussion about this on this list a while back. The issue is > that deferred evaluation could provide a gapping vulnerability. > > You can get the effect you want by using Suppler objects as embedded > expressions. I posted a logging example using a timestamp Supplier. The > Suppler was evaluated by the processor. > > In the new world we are working on, the same thing can done. The plan is > to provide a mapValues method as in st.mapValues(v -> v instanceof Supplier > S ? S.get() : v). > > ? Jim > > ? > > > On Mar 17, 2024, at 12:14?PM, Justin Spindler > wrote: > > > > ? > > I was toying around with the second preview of StringTemplates and I had > a question regarding their design. I was wondering if it had been > considered for the embedded expressions to be evaluated lazily? > > > > One of the first use cases that came to mind when I was exploring > StringTemplates is logging. That is an extremely common case where we want > to produce a form of interpolated value, and the current syntax generally > has the same concerns that a formatted string would, in that the inputs are > removed from where they are defined within the message format. However, if > the log message is below the log level threshold you generally don't want > to incur the cost of building the message, including evaluating those > embedded expressions. Log libraries typically offer a mechanism to defer > evaluation via Suppliers, but that feels like it would be a challenge to > use within StringTemplate. > > > > C# is an example of a language that offers this functionality via > FormattableString, which gives a method the ability to choose whether or > not to interpret the template or evaluate the expressions. That allows > logging below threshold to be more or less a no-op. > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Sun Mar 17 18:38:27 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 17 Mar 2024 14:38:27 -0400 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: Yes, this was considered (because of the exact same use case you mention -- logging).? It turns out to be one of those "seems briefly attractive" ideas. It turns out that *evaluating* the embedded parameters is almost always cheap; it is the production of the final string, including calling `toString` on reference parameters that are not already String, is the expensive part.? (Embedded expressions are usually scalars, which are ultra-cheap to capture.)? So you don't need laziness in capturing the template; you need laziness in evaluating it.? And such laziness is best made explicit. And loggers, as you point out, often provide a Supplier overload, allowing you to do: ?? logger.info(() -> STR."...") The real problem is that the user is being asked to apply the processor to the template too early.? A better strategy is to let the Logger accept StringTemplate: ??? logger.info(RAW."...") Which feeds into the current evolution direction being discussed, just with a slightly different syntax. On 3/17/2024 11:09 AM, Justin Spindler wrote: > I was toying around with?the second preview of StringTemplates and I > had a question regarding their design.? I was wondering if it had been > considered for the embedded expressions to be evaluated lazily? > > One of the first use cases that came to mind when I was exploring > StringTemplates is logging.? That is an extremely common case where we > want to produce a form of interpolated value, and the current syntax > generally has the same concerns that a formatted string would, in that > the inputs are removed from where they are defined within the message > format. However, if the log message is below the log level > threshold?you generally don't want to incur the cost of building the > message, including evaluating those embedded expressions.? Log > libraries typically offer a mechanism to defer evaluation via > Suppliers, but that feels like it would be a challenge to use within > StringTemplate. > > C# is an example of a language that offers this functionality via > FormattableString, which gives a method the ability to choose whether > or not to interpret the template or evaluate the expressions.? That > allows logging below threshold to be more or less a no-op. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kan.izh at gmail.com Sun Mar 17 19:41:41 2024 From: kan.izh at gmail.com (Anatoly Kupriyanov) Date: Sun, 17 Mar 2024 19:41:41 +0000 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: I guess it is not fair to think about it in terms of premature optimization. I am trying to look at it from a logging-framework developer point of view. As a library developer I should give the smallest overhead possible. So library users could safely do LOGGER.debug("Some thing={}, another={}", hugeObject, expensiveObject); and don't care about anything at all as debug is usually disabled. So the debug statement could be used even in a tight computation loop without worries. In other words, logger devs could not assume anything about how their framework would be used, but they will fight for the easiest to use api with the fastest performance. The idea is that the trace is usually disabled and an app could chew-up gigs of data, but while debugging/investigating, the trace could be temporarily enabled and expected to degrade performance. On Sun, 17 Mar 2024 at 17:36, David Alayachew wrote: > Thank you for the code example. That helps clarify your desired behaviour. > > Ok, if your intent is to have absolutely no garbage collection whatsoever, > then yes, my solution does not guarantee that. How much the JIT will help > or optimize, I cannot say. > > That said, a String is a "blessed type" in Java. It has a handful of > optimizations that are specific to it and it alone. I am actually expecting > that StringTemplates will receive at least a small taste of this sort of > optimization. > > Furthermore, the cost of creating a StringTemplate is rarely, if ever, the > expensive part of the operation. In fact, I am certain it is trivial, if > not optimized away like you were asking. > > I won't presume to call this premature optimization -- perhaps your > application needs require it. But if so, I would first wait and see (or > better yet, calculate!) the performance metrics of these StringTemplates. > If they fail to meet your performance needs, then communicate your > performance needs to this mailing list, with a hard example (and ideally > metrics!) in tow. > > On Sun, Mar 17, 2024 at 1:07?PM Anatoly Kupriyanov > wrote: > >> Classically (how it was done a decade or so ago) the logger method would >> look like: >> void debug(String template, Object arg1, Object arg2) { >> if(!enabled) return; >> writeLog(interpolate(template, arg1.toString(), arg2.toString())); >> } >> >> That allows that if enabled==false, then it will be zero-gc, guaranteed. >> It even used a lot of overrides with zero args, one arg, two args to >> avoid ...-ellipsis array allocation. E.g. see >> https://www.slf4j.org/api/org/slf4j/Logger.html >> >> There is a C# hacky approach the String Interpolation Handler ( >> https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/interpolated-string-handler >> ) to use struct (allocated on stack) and magical isEnabled flags to cover >> logging effectively. >> >> I don't see how it would be possible to use the StringTemplate to achieve >> the similar effectiveness, unless there are some good and reliable >> optimizations in JIT to avoid allocations. >> >> On Sun, 17 Mar 2024 at 16:41, David Alayachew >> wrote: >> >>> The cost would be no different than evaluating a StringTemplate lazily. >>> Utlimately, log.debug(blah) is still a method call, and therefore, some >>> object is going into the method. Whether we evaluate the StringTemplate >>> lazily or the Supplier lazily is largely the same thing. >>> >>> Or am I misunderstanding you? >>> >>> On Sun, Mar 17, 2024 at 12:28?PM Anatoly Kupriyanov >>> wrote: >>> >>>> But the "() ->" thing is still a runtime object to be allocated on the >>>> heap... or is the JIT smart enough to always optimize it out? >>>> >>>> On Sun, 17 Mar 2024 at 16:17, David Alayachew >>>> wrote: >>>> >>>>> I feel like that would be on the logging libraries to provide, not so >>>>> much the language. >>>>> >>>>> Let's say that your problem (as I understand it) is like this. >>>>> >>>>> log.debug("metrics = >>>>> \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}"); >>>>> >>>>> Sounds to me like the solution is this instead. >>>>> >>>>> log.debug(() -> "metrics = >>>>> \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}" >>>>> ) >>>>> >>>>> So, your logging library just needs to add a new overload for a >>>>> Supplier, and then this problem is solved entirely outside >>>>> of the language. Log4J and friends are pretty good about keeping up with >>>>> new additions, so it should not take long. >>>>> >>>>> I know it's a little less convenient, but doing it this way helps keep >>>>> out complexity from the feature, and only introduces it where necessary >>>>> (and it's only necessary at use-site). >>>>> >>>>> Would this meet your needs? >>>>> >>>>> On Sun, Mar 17, 2024 at 11:15?AM Justin Spindler < >>>>> justin.spindler at gmail.com> wrote: >>>>> >>>>>> I was toying around with the second preview of StringTemplates and I >>>>>> had a question regarding their design. I was wondering if it had been >>>>>> considered for the embedded expressions to be evaluated lazily? >>>>>> >>>>>> One of the first use cases that came to mind when I was exploring >>>>>> StringTemplates is logging. That is an extremely common case where we want >>>>>> to produce a form of interpolated value, and the current syntax generally >>>>>> has the same concerns that a formatted string would, in that the inputs are >>>>>> removed from where they are defined within the message format. However, if >>>>>> the log message is below the log level threshold you generally don't want >>>>>> to incur the cost of building the message, including evaluating those >>>>>> embedded expressions. Log libraries typically offer a mechanism to defer >>>>>> evaluation via Suppliers, but that feels like it would be a challenge to >>>>>> use within StringTemplate. >>>>>> >>>>>> C# is an example of a language that offers this functionality via >>>>>> FormattableString, which gives a method the ability to choose whether or >>>>>> not to interpret the template or evaluate the expressions. That allows >>>>>> logging below threshold to be more or less a no-op. >>>>>> >>>>>> >>>>>> >>>> >>>> -- >>>> WBR, Anatoly. >>>> >>> >> >> -- >> WBR, Anatoly. >> > -- WBR, Anatoly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Sun Mar 17 19:45:29 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 17 Mar 2024 15:45:29 -0400 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: It depends on how you measure "don't care anything".? Even this, which is pretty fast, still creates an object with a list containing { hugeObject, expensiveObject }. This is pretty cheap but not necessarily free.? (It may be free if the debug() call gets inlined.) I think for 9x% of the cases, this is entirely cheap enough; for those cases where it is not, the old fussy technique of ??? if (isDebugLogging) ??????? logger.debug(...) will recapture the rest. On 3/17/2024 3:41 PM, Anatoly Kupriyanov wrote: > I guess it is not fair to think about it in terms of premature > optimization. I am trying to look at it from a logging-framework > developer point of view. As a library developer I should give the > smallest overhead possible. So library users could safely do > ? LOGGER.debug("Some thing={}, another={}", hugeObject, expensiveObject); > and don't care about anything at all as debug is usually disabled. So > the debug statement could be used even in a tight computation loop > without worries. In other words, logger devs could not assume anything > about how their framework would be used, but they will fight for the > easiest to use api with the fastest performance. > The idea is that the trace is usually disabled and an app could > chew-up gigs of data, but while debugging/investigating, the trace > could be temporarily enabled and expected to degrade performance. > > > On Sun, 17 Mar 2024 at 17:36, David Alayachew > wrote: > > Thank you for the code example. That helps clarify your desired > behaviour. > > Ok, if your intent is to have absolutely no garbage collection > whatsoever, then yes, my solution does not guarantee that. How > much the JIT will help or optimize, I cannot say. > > That said, a String is a "blessed type" in Java. It has a handful > of optimizations that are specific to it and it alone. I am > actually expecting that StringTemplates will receive at least a > small taste of this sort of optimization. > > Furthermore, the cost of creating a StringTemplate is rarely, if > ever, the expensive part of the operation. In fact, I am certain > it is trivial, if not optimized away like you were asking. > > I won't presume to call this premature optimization -- perhaps > your application needs require it. But if so, I would first wait > and see (or better yet, calculate!) the performance metrics of > these StringTemplates. If they fail to meet your performance > needs, then communicate your performance needs to this mailing > list, with a hard example (and ideally metrics!) in tow. > > On Sun, Mar 17, 2024 at 1:07?PM Anatoly Kupriyanov > wrote: > > Classically (how it was done a decade or so ago) the logger > method would look like: > void debug(String template, Object arg1, Object arg2) { > ? if(!enabled) return; > writeLog(interpolate(template, arg1.toString(), arg2.toString())); > } > > That allows that? if enabled==false, then it will be zero-gc, > guaranteed. > It even used a lot of overrides with zero args, one arg, two > args to avoid ...-ellipsis array allocation. E.g. see > https://www.slf4j.org/api/org/slf4j/Logger.html > > There is a C# hacky approach the String Interpolation Handler > ( > https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/interpolated-string-handler > ) to use struct (allocated on stack) and magical isEnabled > flags to cover logging effectively. > > I don't see how it would be possible to use the StringTemplate > to achieve the similar effectiveness, unless there are some > good and reliable optimizations in JIT to avoid allocations. > > On Sun, 17 Mar 2024 at 16:41, David Alayachew > wrote: > > The cost would be no different than evaluating a > StringTemplate lazily. Utlimately, log.debug(blah) is > still a method call, and therefore, some object is going > into the method. Whether we evaluate the StringTemplate > lazily or the Supplier lazily is largely > the same thing. > > Or am I misunderstanding you? > > On Sun, Mar 17, 2024 at 12:28?PM Anatoly Kupriyanov > wrote: > > But the "() ->" thing is still a runtime object to be > allocated on the heap... or is the JIT smart enough to > always optimize it out? > > On Sun, 17 Mar 2024 at 16:17, David Alayachew > wrote: > > I feel like that would be on the logging libraries > to provide, not so much the language. > > Let's say that your problem (as I understand it) > is like this. > > log.debug("metrics = > \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}"); > > Sounds to me like the solution is this instead. > > log.debug(() -> "metrics = > \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}") > > So, your logging library just needs to add a new > overload for a Supplier, and then > this problem is solved entirely outside of the > language. Log4J and friends are pretty good about > keeping up with new additions, so it should not > take long. > > I know it's a little less convenient, but doing it > this way helps keep out complexity from the > feature, and only introduces it where necessary > (and it's only necessary at use-site). > > Would this meet your needs? > > On Sun, Mar 17, 2024 at 11:15?AM Justin Spindler > wrote: > > I was toying around with?the second preview of > StringTemplates and I had a question regarding > their design.? I was wondering if it had been > considered for the embedded expressions to be > evaluated lazily? > > One of the first use cases that came to mind > when I was exploring StringTemplates is > logging.? That is an extremely common case > where we want to produce a form of > interpolated value, and the current syntax > generally has the same concerns that a > formatted string would, in that the inputs are > removed from where they are defined within the > message format. However, if the log message is > below the log level threshold?you generally > don't want to incur the cost of building the > message, including evaluating those embedded > expressions.? Log libraries typically offer a > mechanism to defer evaluation via Suppliers, > but that feels like it would be a challenge to > use within StringTemplate. > > C# is an example of a language that offers > this functionality via FormattableString, > which gives a method the ability to choose > whether or not to interpret the template or > evaluate the expressions.? That allows logging > below threshold to be more or less a no-op. > > > > > -- > WBR, Anatoly. > > > > -- > WBR, Anatoly. > > > > -- > WBR, Anatoly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron.pressler at oracle.com Sun Mar 17 20:13:03 2024 From: ron.pressler at oracle.com (Ron Pressler) Date: Sun, 17 Mar 2024 20:13:03 +0000 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: <806AA995-FBA4-404F-8ED1-1A5A340681BE@oracle.com> > On 17 Mar 2024, at 17:39, Justin Spindler wrote: > > > > You can get the effect you want by using Suppler objects as embedded expressions. > > Sure, but that increases the amount of code that the developer needs to write pretty substantially, especially since there are no natural types for lambdas so the developer would have to assign that to a typed variable ahead of time or cast explicitly to a Supplier in the embedded expression. > FOO."This is a test: \{(Supplier) () -> calc(x, y, z)}" This isn?t how logging libraries should do it. A better approach from a code perspective could be: public static Supplier defer(Supplier s) { return s; } which would be used like so: logger.log(RAW.?? \{defer(() -> foo(x))} ??); But this may be an attempt (and possibly a failed one) at premature optimisation. AFAIK, C# doesn?t offer deferred evaluation, either, just deferred formatting of the message ? just like raw string templates. Finally, note that any kind of deferred evaluation requires some form of capture (and so possibly allocation in cases where the JIT doesn?t optimise it away). If you log inside a loop with `x` being the loop counter and used in a template argument, then `x` has to be captured for the log message to be correct. Furthermore, the cost of allocation with the JDK?s state-of-the-art collectors isn?t what it used be in the past. As always, platform optimisations must be guided by actual profiles of actual applications; a 1000x optimisation cannot improve application performance by more than 1% if the optimised operation is 1% of the profile to begin with. We strive to optimise what actually needs to be optimised, not what we speculate could benefit from optimisation in artificial scenarios. The need for something like `defer` is dependent on such evaluations being empirically found to be a common bottleneck in real-world applications. Maybe the need for more code is a problem that requires addressing, and maybe it?s not a problem at all. That we could imagine it to be a problem in hypothetical applications is not sufficient to justify optimising it. ? Ron From kan.izh at gmail.com Sun Mar 17 21:11:26 2024 From: kan.izh at gmail.com (Anatoly Kupriyanov) Date: Sun, 17 Mar 2024 21:11:26 +0000 Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: Re "list containing { hugeObject, expensiveObject }", please look at the real API of a real library: https://www.slf4j.org/api/org/slf4j/Logger.html#debug(java.lang.String,java.lang.Object) . All these one and two args overrides were aiming to overcome the " if(isDebugLogging)" boilerplate. I understand that StringTemplate was not aiming to solve all the possible problems, but obviously it is not good enough for logging frameworks. Meanwhile, the C# covered the issue, maybe not so nicely, by the string template interceptors. On Sun, 17 Mar 2024 at 19:45, Brian Goetz wrote: > It depends on how you measure "don't care anything". Even this, which is > pretty fast, still creates an object with a list containing { hugeObject, > expensiveObject }. This is pretty cheap but not necessarily free. (It may > be free if the debug() call gets inlined.) > > I think for 9x% of the cases, this is entirely cheap enough; for those > cases where it is not, the old fussy technique of > > if (isDebugLogging) > logger.debug(...) > > will recapture the rest. > > On 3/17/2024 3:41 PM, Anatoly Kupriyanov wrote: > > I guess it is not fair to think about it in terms of premature > optimization. I am trying to look at it from a logging-framework developer > point of view. As a library developer I should give the smallest overhead > possible. So library users could safely do > LOGGER.debug("Some thing={}, another={}", hugeObject, expensiveObject); > and don't care about anything at all as debug is usually disabled. So the > debug statement could be used even in a tight computation loop without > worries. In other words, logger devs could not assume anything about how > their framework would be used, but they will fight for the easiest to use > api with the fastest performance. > The idea is that the trace is usually disabled and an app could chew-up > gigs of data, but while debugging/investigating, the trace could be > temporarily enabled and expected to degrade performance. > > > On Sun, 17 Mar 2024 at 17:36, David Alayachew > wrote: > >> Thank you for the code example. That helps clarify your desired behaviour. >> >> Ok, if your intent is to have absolutely no garbage collection >> whatsoever, then yes, my solution does not guarantee that. How much the JIT >> will help or optimize, I cannot say. >> >> That said, a String is a "blessed type" in Java. It has a handful of >> optimizations that are specific to it and it alone. I am actually expecting >> that StringTemplates will receive at least a small taste of this sort of >> optimization. >> >> Furthermore, the cost of creating a StringTemplate is rarely, if ever, >> the expensive part of the operation. In fact, I am certain it is trivial, >> if not optimized away like you were asking. >> >> I won't presume to call this premature optimization -- perhaps your >> application needs require it. But if so, I would first wait and see (or >> better yet, calculate!) the performance metrics of these StringTemplates. >> If they fail to meet your performance needs, then communicate your >> performance needs to this mailing list, with a hard example (and ideally >> metrics!) in tow. >> >> On Sun, Mar 17, 2024 at 1:07?PM Anatoly Kupriyanov >> wrote: >> >>> Classically (how it was done a decade or so ago) the logger method would >>> look like: >>> void debug(String template, Object arg1, Object arg2) { >>> if(!enabled) return; >>> writeLog(interpolate(template, arg1.toString(), arg2.toString())); >>> } >>> >>> That allows that if enabled==false, then it will be zero-gc, >>> guaranteed. >>> It even used a lot of overrides with zero args, one arg, two args to >>> avoid ...-ellipsis array allocation. E.g. see >>> https://www.slf4j.org/api/org/slf4j/Logger.html >>> >>> There is a C# hacky approach the String Interpolation Handler ( >>> https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/interpolated-string-handler >>> ) to use struct (allocated on stack) and magical isEnabled flags to cover >>> logging effectively. >>> >>> I don't see how it would be possible to use the StringTemplate to >>> achieve the similar effectiveness, unless there are some good and reliable >>> optimizations in JIT to avoid allocations. >>> >>> On Sun, 17 Mar 2024 at 16:41, David Alayachew >>> wrote: >>> >>>> The cost would be no different than evaluating a StringTemplate lazily. >>>> Utlimately, log.debug(blah) is still a method call, and therefore, some >>>> object is going into the method. Whether we evaluate the StringTemplate >>>> lazily or the Supplier lazily is largely the same thing. >>>> >>>> Or am I misunderstanding you? >>>> >>>> On Sun, Mar 17, 2024 at 12:28?PM Anatoly Kupriyanov >>>> wrote: >>>> >>>>> But the "() ->" thing is still a runtime object to be allocated on the >>>>> heap... or is the JIT smart enough to always optimize it out? >>>>> >>>>> On Sun, 17 Mar 2024 at 16:17, David Alayachew < >>>>> davidalayachew at gmail.com> wrote: >>>>> >>>>>> I feel like that would be on the logging libraries to provide, not so >>>>>> much the language. >>>>>> >>>>>> Let's say that your problem (as I understand it) is like this. >>>>>> >>>>>> log.debug("metrics = >>>>>> \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}"); >>>>>> >>>>>> Sounds to me like the solution is this instead. >>>>>> >>>>>> log.debug(() -> "metrics = >>>>>> \{this.expensiveComputationThatNormallyWouldntRunUnlessYouActivateDebug()}") >>>>>> >>>>>> So, your logging library just needs to add a new overload for a >>>>>> Supplier, and then this problem is solved entirely outside >>>>>> of the language. Log4J and friends are pretty good about keeping up with >>>>>> new additions, so it should not take long. >>>>>> >>>>>> I know it's a little less convenient, but doing it this way helps >>>>>> keep out complexity from the feature, and only introduces it where >>>>>> necessary (and it's only necessary at use-site). >>>>>> >>>>>> Would this meet your needs? >>>>>> >>>>>> On Sun, Mar 17, 2024 at 11:15?AM Justin Spindler < >>>>>> justin.spindler at gmail.com> wrote: >>>>>> >>>>>>> I was toying around with the second preview of StringTemplates and I >>>>>>> had a question regarding their design. I was wondering if it had been >>>>>>> considered for the embedded expressions to be evaluated lazily? >>>>>>> >>>>>>> One of the first use cases that came to mind when I was exploring >>>>>>> StringTemplates is logging. That is an extremely common case where we want >>>>>>> to produce a form of interpolated value, and the current syntax generally >>>>>>> has the same concerns that a formatted string would, in that the inputs are >>>>>>> removed from where they are defined within the message format. However, if >>>>>>> the log message is below the log level threshold you generally don't want >>>>>>> to incur the cost of building the message, including evaluating those >>>>>>> embedded expressions. Log libraries typically offer a mechanism to defer >>>>>>> evaluation via Suppliers, but that feels like it would be a challenge to >>>>>>> use within StringTemplate. >>>>>>> >>>>>>> C# is an example of a language that offers this functionality via >>>>>>> FormattableString, which gives a method the ability to choose whether or >>>>>>> not to interpret the template or evaluate the expressions. That allows >>>>>>> logging below threshold to be more or less a no-op. >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> WBR, Anatoly. >>>>> >>>> >>> >>> -- >>> WBR, Anatoly. >>> >> > > -- > WBR, Anatoly. > > > -- WBR, Anatoly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron.pressler at oracle.com Sun Mar 17 22:15:29 2024 From: ron.pressler at oracle.com (Ron Pressler) Date: Sun, 17 Mar 2024 22:15:29 +0000 Subject: [External] : Re: StringTemplates deferred evaluation In-Reply-To: References: <806AA995-FBA4-404F-8ED1-1A5A340681BE@oracle.com> Message-ID: > On 17 Mar 2024, at 20:46, Justin Spindler wrote: > > C# does allow shortcircuiting interpolation through a custom interpolation handler. In that case none of the expressions are evaluated. This is a general purpose mechanism, but the primary use case was logging frameworks. Importantly, the code doesn't require the developer to do anything special, such as call a different overload or helper methods, it "just works" by default creating a pit of success. I believe we?re talking about different things. If I?m not mistaken, C#?s feature for conditional evaluation of template arguments is only available in the special case of string interpolation. Whether or not such a capability is sufficient useful, even for logging ? and if so, whether allowing library code to determine whether (possibly side-effecting) Java expressions are evaluated is a good idea and could be said to ?just work? ? is an open question. Java?s string templates, however, don?t focus on string interpolation at all, but on the embedding of ?code? such as SQL, JSON, or HTML, in a way that avoids code injection. This use case is more analogous to assigning a template to a FormattableString in C#, where ? again, if I?m not mistaken ? conditional evaluation is not needed. However important string interpolation may be, I think it is safe to say that string templates? focus is on a more important use-case ? certainly one that is much harder to do correctly given existing mechanisms. > > In the Spring services I maintain, the only real use case I'd have for string templates would be logging. As mentioned we've had several cases in the past where a developer accidentally included debug/trace level logging with a calculated value, like a stack trace, that caused observable performance regressions. We try to guard against common cases with PMD and education, but sometimes it still slips through. Different people may have different needs, but services do often have a need to generate SQL, JSON, or HTML, and the high prevalence and and damage of code-injection vulnerabilities makes it a high priority to allow libraries to offer APIs that are simple and pleasurable to use while at the same time being safe. If it *also* turns out that logging expressions that are costly to evaluate beyond the formatting of the message is a common-enough problem that the use of something like the `defer` merits attention, we could tackle that problem another day. In any event, the easy confusion that arises from designs like C#?s between interpolating strings and safely embedding code is something we?d like to avoid. ? Ron From ron.pressler at oracle.com Sun Mar 17 22:19:51 2024 From: ron.pressler at oracle.com (Ron Pressler) Date: Sun, 17 Mar 2024 22:19:51 +0000 Subject: [External] : Re: StringTemplates deferred evaluation In-Reply-To: References: <806AA995-FBA4-404F-8ED1-1A5A340681BE@oracle.com> Message-ID: <4DDE3810-4210-4AFA-96AF-BBEB70906066@oracle.com> > On 17 Mar 2024, at 22:15, Ron Pressler wrote: > > > >> On 17 Mar 2024, at 20:46, Justin Spindler wrote: >> >> C# does allow shortcircuiting interpolation through a custom interpolation handler. In that case none of the expressions are evaluated. This is a general purpose mechanism, but the primary use case was logging frameworks. Importantly, the code doesn't require the developer to do anything special, such as call a different overload or helper methods, it "just works" by default creating a pit of success. > > I believe we?re talking about different things. If I?m not mistaken, C#?s feature for conditional evaluation of template arguments is only available in the special case of string interpolation. Whether or not such a capability is sufficient useful, even for logging ? and if so, whether allowing library code to determine whether (possibly side-effecting) Java expressions are evaluated is a good idea and could be said to ?just work? ? is an open question. > > Java?s string templates, however, don?t focus on string interpolation at all, but on the embedding of ?code? such as SQL, JSON, or HTML, in a way that avoids code injection. This use case is more analogous to assigning a template to a FormattableString in C#, where ? again, if I?m not mistaken ? conditional evaluation is not needed. > > However important string interpolation may be, I think it is safe to say that string templates? focus is on a more important use-case ? certainly one that is much harder to do correctly given existing mechanisms. > >> >> In the Spring services I maintain, the only real use case I'd have for string templates would be logging. As mentioned we've had several cases in the past where a developer accidentally included debug/trace level logging with a calculated value, like a stack trace, that caused observable performance regressions. We try to guard against common cases with PMD and education, but sometimes it still slips through. > > Different people may have different needs, but services do often have a need to generate SQL, JSON, or HTML, and the high prevalence and and damage of code-injection vulnerabilities makes it a high priority to allow libraries to offer APIs that are simple and pleasurable to use while at the same time being safe. If it *also* turns out that logging expressions that are costly to evaluate beyond the formatting of the message is a common-enough problem that the use of something like the `defer` merits attention, we could tackle that problem another day. > > In any event, the easy confusion that arises from designs like C#?s between interpolating strings and safely embedding code is something we?d like to avoid. > > ? Ron Sorry, where I wrote "conditional evaluation is not needed? I meant to write "not supported." From forax at univ-mlv.fr Mon Mar 18 08:52:42 2024 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 18 Mar 2024 09:52:42 +0100 (CET) Subject: StringTemplates deferred evaluation In-Reply-To: References: Message-ID: <794803864.32351935.1710751962410.JavaMail.zimbra@univ-eiffel.fr> ----- Original Message ----- > From: "Jim Laskey" > To: "Justin Spindler" > Cc: "amber-dev" > Sent: Sunday, March 17, 2024 6:28:07 PM > Subject: Re: StringTemplates deferred evaluation > There was a discussion about this on this list a while back. The issue is that > deferred evaluation could provide a gapping vulnerability. > > You can get the effect you want by using Suppler objects as embedded > expressions. I posted a logging example using a timestamp Supplier. The Suppler > was evaluated by the processor. > > In the new world we are working on, the same thing can done. The plan is to > provide a mapValues method as in st.mapValues(v -> v instanceof Supplier S ? > S.get() : v). Hello Jim, I think i would prefer the method interpolate to take a Function instead of having a method mapValues, to force users to think about escaping at least the values even if users can still use st.interpolate(String::valueOf). > > ? Jim > > ? regards, R?mi > >> On Mar 17, 2024, at 12:14?PM, Justin Spindler wrote: >> >> ? >> I was toying around with the second preview of StringTemplates and I had a >> question regarding their design. I was wondering if it had been considered for >> the embedded expressions to be evaluated lazily? >> >> One of the first use cases that came to mind when I was exploring >> StringTemplates is logging. That is an extremely common case where we want to >> produce a form of interpolated value, and the current syntax generally has the >> same concerns that a formatted string would, in that the inputs are removed >> from where they are defined within the message format. However, if the log >> message is below the log level threshold you generally don't want to incur the >> cost of building the message, including evaluating those embedded expressions. >> Log libraries typically offer a mechanism to defer evaluation via Suppliers, >> but that feels like it would be a challenge to use within StringTemplate. >> >> C# is an example of a language that offers this functionality via >> FormattableString, which gives a method the ability to choose whether or not to >> interpret the template or evaluate the expressions. That allows logging below >> threshold to be more or less a no-op. >> From james.laskey at oracle.com Mon Mar 18 10:46:48 2024 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 18 Mar 2024 10:46:48 +0000 Subject: [External] : Re: StringTemplates deferred evaluation In-Reply-To: <794803864.32351935.1710751962410.JavaMail.zimbra@univ-eiffel.fr> References: <794803864.32351935.1710751962410.JavaMail.zimbra@univ-eiffel.fr> Message-ID: <2A1015E9-DB54-4B25-A349-2BFB1D2C88A8@oracle.com> Good point. I?ll play with that. ? > On Mar 18, 2024, at 5:52?AM, Remi Forax wrote: > > ?----- Original Message ----- >> From: "Jim Laskey" >> To: "Justin Spindler" >> Cc: "amber-dev" >> Sent: Sunday, March 17, 2024 6:28:07 PM >> Subject: Re: StringTemplates deferred evaluation > >> There was a discussion about this on this list a while back. The issue is that >> deferred evaluation could provide a gapping vulnerability. >> >> You can get the effect you want by using Suppler objects as embedded >> expressions. I posted a logging example using a timestamp Supplier. The Suppler >> was evaluated by the processor. >> >> In the new world we are working on, the same thing can done. The plan is to >> provide a mapValues method as in st.mapValues(v -> v instanceof Supplier S ? >> S.get() : v). > > Hello Jim, > I think i would prefer the method interpolate to take a Function instead of having a method mapValues, to force users to think about escaping at least the values even if users can still use st.interpolate(String::valueOf). > >> >> ? Jim >> >> ? > > regards, > R?mi > >> >>>> On Mar 17, 2024, at 12:14?PM, Justin Spindler wrote: >>> >>> ? >>> I was toying around with the second preview of StringTemplates and I had a >>> question regarding their design. I was wondering if it had been considered for >>> the embedded expressions to be evaluated lazily? >>> >>> One of the first use cases that came to mind when I was exploring >>> StringTemplates is logging. That is an extremely common case where we want to >>> produce a form of interpolated value, and the current syntax generally has the >>> same concerns that a formatted string would, in that the inputs are removed >>> from where they are defined within the message format. However, if the log >>> message is below the log level threshold you generally don't want to incur the >>> cost of building the message, including evaluating those embedded expressions. >>> Log libraries typically offer a mechanism to defer evaluation via Suppliers, >>> but that feels like it would be a challenge to use within StringTemplate. >>> >>> C# is an example of a language that offers this functionality via >>> FormattableString, which gives a method the ability to choose whether or not to >>> interpret the template or evaluate the expressions. That allows logging below >>> threshold to be more or less a no-op. >>> From ron.pressler at oracle.com Mon Mar 18 11:42:56 2024 From: ron.pressler at oracle.com (Ron Pressler) Date: Mon, 18 Mar 2024 11:42:56 +0000 Subject: [External] : Re: StringTemplates deferred evaluation In-Reply-To: References: <806AA995-FBA4-404F-8ED1-1A5A340681BE@oracle.com> Message-ID: <2FB6FBAB-A558-400C-B395-BC0336F0589F@oracle.com> > On 17 Mar 2024, at 23:16, Justin Spindler wrote: > > C#s approach handles both cases. It's not just interpolation, the formatting of the string is handled by the consumer of the FormattedString or by the custom interpolation handler. Both work with the raw string fragments and embedded values and can safely construct the string for the given use case. The problem with that design is that a template could be interpolated *implicitly* when assigned/passed to a string (and worse: interpolation is *preferred* when the template is assigned to a var). Given how dangerous interpolation is, this is something we?d like to avoid; any utility that interpolation has is easily offset by its considerable risk. So any interpolation feature ? be it in the core library or perhaps in some future language feature ? should be explicit. > > C# did add custom interpolation handlers in a later release, based of the feedback from the logging use case. If Java wants to take a few cycles to see how the feedback shakes out to see whether it's worth considering, that's fine. The most I might ask is that the current design doesn't preclude such enhancements, and from what I can see it wouldn't. Right, although a feature that allows library code to control the evaluation of Java expressions would need to be sufficiently justified as solving a significant and common problem. That there are situations where deferring evaluation of expressions would be useful does not mean that it?s a serious-enough problem to merit a solution in the language, let alone one with subtle and non-obvious implications such as library-controlled evaluation. Not every problem deserves a language solution. ? Ron From pholder at gmail.com Tue Mar 19 19:56:15 2024 From: pholder at gmail.com (P Holder) Date: Tue, 19 Mar 2024 15:56:15 -0400 Subject: StringTemplate evolution Message-ID: Now that it appears that StringTemplates are evolving to a special introducer token, I hope it's also being recognized that they can then also change to support a syntax that is more friendly and common with other languages. In particular, there would appear to be no need any more to require the backslash to introduce the values. The backslash can now be reversed and offered to escape the open brace when it's desired as a literal. To get the result: The value of foo is 42. Go from: STR."The value of \{name} is \{value}."; to $"The value of {name} is {value}."; and to get the result: The value of {foo} is {42}. Go from: STR."The value of {\{name}} is {\{value}}"; to: $"The value of \{{name}} is \{{value}}"; I'd also propose that you could use a keyword token instead of the $ without causing too much grief. This would have the benefit of looking like a name, and allowing for future evolution in a direction where you might want something additional and don't want to find an additional punctuation character. So, where ST stands for StringTemplate a string template could look like: ST."The value of {name} is {value}."; -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Tue Mar 19 20:09:04 2024 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 19 Mar 2024 16:09:04 -0400 Subject: StringTemplate evolution In-Reply-To: References: Message-ID: All fair comments, but as a gentle reminder: amber-dev is not the parallel-but-public design list to amber-spec-experts. On 3/19/2024 3:56 PM, P Holder wrote: > Now that it appears that StringTemplates are evolving to a special > introducer token, I hope it's also being recognized that they can then > also change to support a syntax that is more friendly and common with > other languages.? In particular, there would appear to be no need any > more to require the backslash to introduce the values.? The backslash > can now be reversed and offered to escape the open brace when it's > desired as a literal. > > To get the result: The value of foo is 42. > Go from: > ? STR."The value of \{name} is \{value}."; > to > ? $"The value of {name} is {value}."; > > and to get the result: The value of {foo} is {42}. > Go from: > ? STR."The value of {\{name}} is {\{value}}"; > to: > ? $"The value of \{{name}} is \{{value}}"; > > I'd also propose that you could use a keyword token instead of the $ > without causing too much grief.? This would have the benefit of > looking like a name, and allowing for future evolution in a direction > where you might want something additional and don't want to find an > additional punctuation character. > > So, where ST stands for StringTemplate a string template could look like: > ? ST."The value of {name} is {value}."; -------------- next part -------------- An HTML attachment was scrubbed... URL: From shahan at google.com Wed Mar 27 23:20:44 2024 From: shahan at google.com (Shahan Yang) Date: Wed, 27 Mar 2024 19:20:44 -0400 Subject: JEP 8323072 and Serialization Message-ID: My apologies if this is the wrong place for this kind of question. I maintain an Unsafe-heavy serialization library and I am weighing options in light of Unsafe memory access deprecation. It appears that the recommended replacement is to use VarHandles. This seems reasonable at first glance, but it looks like VarHandles do not support writes to final fields. The problem is that there's a large codebase with a lot of application logic objects to be serialized. The presence of final fields on those objects aids in their readability. Currently, those fields can just be set using Unsafe.put* methods during deserialization, but that isn't possible with VarHandles. I don't think it's reasonable to ask developers to make all of the application object fields non-final for deserialization. It would also be an infeasibly large amount of toil to add custom code to serialize and deserialize all those objects, relative to what is possible with reflection. I can think of a number of alternatives, but so far I haven't come up with anything that avoids harming the readability of the code, increasing toil, making serialization more error prone or slower (just using regular Field reflection instead of Unsafe to set final fields?). I'm less worried about records because the tight contract between their constructors and their fields makes a code generation-based solution quite clean. Do you have any suggestions? Am I overlooking something that other serialization libraries are doing here? Thanks, Shahan -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.buckley at oracle.com Wed Mar 27 23:56:19 2024 From: alex.buckley at oracle.com (Alex Buckley) Date: Wed, 27 Mar 2024 16:56:19 -0700 Subject: JEP 8323072 and Serialization In-Reply-To: References: Message-ID: <37c6e557-37f6-43ed-91da-1bb72011f2e5@oracle.com> Hi Shahan, I thought we'd put the following in 8323072 but it turns out to be in https://openjdk.org/jeps/8305968#Embracing-integrity-by-default: > Some classes already take responsibility for their serialization and > deserialization by implementing the java.io.Serializable interface; > libraries can take advantage by invoking writeObject and readObject > on the classes via sun.reflect.ReflectionFactory, which is supported > for this purpose. In the long term, we expect the Java Platform to > offer better serialization. Is sun.reflect.ReflectionFactory suitable for your objects? Alex On 3/27/2024 4:20 PM, Shahan Yang wrote: > My apologies if this is the wrong place for this kind of question. I > maintain an Unsafe-heavy serialization library and I am weighing options > in light of Unsafe memory access deprecation. > > It appears that the recommended replacement is to use VarHandles. This > seems reasonable at first glance, but it looks like VarHandles do not > support writes to final fields. > > The problem is that there's a large codebase with a lot of application > logic objects to be serialized. The presence of final fields on those > objects aids in their readability. Currently, those fields can just be > set using Unsafe.put* methods during deserialization, but that isn't > possible with VarHandles. I don't think it's reasonable to ask > developers to make all of the application object fields non-final for > deserialization. It would also be an infeasibly large amount of toil to > add custom code to serialize and deserialize all those objects, relative > to what is possible with reflection. > > I can think of a number of alternatives, but so far I haven't come up > with anything that avoids?harming the readability of the code, > increasing toil, making serialization more error prone or slower (just > using regular Field reflection instead of Unsafe to set?final fields?). > I'm less worried about records because the tight contract between their > constructors and their fields makes a code generation-based solution > quite clean. > > Do you have any suggestions? Am I overlooking something that other > serialization libraries are doing here? > > > Thanks, > Shahan From shahan at google.com Thu Mar 28 01:04:29 2024 From: shahan at google.com (Shahan Yang) Date: Wed, 27 Mar 2024 21:04:29 -0400 Subject: JEP 8323072 and Serialization Message-ID: > Is sun.reflect.ReflectionFactory suitable for your objects? Hi Alex, thanks for the response. I don't think sun.reflect.ReflectionFactory suitable will be suitable. It seems that it would require per-class serialization code, which is going a huge amount of toil compared to a reflection-based approach. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bitterfoxc at gmail.com Thu Mar 28 04:47:32 2024 From: bitterfoxc at gmail.com (ShinyaYoshida) Date: Thu, 28 Mar 2024 13:47:32 +0900 Subject: Feedback for StringTemplate; indify StringTemplate Processor invocation to cache compiled Processor at runtime Message-ID: Hi, (Please tell me if there's another proper ML than amber-dev) I recently played around with StringTemplate. I developed a StringTemplate Processor for JSON recently: https://github.com/bitterfox/json-string-template This parses JSON format strings in fragments with placeholders with referring values array element. In the first naive implementation, it tokenizes fragments and parses the tokens creating Json Objects taking elements from values, and putting the value as the Json Object key or value. So this is the one-pass implementation, but it parses fragments every time the JSON String Template processing is called. https://github.com/bitterfox/json-string-template/blob/main/json-string-template-core/src/main/java/io/github/bitterfox/json/string/template/core/JsonParserV1.java In the second implementation, it parses fragments and creates AST for JSON. After that, it reads the AST again and creates a Lambda expression for `(Object[] value) -> JSON` that puts an element of values to a part of JsonObject. https://github.com/bitterfox/json-string-template/blob/main/json-string-template-core/src/main/java/io/github/bitterfox/json/string/template/core/JsonCompiler.java Thanks to returning (Object[] value) -> JSON and immutability of fragments of the string template expression (as long as we use through the string template expression, it will be the same fragments for the same invocation), We can now cache the lambda expression and skip parsing fragments to JSON AST. This provides us a space for optimization of the performance of JSON String Template by caching, however, StringTemplate.Processor is called for the same receiver usually because we usually define StringTemplate.Processor instance as static final field and use string template expression through it. So we need to implement a caching layer by ourselves. https://github.com/bitterfox/json-string-template/blob/main/json-string-template-core/src/main/java/io/github/bitterfox/json/string/template/core/JsonStringTemplateProcessorV2CachedImpl.java The downside of such caching implementation is - The caching layer could be the bottleneck and cause another performance issue - We need to depend on the implementation detail of Javac, and JDK for better performance of caching. For example, I assumed runtime always creates the same fragments reference - https://github.com/bitterfox/json-string-template/blob/b53898da2d2c70aeee9a36cdffc34ba31460ba28/json-string-template-core/src/main/java/io/github/bitterfox/json/string/template/core/JsonStringTemplateProcessorV2CachedImpl.java#L63 - This might not work on someday Now I'm working on a third implementation with Java22 using ClassFileAPI. Using ClassFileAPI, we can generate a Java class after parsing fragments that is ``` static JsonObject createJson(Object[] values) { return Json.createObjectBuilder().add("name", Json.createValue(value[0].toString())); } ``` for JSON"{'name': \{name}}". Once it's compiled to the Java class and loaded, calling it will have almost zero overhead even though we run it through string template expression. However, as the same reason for the 2nd implementation, there's no caching mechanism in StringTemplate, still, we need to implement the caching layer as well as the 2nd approach with the same problems. Suppose we can couple the compiled class (or (values: Object[]) -> T) to the invocation of string template processor at string template expression. In that case, we can solve this issue and provide the best performance for every StringTemplate Processor in the Java world. Now we already have a tool for it in JVM, invokedynamic. What do you think about using invokedynamic not only for creating StringTemplate objects but also calling StringTemplate Processor invocation and providing a caching layer at Java spec? I made a bytecode level PoC for StringTemplate Processor invocation using invokedynamic with caching for each StringTemplate expression. https://github.com/bitterfox/indy-string-template-processing/ In the PoC, I changed the interface of StringTemplate Processor. I added StringTemplateProcessorFactory: (String[] fragments) -> StringTemplateProcessor. StringTemplateProcessor is (Object[] values) -> Object (i.e. T). And it caches StringTemplateProcessor. StringTemplateProcessorFactory has a method to indicate to cache the processor or create the processor every time to runtime. StringTemplateRuntime runs the processing for StringTemplateProcessorFactory and StringTemplate(fragments and values). If cache is required, it creates a processor only once. StringTemplateRuntime is used for the MethodHandle coupled to the invokedynamic. StringTemplateBSM#createStringTemplateRuntimeCallSite is a BSM for the invokedynamic. If we compile doStringTemplate() { Sting name = "duke"; System.out.println(MY_STP."Hello \{name}"); } It will generate classfile like public static void doStringTemplate(); descriptor: ()V flags: (0x0009) ACC_PUBLIC, ACC_STATIC Code: stack=6, locals=2, args_size=0 0: getstatic #10 // Field StringTemplateSTRDebug.STR:LStringTemplateProcessorFactory; 3: iconst_2 4: anewarray #12 // class java/lang/String 7: dup 8: iconst_0 9: ldc #14 // String Hello 11: aastore 12: dup 13: iconst_1 14: ldc #16 // String 16: aastore 17: iconst_1 18: anewarray #18 // class java/lang/Object 21: dup 22: iconst_0 23: ldc #20 // String duke 25: aastore 26: invokedynamic #31, 0 // InvokeDynamic #0:process:(LStringTemplateProcessorFactory;[Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/Object; 31: astore_1 32: getstatic #37 // Field java/lang/System.out:Ljava/io/PrintStream; 35: aload_1 36: invokevirtual #43 // Method java/io/PrintStream.println:(Ljava/lang/Object;)V 39: return BootstrapMethods: 0: #27 REF_invokeStatic StringTemplateBSM.createStringTemplateRuntimeCallSite:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite; Method arguments: This creates string template processor only once for this method ``` // call doStringTemplate Create new processor, some processor may parse fragments, so slow Process string template Hello duke // call doStringTemplate Process string template Hello duke // call doStringTemplate Process string template Hello duke ``` This kind of mechanism will be useful for most of string template processors like - FMT: This parses formatter string like %02d - SQL: Same overhead with queryBuilder.select("*").from(...).where(...) is preferred - LocalizationProcessor: should avoid loading resource bundle every time - ... Let me know your thoughts about this approach for StringTemplate Processor invocation Best regards, Shinya Yoshida (@bitter_fox, @shinyafox) -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Thu Mar 28 07:53:54 2024 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 28 Mar 2024 08:53:54 +0100 (CET) Subject: JEP 8323072 and Serialization In-Reply-To: References: Message-ID: <1507414596.41135404.1711612434035.JavaMail.zimbra@univ-eiffel.fr> > From: "Shahan Yang" > To: "amber-dev" > Sent: Thursday, March 28, 2024 12:20:44 AM > Subject: JEP 8323072 and Serialization Hello, > My apologies if this is the wrong place for this kind of question. I maintain an > Unsafe-heavy serialization library and I am weighing options in light of Unsafe > memory access deprecation. > It appears that the recommended replacement is to use VarHandles. This seems > reasonable at first glance, but it looks like VarHandles do not support writes > to final fields. > The problem is that there's a large codebase with a lot of application logic > objects to be serialized. The presence of final fields on those objects aids in > their readability. Currently, those fields can just be set using Unsafe.put* > methods during deserialization, but that isn't possible with VarHandles. I > don't think it's reasonable to ask developers to make all of the application > object fields non-final for deserialization. It would also be an infeasibly > large amount of toil to add custom code to serialize and deserialize all those > objects, relative to what is possible with reflection. > I can think of a number of alternatives, but so far I haven't come up with > anything that avoids harming the readability of the code, increasing toil, > making serialization more error prone or slower (just using regular Field > reflection instead of Unsafe to set final fields?). I'm less worried about > records because the tight contract between their constructors and their fields > makes a code generation-based solution quite clean. > Do you have any suggestions? Am I overlooking something that other serialization > libraries are doing here? You ask for suggestions, my suggestion is that your serialization framework should ask users to provide the equivalent of a record canonical constructor, i.e. a constructor with the same visibility as the class that is able to initialize the class with either an annotation for each parameter that contains the corresponding name of the field or you can ask users to compile with "-parameters" then at runtime like with records, you just call that constructor. In my opinion, that's the simpler future-proof solution. I think Brian as alreay proposes something quite similar for the Java serialization. Why future-proof ? As part of Valhalla, we are introducing a new kind of fields, "really really final" fields that must be initialized *before* calling the super constructor and that can not be updated after the call to the super constructor. So even if we do not disable Unsafe.putField for those fields, the JIT is allowed to not use the result of an Unsafe.putField and use a previously seen value of the field. Otherwise, you can set the value of a final field by getting the field, calling setAccessible on it, then uses Lookup.unreflectSetter(field) on that field. You will get a method handle able to set a final field. But as i said above, it will be a short victory. > Thanks, > Shahan regards, R?mi -------------- next part -------------- An HTML attachment was scrubbed... URL: From shahan at google.com Thu Mar 28 13:07:10 2024 From: shahan at google.com (Shahan Yang) Date: Thu, 28 Mar 2024 09:07:10 -0400 Subject: JEP 8323072 and Serialization In-Reply-To: <1507414596.41135404.1711612434035.JavaMail.zimbra@univ-eiffel.fr> References: <1507414596.41135404.1711612434035.JavaMail.zimbra@univ-eiffel.fr> Message-ID: On Thu, Mar 28, 2024, 3:53?AM Remi Forax wrote: > > > ------------------------------ > > *From: *"Shahan Yang" > *To: *"amber-dev" > *Sent: *Thursday, March 28, 2024 12:20:44 AM > *Subject: *JEP 8323072 and Serialization > > > Hello, > > > My apologies if this is the wrong place for this kind of question. I > maintain an Unsafe-heavy serialization library and I am weighing options in > light of Unsafe memory access deprecation. > > It appears that the recommended replacement is to use VarHandles. This > seems reasonable at first glance, but it looks like VarHandles do not > support writes to final fields. > > The problem is that there's a large codebase with a lot of application > logic objects to be serialized. The presence of final fields on those > objects aids in their readability. Currently, those fields can just be set > using Unsafe.put* methods during deserialization, but that isn't possible > with VarHandles. I don't think it's reasonable to ask developers to make > all of the application object fields non-final for deserialization. It > would also be an infeasibly large amount of toil to add custom code to > serialize and deserialize all those objects, relative to what is possible > with reflection. > > I can think of a number of alternatives, but so far I haven't come up with > anything that avoids harming the readability of the code, increasing toil, > making serialization more error prone or slower (just using regular Field > reflection instead of Unsafe to set final fields?). I'm less worried about > records because the tight contract between their constructors and their > fields makes a code generation-based solution quite clean. > > Do you have any suggestions? Am I overlooking something that other > serialization libraries are doing here? > > > You ask for suggestions, my suggestion is that your serialization > framework should ask users to provide the equivalent of a record canonical > constructor, i.e. a constructor with the same visibility as the class that > is able to initialize the class with either an annotation for each > parameter that contains the corresponding name of the field or you can ask > users to compile with "-parameters" then at runtime like with records, you > just call that constructor. In my opinion, that's the simpler future-proof > solution. > I think Brian as alreay proposes something quite similar for the Java > serialization. > Very well, thank you. Ironically, the serialization code started with that approach and quite a few classes are still serialized that way when reflection doesn't work. It's a bit more error prone and more toil which is why the reflection-based approach came into being. I guess we'll move back to constructor-based. > Why future-proof ? > As part of Valhalla, we are introducing a new kind of fields, "really > really final" fields that must be initialized *before* calling the super > constructor and that can not be updated after the call to the super > constructor. So even if we do not disable Unsafe.putField for those fields, > the JIT is allowed to not use the result of an Unsafe.putField and use a > previously seen value of the field. > Thank you for explaining this. Can't wait to see Valhalla. > > > Otherwise, you can set the value of a final field by getting the field, > calling setAccessible on it, then uses Lookup.unreflectSetter(field) on > that field. You will get a method handle able to set a final field. > But as i said above, it will be a short victory. > > > > Thanks, > Shahan > > > regards, > R?mi > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Thu Mar 28 16:46:11 2024 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 28 Mar 2024 17:46:11 +0100 (CET) Subject: Feedback for StringTemplate; indify StringTemplate Processor invocation to cache compiled Processor at runtime In-Reply-To: References: Message-ID: <69109941.41790795.1711644371704.JavaMail.zimbra@univ-eiffel.fr> > From: "ShinyaYoshida" > To: "amber-dev" > Sent: Thursday, March 28, 2024 5:47:32 AM > Subject: Feedback for StringTemplate; indify StringTemplate Processor invocation > to cache compiled Processor at runtime > Hi, > (Please tell me if there's another proper ML than amber-dev) > What do you think about using invokedynamic not only for creating StringTemplate objects but also calling StringTemplate Processor invocation and providing a caching layer at Java spec? I agree, i've sent a proposal to the expert list to goes in that direction. regards, R?mi -------------- next part -------------- An HTML attachment was scrubbed... URL: