crazy idea: weaken the effectively final restriction

forax at univ-mlv.fr forax at univ-mlv.fr
Wed Oct 6 17:22:07 UTC 2021


----- Original Message -----
> From: "mark yagnatinsky" <mark.yagnatinsky at barclays.com>
> To: "Remi Forax" <forax at univ-mlv.fr>, "Brian Goetz" <brian.goetz at oracle.com>
> Cc: "raffaello giulietti" <raffaello.giulietti at gmail.com>, "jdk-dev" <jdk-dev at openjdk.java.net>
> Sent: Mercredi 6 Octobre 2021 19:05:57
> Subject: RE: crazy idea: weaken the effectively final restriction

> Re: guarded switch: I confess I have not been watching the latest developments
> in pattern matching.  (Enhanced instanceof is great though!)
> If it turns out that effectively final makes sense there too, then I guess
> that's another strike against my idea; might as well re-use the same concept
> with lambdas too.
> 
> Re: reading code too fast could result in overlooking an assignment: this has
> recently become a "solved problem" for me, because IntelliJ by default
> underlines local variables that are NOT effectively final.

IDEs are marvelous but if an IDE is required to understand the semantics of the languages then something is very wrong.

> 
> I do have one quibble with what you said though: my proposed semantics do NOT
> behave differently in the two examples you listed.
> In particular, neither example compiles, since there is an assignment to the
> variable AFTER it was already captured.
> Of course, your misunderstanding is also a strike against my idea: if I can't
> even explain it to experts, what hope is there that mere mortals can keep this
> straight?

yes, the two lines should be swapped, for my defense, i typed that message at a swimming pool cheering for my daughter at the same time :)

Rémi

> 
> -----Original Message-----
> From: Remi Forax <forax at univ-mlv.fr>
> Sent: Wednesday, October 6, 2021 12:24 PM
> To: Brian Goetz <brian.goetz at oracle.com>
> Cc: raffaello giulietti <raffaello.giulietti at gmail.com>; Yagnatinsky, Mark :
> Markets Pre Trade <mark.yagnatinsky at barclays.com>; jdk-dev
> <jdk-dev at openjdk.java.net>
> Subject: Re: crazy idea: weaken the effectively final restriction
> 
> 
> CAUTION: This email originated from outside our organisation - forax at univ-mlv.fr
> Do not click on links, open attachments, or respond unless you recognize the
> sender and can validate the content is safe.
> Hi Mark,
> to complete Brian answer,
> 
> - Reassigning a local variable without being inside a loop is usually seen as a
> code smell, because when you read the code a little to fast you can miss the
> second assignment thus believe that the code behave differently than what is
> written.
> 
> - The semantics you propose behave differently if the captured variable is
> available outside the loop or not, so
>  Runnable r = () -> println(s);
>  s = normalize(s);
> 
> will behave differently from
>  var s =
>  for(...) {
>    Runnable r = () -> println(s);
>    s = normalize(s);
>  }
>  ... s ...
> 
> If you see the local variable in SSA form, the semantics depend if there is a
> phi or not.
> 
> - We also did not want to have a semantics depending on what you call "the point
> of capture", i.e. we wanted to have a semantics that does not explicitly
> specify when the lambda is created, this allows implementations/translation
> strategies to create reusable lambda instance either lazily or eagerly. This is
> something important because we can support both invokedynamic which does the
> allocation as late as possible and GraalVM native image / AppCDS that may store
> the lambda instance directly in the image.
> 
> You may think that we are talking about capture here so a lambda instance can
> not be reused but a captured value can be proved to be a constant by a
> static/runtime analysis.
> 
> - Effectively final is a semantics that is used not only for lambdas but also
> for anonymous classes and recently for guarded patterns, for a pattern the "the
> point of capture" is not clearly defined. Is it when you want to evaluate the
> switch ? or when you evaluate the guard of the pattern ?
> 
> 
> So yes, we can try to tweak that rule but as Brian said, it just doesn’t seem
> worth the complexity.
> 
> regards,
> Rémi
> 
> 
> ----- Original Message -----
>> From: "Brian Goetz" <brian.goetz at oracle.com>
>> To: "raffaello giulietti" <raffaello.giulietti at gmail.com>
>> Cc: "mark yagnatinsky" <mark.yagnatinsky at barclays.com>, "jdk-dev"
>> <jdk-dev at openjdk.java.net>
>> Sent: Mercredi 6 Octobre 2021 17:12:18
>> Subject: Re: crazy idea: weaken the effectively final restriction
> 
>> A good way to think about this is to zoom out and realize that there
>> is a spectrum, with trade-offs.  At one end of the spectrum is “no
>> capture at all”, which is simple and easy to reason about, but …
>> unsatisfying.  At the other end is that we close over variables rather
>> than values, which some languages do, which is defensible but leads to
>> surprising bugs.  In the middle are options like “capture final
>> variables only”, “capture effectively final variables only”, etc.  Your option
>> is somewhat to the “right” of “effectively final.”
>> 
>> What you’re saying is that we could capture things that are not
>> effectively final, if we can prove that all writes happen before the
>> capture point in the execution order.  Java does flow analysis to
>> determine definite assignment, so such an enhanced flow analysis is imaginable.
>> 
>> The way to think about this is whether the incremental expressiveness
>> warrants the incremental complexity.  Yes, there are a small number of
>> cases where the more refined analysis would make a difference, but it’s really
>> not that many.
>> On the other side, “effectively final” is an easier concept to explain
>> to developers; there is definitely incremental complexity in the user model
>> here.
>> Is the trade worth it?  My sense: meh.  I don’t see a huge degree of
>> leverage here.
>> 
>> A similar example where we chose a simpler rule than necessary was in `var`.
>> The intent is that local variable type inference is for
>> implementation, not for API.  So you can use it for locals, but not
>> for, say, method returns, because that’s API, not just implementation.
>> Invariably, some slightly-too-clever person asks “but, then why can’t
>> I use it for the return of a *private* method?  That’s not API.  AHA!  Gotcha!”
>> 
>> The answer is of course simple; we could make the boundary of the
>> feature arbitrarily complex, but if the incremental complexity makes
>> it harder for people to reason about when they can use var and when
>> not, then we’re not necessarily helping by making it incrementally
>> more expressive.  A simple rule that is easy to reason about is often
>> better than a fractally complex one that is slightly more powerful.
>> 
>> So, to summarize, the idea isn’t crazy, it just doesn’t seem worth it in the
>> balance between complexity and expressiveness.   I’d rather spend that
>> complexity budget on something with more leverage.
>> 
>> Cheers,
>> -Brian
>> 
>>> On Oct 6, 2021, at 8:51 AM, Raffaello Giulietti
>>> <raffaello.giulietti at gmail.com>
>>> wrote:
>>> 
>>> Hi,
>>> 
>>> what if the variable is reassigned after the capture?
>>> 
>>> Runnable f(String s) {
>>>    Runnable r = () -> println(s);
>>>    s = normalize(s); // perfectly useless assignment
>>>    return r;
>>> }
>>> 
>>> It would not be "effectively final from point of capture", so would
>>> this lead to a compilation error?
>>> 
>>> 
>>> Greetings
>>> Raffaello
>>> 
>>> 
>>> 
>>> On 2021-10-05 23:55, mark.yagnatinsky at barclays.com wrote:
>>>> I'm not sure if this is the right list for this; if not I hope
>>>> someone can redirect me.
>>>> Java requires that local variables that are captured by a lambda
>>>> must be effectively final.
>>>> This restriction has the benefit that no one needs to worry about
>>>> annoying questions such as "what are the semantics of a data race on a local
>>>> variable"
>>>> and other such horrors.
>>>> But this benefit can still be obtained by a weaker restriction.  For
>>>> instance, consider this code, which currently does not compile:
>>>> Runnable f(String s) {
>>>>                 s = normalize(s);
>>>>                 return () -> println(s); // no can do: s is not
>>>> effectively final } However, this seems a bit silly because although
>>>> s is not final throughout the entire method, it's final where it
>>>> actually matters.
>>>> Namely, it is final from the point of capture to the end of the
>>>> method, and that's the only condition we need to avoid those annoying questions.
>>>> What do we gain by allowing the code above?  Well, if we don't allow
>>>> it, how can the code above be fixed?  We would have to introduce a new local
>>>> variable.
>>>> That would actually be fine, except for another rule Java has, also
>>>> for a good
>>>> reason: one local variable name can NOT "shadow" another: all local
>>>> variables must have distinct names.
>>>> That is, we can do something like this: String s = "hello"; String s
>>>> = s.trim(); So we need to come up with a new name, and this has two costs:
>>>> First, coming up with good names is hard, and coming up with two
>>>> good names for what is basically the same thing is harder.
>>>> Second, suppose that when we first wrote the method, the variable
>>>> really was effectively final, and we needed to change it many
>>>> months/years/decades later.
>>>> We now need to update the appropriate usages to the new name.  This
>>>> tends to clutter line-based diffs, creating more work for reviewers
>>>> during pull requests, and also for code archeologists.
>>>> Third (did I say two?): we must spend mental bandwidth deciding how
>>>> to accomplish renaming things.  In the example above we have at least two
>>>> options:
>>>> Runnable f(String t) {// option 1: rename original
>>>>                 String s = normalize(t);
>>>>                 return () -> println(s); } Runnable f(String s) {//
>>>> option 2: keep original
>>>>                 String t = normalize(s);
>>>>                 return () -> println(t); } Deciding which option is
>>>> better might actually involve non-trivial tradeoffs.
>>>> Perhaps option 1 leads to a smaller diff, but option 2 leads to a
>>>> better parameter name for the method signature.
>>>> How important are good parameter names?  What if it's a private
>>>> method with only one caller?  Etc.
>>>> It seems that all this nuisance would go away almost for free if we
>>>> just weaken the restriction to "effectively final from point of capture".
>>>> The only possible downside I can see is that this would be a bit
>>>> more annoying to properly specify in the language spec.
>>>> So... opinions?  Good idea?  Bad idea?
>>>> Mark.
>>>> ____________________________________________________________________
>>>> ____________________________________________________________________
>>>> ____________________________________________________________________
>>>> _____________________ “This message is for information purposes
>>>> only, it is not a recommendation, advice, offer or solicitation to
>>>> buy or sell a product or service nor an official confirmation of any
>>>> transaction. It is directed at persons who are professionals and is
>>>> not intended for retail customer use. Intended for recipient only.
>>>> This message is subject to the terms at:
>>>> https://clicktime.symantec.com/3Tkosemgs26WYCPSDAk2JqL6H2?u=www.barclays.com%2Femaildisclaimer.
>>>> For important disclosures, please see:
>>>> https://clicktime.symantec.com/3NjHsmWQRPyUoRU5NWeBWY66H2?u=www.barc
>>>> lays.com%2Fsalesandtradingdisclaimer regarding market commentary
>>>> from Barclays Sales and/or Trading, who are active market
>>>> participants;
>>>> https://clicktime.symantec.com/3DbDfoZBpbhGpNWA6fGnQRJ6H2?u=https%3A
>>>> %2F%2Fwww.investmentbank.barclays.com%2Fdisclosures%2Fbarclays-globa
>>>> l-markets-disclosures.html regarding our standard terms for the
>>>> Investment Bank of Barclays where we trade with you in
>>>> principal-to-principal wholesale markets transactions; and in
>>>> respect of Barclays Research, including disclosures relating to
>>>> specific issuers, please see http://publicresearch.barclays.com.”
>>>> ____________________________________________________________________
>>>> ____________________________________________________________________
>>>> ____________________________________________________________________
>>>> _____________________ If you are incorporated or operating in
>>>> Australia, please see
>>>> https://clicktime.symantec.com/3WstoNBgC44ZMYiFozp7zBb6H2?u=https%3A
>>>> %2F%2Fwww.home.barclays%2Fdisclosures%2Fimportantapacdisclosures.html for
>>>> important disclosure.
>>>> ____________________________________________________________________
>>>> ____________________________________________________________________
>>>> ____________________________________________________________________
>>>> _____________________ How we use personal information  see our
>>>> privacy notice
>>>> https://clicktime.symantec.com/3X9r5gJySmM7eDHfZpFzN5P6H2?u=https%3A
>>>> %2F%2Fwww.investmentbank.barclays.com%2Fdisclosures%2Fpersonalinform
>>>> ationuse.html
>> >> ___________________________________________________________________
>> >> ___________________________________________________________________
>> >> ___________________________________________________________________
>> >> ________________________
> 
> _________________________________________________________________________________________________________________________________________________________________________________________________________________________________
> �This message is for information purposes only, it is not a recommendation,
> advice, offer or solicitation to buy or sell a product or service nor an
> official confirmation of any transaction. It is directed at persons who are
> professionals and is not intended for retail customer use. Intended for
> recipient only. This message is subject to the terms at:
> www.barclays.com/emaildisclaimer.
> 
> For important disclosures, please see:
> www.barclays.com/salesandtradingdisclaimer regarding market commentary from
> Barclays Sales and/or Trading, who are active market participants;
> https://www.investmentbank.barclays.com/disclosures/barclays-global-markets-disclosures.html
> regarding our standard terms for the Investment Bank of Barclays where we trade
> with you in principal-to-principal wholesale markets transactions; and in
> respect of Barclays Research, including disclosures relating to specific
> issuers, please see http://publicresearch.barclays.com.�
> _________________________________________________________________________________________________________________________________________________________________________________________________________________________________
> If you are incorporated or operating in Australia, please see
> https://www.home.barclays/disclosures/importantapacdisclosures.html for
> important disclosure.
> _________________________________________________________________________________________________________________________________________________________________________________________________________________________________
> How we use personal information  see our privacy notice
> https://www.investmentbank.barclays.com/disclosures/personalinformationuse.html
> _________________________________________________________________________________________________________________________________________________________________________________________________________________________________


More information about the jdk-dev mailing list