crazy idea: weaken the effectively final restriction

mark.yagnatinsky at barclays.com mark.yagnatinsky at barclays.com
Wed Oct 6 17:05:57 UTC 2021 

Re: guarded switch: I confess I have not been watching the latest developments in pattern matching.  (Enhanced instanceof is great though!)
If it turns out that effectively final makes sense there too, then I guess that's another strike against my idea; might as well re-use the same concept with lambdas too.

Re: reading code too fast could result in overlooking an assignment: this has recently become a "solved problem" for me, because IntelliJ by default underlines local variables that are NOT effectively final.

I do have one quibble with what you said though: my proposed semantics do NOT behave differently in the two examples you listed.
In particular, neither example compiles, since there is an assignment to the variable AFTER it was already captured.
Of course, your misunderstanding is also a strike against my idea: if I can't even explain it to experts, what hope is there that mere mortals can keep this straight?

-----Original Message-----
From: Remi Forax <forax at univ-mlv.fr> 
Sent: Wednesday, October 6, 2021 12:24 PM
To: Brian Goetz <brian.goetz at oracle.com>
Cc: raffaello giulietti <raffaello.giulietti at gmail.com>; Yagnatinsky, Mark : Markets Pre Trade <mark.yagnatinsky at barclays.com>; jdk-dev <jdk-dev at openjdk.java.net>
Subject: Re: crazy idea: weaken the effectively final restriction


CAUTION: This email originated from outside our organisation - forax at univ-mlv.fr Do not click on links, open attachments, or respond unless you recognize the sender and can validate the content is safe.
Hi Mark,
to complete Brian answer,

- Reassigning a local variable without being inside a loop is usually seen as a code smell, because when you read the code a little to fast you can miss the second assignment thus believe that the code behave differently than what is written.

- The semantics you propose behave differently if the captured variable is available outside the loop or not, so
  Runnable r = () -> println(s);
  s = normalize(s);

will behave differently from
  var s =
  for(...) {
    Runnable r = () -> println(s);
    s = normalize(s);
  }
  ... s ...

If you see the local variable in SSA form, the semantics depend if there is a phi or not. 

- We also did not want to have a semantics depending on what you call "the point of capture", i.e. we wanted to have a semantics that does not explicitly specify when the lambda is created, this allows implementations/translation strategies to create reusable lambda instance either lazily or eagerly. This is something important because we can support both invokedynamic which does the allocation as late as possible and GraalVM native image / AppCDS that may store the lambda instance directly in the image.

You may think that we are talking about capture here so a lambda instance can not be reused but a captured value can be proved to be a constant by a static/runtime analysis.

- Effectively final is a semantics that is used not only for lambdas but also for anonymous classes and recently for guarded patterns, for a pattern the "the point of capture" is not clearly defined. Is it when you want to evaluate the switch ? or when you evaluate the guard of the pattern ?


So yes, we can try to tweak that rule but as Brian said, it just doesn’t seem worth the complexity.

regards,
Rémi


----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "raffaello giulietti" <raffaello.giulietti at gmail.com>
> Cc: "mark yagnatinsky" <mark.yagnatinsky at barclays.com>, "jdk-dev" 
> <jdk-dev at openjdk.java.net>
> Sent: Mercredi 6 Octobre 2021 17:12:18
> Subject: Re: crazy idea: weaken the effectively final restriction

> A good way to think about this is to zoom out and realize that there 
> is a spectrum, with trade-offs.  At one end of the spectrum is “no 
> capture at all”, which is simple and easy to reason about, but … 
> unsatisfying.  At the other end is that we close over variables rather 
> than values, which some languages do, which is defensible but leads to 
> surprising bugs.  In the middle are options like “capture final 
> variables only”, “capture effectively final variables only”, etc.  Your option is somewhat to the “right” of “effectively final.”
> 
> What you’re saying is that we could capture things that are not 
> effectively final, if we can prove that all writes happen before the 
> capture point in the execution order.  Java does flow analysis to 
> determine definite assignment, so such an enhanced flow analysis is imaginable.
> 
> The way to think about this is whether the incremental expressiveness 
> warrants the incremental complexity.  Yes, there are a small number of 
> cases where the more refined analysis would make a difference, but it’s really not that many.
> On the other side, “effectively final” is an easier concept to explain 
> to developers; there is definitely incremental complexity in the user model here.
> Is the trade worth it?  My sense: meh.  I don’t see a huge degree of 
> leverage here.
> 
> A similar example where we chose a simpler rule than necessary was in `var`.
> The intent is that local variable type inference is for 
> implementation, not for API.  So you can use it for locals, but not 
> for, say, method returns, because that’s API, not just implementation.  
> Invariably, some slightly-too-clever person asks “but, then why can’t 
> I use it for the return of a *private* method?  That’s not API.  AHA!  Gotcha!”
> 
> The answer is of course simple; we could make the boundary of the 
> feature arbitrarily complex, but if the incremental complexity makes 
> it harder for people to reason about when they can use var and when 
> not, then we’re not necessarily helping by making it incrementally 
> more expressive.  A simple rule that is easy to reason about is often 
> better than a fractally complex one that is slightly more powerful.
> 
> So, to summarize, the idea isn’t crazy, it just doesn’t seem worth it in the
> balance between complexity and expressiveness.   I’d rather spend that
> complexity budget on something with more leverage.
> 
> Cheers,
> -Brian
> 
>> On Oct 6, 2021, at 8:51 AM, Raffaello Giulietti 
>> <raffaello.giulietti at gmail.com>
>> wrote:
>> 
>> Hi,
>> 
>> what if the variable is reassigned after the capture?
>> 
>> Runnable f(String s) {
>>    Runnable r = () -> println(s);
>>    s = normalize(s); // perfectly useless assignment
>>    return r;
>> }
>> 
>> It would not be "effectively final from point of capture", so would 
>> this lead to a compilation error?
>> 
>> 
>> Greetings
>> Raffaello
>> 
>> 
>> 
>> On 2021-10-05 23:55, mark.yagnatinsky at barclays.com wrote:
>>> I'm not sure if this is the right list for this; if not I hope 
>>> someone can redirect me.
>>> Java requires that local variables that are captured by a lambda 
>>> must be effectively final.
>>> This restriction has the benefit that no one needs to worry about 
>>> annoying questions such as "what are the semantics of a data race on a local variable"
>>> and other such horrors.
>>> But this benefit can still be obtained by a weaker restriction.  For 
>>> instance, consider this code, which currently does not compile:
>>> Runnable f(String s) {
>>>                 s = normalize(s);
>>>                 return () -> println(s); // no can do: s is not 
>>> effectively final } However, this seems a bit silly because although 
>>> s is not final throughout the entire method, it's final where it 
>>> actually matters.
>>> Namely, it is final from the point of capture to the end of the 
>>> method, and that's the only condition we need to avoid those annoying questions.
>>> What do we gain by allowing the code above?  Well, if we don't allow 
>>> it, how can the code above be fixed?  We would have to introduce a new local variable.
>>> That would actually be fine, except for another rule Java has, also 
>>> for a good
>>> reason: one local variable name can NOT "shadow" another: all local 
>>> variables must have distinct names.
>>> That is, we can do something like this: String s = "hello"; String s 
>>> = s.trim(); So we need to come up with a new name, and this has two costs:
>>> First, coming up with good names is hard, and coming up with two 
>>> good names for what is basically the same thing is harder.
>>> Second, suppose that when we first wrote the method, the variable 
>>> really was effectively final, and we needed to change it many months/years/decades later.
>>> We now need to update the appropriate usages to the new name.  This 
>>> tends to clutter line-based diffs, creating more work for reviewers 
>>> during pull requests, and also for code archeologists.
>>> Third (did I say two?): we must spend mental bandwidth deciding how 
>>> to accomplish renaming things.  In the example above we have at least two options:
>>> Runnable f(String t) {// option 1: rename original
>>>                 String s = normalize(t);
>>>                 return () -> println(s); } Runnable f(String s) {// 
>>> option 2: keep original
>>>                 String t = normalize(s);
>>>                 return () -> println(t); } Deciding which option is 
>>> better might actually involve non-trivial tradeoffs.
>>> Perhaps option 1 leads to a smaller diff, but option 2 leads to a 
>>> better parameter name for the method signature.
>>> How important are good parameter names?  What if it's a private 
>>> method with only one caller?  Etc.
>>> It seems that all this nuisance would go away almost for free if we 
>>> just weaken the restriction to "effectively final from point of capture".
>>> The only possible downside I can see is that this would be a bit 
>>> more annoying to properly specify in the language spec.
>>> So... opinions?  Good idea?  Bad idea?
>>> Mark.
>>> ____________________________________________________________________
>>> ____________________________________________________________________
>>> ____________________________________________________________________
>>> _____________________ “This message is for information purposes 
>>> only, it is not a recommendation, advice, offer or solicitation to 
>>> buy or sell a product or service nor an official confirmation of any 
>>> transaction. It is directed at persons who are professionals and is 
>>> not intended for retail customer use. Intended for recipient only. 
>>> This message is subject to the terms at:
>>> https://clicktime.symantec.com/3Tkosemgs26WYCPSDAk2JqL6H2?u=www.barclays.com%2Femaildisclaimer.
>>> For important disclosures, please see:
>>> https://clicktime.symantec.com/3NjHsmWQRPyUoRU5NWeBWY66H2?u=www.barc
>>> lays.com%2Fsalesandtradingdisclaimer regarding market commentary 
>>> from Barclays Sales and/or Trading, who are active market 
>>> participants; 
>>> https://clicktime.symantec.com/3DbDfoZBpbhGpNWA6fGnQRJ6H2?u=https%3A
>>> %2F%2Fwww.investmentbank.barclays.com%2Fdisclosures%2Fbarclays-globa
>>> l-markets-disclosures.html regarding our standard terms for the 
>>> Investment Bank of Barclays where we trade with you in 
>>> principal-to-principal wholesale markets transactions; and in 
>>> respect of Barclays Research, including disclosures relating to 
>>> specific issuers, please see http://publicresearch.barclays.com.”
>>> ____________________________________________________________________
>>> ____________________________________________________________________
>>> ____________________________________________________________________
>>> _____________________ If you are incorporated or operating in 
>>> Australia, please see 
>>> https://clicktime.symantec.com/3WstoNBgC44ZMYiFozp7zBb6H2?u=https%3A
>>> %2F%2Fwww.home.barclays%2Fdisclosures%2Fimportantapacdisclosures.html for important disclosure.
>>> ____________________________________________________________________
>>> ____________________________________________________________________
>>> ____________________________________________________________________
>>> _____________________ How we use personal information  see our 
>>> privacy notice 
>>> https://clicktime.symantec.com/3X9r5gJySmM7eDHfZpFzN5P6H2?u=https%3A
>>> %2F%2Fwww.investmentbank.barclays.com%2Fdisclosures%2Fpersonalinform
>>> ationuse.html
> >> ___________________________________________________________________
> >> ___________________________________________________________________
> >> ___________________________________________________________________
> >> ________________________

_________________________________________________________________________________________________________________________________________________________________________________________________________________________________
“This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.

For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; https://www.investmentbank.barclays.com/disclosures/barclays-global-markets-disclosures.html regarding our standard terms for the Investment Bank of Barclays where we trade with you in principal-to-principal wholesale markets transactions; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.”  
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________
If you are incorporated or operating in Australia, please see https://www.home.barclays/disclosures/importantapacdisclosures.html for important disclosure.
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________
How we use personal information  see our privacy notice https://www.investmentbank.barclays.com/disclosures/personalinformationuse.html 
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________

More information about the jdk-dev mailing list