crazy idea: weaken the effectively final restriction

[mark.yagnatinsky at barclays.com]

Thank you the detailed response!

I wonder if local variable type inference is the right analogy here.  Let me propose my own analogy: type inference for method overloads and its interaction with lambdas and such.
The rules there are insanely complicated, to the point where hardly anyone understands them.
But all that complexity accomplishes something very nice: things tend to Just Work, so no one has to even think about the rules most of the time.
Sometimes you hit a case where it breaks down and you have to be explicit about something the compiler "should" be able to infer, but it's pretty rare.
(Though I few times I've hit a case where the compiler didn't like my comparator because of this.)

Similarly, this proposal trades off explain-ability of rules (though they still remain simpler than the stuff that goes on with method overloads I think), and in exchange buys freedom from the rules getting in your way.
Whether this trade-off is worth it I think should ultimately depend on how often such "pointless" variable declarations end up being needed.
For instance, I think you would agree that if such variable declarations made up 10% of all local variables in a typical Java code base, this feature probably would be worth the complexity.
I'm not sure how common they actually are; obviously, they are nowhere near 10%.

Okay I'll stop arguing/whining now :)
Mark.

P.S. thanks again for the detailed response!

-----Original Message-----
From: Brian Goetz <brian.goetz at oracle.com> 
Sent: Wednesday, October 6, 2021 11:12 AM
To: Raffaello Giulietti <raffaello.giulietti at gmail.com>
Cc: Yagnatinsky, Mark : Markets Pre Trade <mark.yagnatinsky at barclays.com>; jdk-dev at openjdk.java.net
Subject: Re: crazy idea: weaken the effectively final restriction


CAUTION: This email originated from outside our organisation - brian.goetz at oracle.com Do not click on links, open attachments, or respond unless you recognize the sender and can validate the content is safe.
A good way to think about this is to zoom out and realize that there is a spectrum, with trade-offs.  At one end of the spectrum is “no capture at all”, which is simple and easy to reason about, but … unsatisfying.  At the other end is that we close over variables rather than values, which some languages do, which is defensible but leads to surprising bugs.  In the middle are options like “capture final variables only”, “capture effectively final variables only”, etc.  Your option is somewhat to the “right” of “effectively final.”  

What you’re saying is that we could capture things that are not effectively final, if we can prove that all writes happen before the capture point in the execution order.  Java does flow analysis to determine definite assignment, so such an enhanced flow analysis is imaginable.  

The way to think about this is whether the incremental expressiveness warrants the incremental complexity.  Yes, there are a small number of cases where the more refined analysis would make a difference, but it’s really not that many.  On the other side, “effectively final” is an easier concept to explain to developers; there is definitely incremental complexity in the user model here.  Is the trade worth it?  My sense: meh.  I don’t see a huge degree of leverage here.  

A similar example where we chose a simpler rule than necessary was in `var`.  The intent is that local variable type inference is for implementation, not for API.  So you can use it for locals, but not for, say, method returns, because that’s API, not just implementation.  Invariably, some slightly-too-clever person asks “but, then why can’t I use it for the return of a *private* method?  That’s not API.  AHA!  Gotcha!”  

The answer is of course simple; we could make the boundary of the feature arbitrarily complex, but if the incremental complexity makes it harder for people to reason about when they can use var and when not, then we’re not necessarily helping by making it incrementally more expressive.  A simple rule that is easy to reason about is often better than a fractally complex one that is slightly more powerful.  

So, to summarize, the idea isn’t crazy, it just doesn’t seem worth it in the balance between complexity and expressiveness.   I’d rather spend that complexity budget on something with more leverage.

Cheers,
-Brian

> On Oct 6, 2021, at 8:51 AM, Raffaello Giulietti <raffaello.giulietti at gmail.com> wrote:
> 
> Hi,
> 
> what if the variable is reassigned after the capture?
> 
> Runnable f(String s) {
>    Runnable r = () -> println(s);
>    s = normalize(s); // perfectly useless assignment
>    return r;
> }
> 
> It would not be "effectively final from point of capture", so would this lead to a compilation error?
> 
> 
> Greetings
> Raffaello
> 
> 
> 
> On 2021-10-05 23:55, mark.yagnatinsky at barclays.com wrote:
>> I'm not sure if this is the right list for this; if not I hope someone can redirect me.
>> Java requires that local variables that are captured by a lambda must be effectively final.
>> This restriction has the benefit that no one needs to worry about annoying questions such as "what are the semantics of a data race on a local variable" and other such horrors.
>> But this benefit can still be obtained by a weaker restriction.  For instance, consider this code, which currently does not compile:
>> Runnable f(String s) {
>>                 s = normalize(s);
>>                 return () -> println(s); // no can do: s is not 
>> effectively final } However, this seems a bit silly because although 
>> s is not final throughout the entire method, it's final where it actually matters.
>> Namely, it is final from the point of capture to the end of the method, and that's the only condition we need to avoid those annoying questions.
>> What do we gain by allowing the code above?  Well, if we don't allow it, how can the code above be fixed?  We would have to introduce a new local variable.
>> That would actually be fine, except for another rule Java has, also for a good reason: one local variable name can NOT "shadow" another: all local variables must have distinct names.
>> That is, we can do something like this: String s = "hello"; String s 
>> = s.trim(); So we need to come up with a new name, and this has two costs:
>> First, coming up with good names is hard, and coming up with two good names for what is basically the same thing is harder.
>> Second, suppose that when we first wrote the method, the variable really was effectively final, and we needed to change it many months/years/decades later.
>> We now need to update the appropriate usages to the new name.  This tends to clutter line-based diffs, creating more work for reviewers during pull requests, and also for code archeologists.
>> Third (did I say two?): we must spend mental bandwidth deciding how to accomplish renaming things.  In the example above we have at least two options:
>> Runnable f(String t) {// option 1: rename original
>>                 String s = normalize(t);
>>                 return () -> println(s); } Runnable f(String s) {// 
>> option 2: keep original
>>                 String t = normalize(s);
>>                 return () -> println(t); } Deciding which option is 
>> better might actually involve non-trivial tradeoffs.  Perhaps option 1 leads to a smaller diff, but option 2 leads to a better parameter name for the method signature.
>> How important are good parameter names?  What if it's a private method with only one caller?  Etc.
>> It seems that all this nuisance would go away almost for free if we just weaken the restriction to "effectively final from point of capture".
>> The only possible downside I can see is that this would be a bit more annoying to properly specify in the language spec.
>> So... opinions?  Good idea?  Bad idea?
>> Mark.
>> _____________________________________________________________________
>> _____________________________________________________________________
>> _____________________________________________________________________
>> __________________ “This message is for information purposes only, it 
>> is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: https://clicktime.symantec.com/3RvVXadG34hwCBkP8n8LdA6H2?u=www.barclays.com%2Femaildisclaimer.
>> For important disclosures, please see: https://clicktime.symantec.com/32YvpMq3ywF4MBUgrYcExHk6H2?u=www.barclays.com%2Fsalesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; https://clicktime.symantec.com/3CeFiviotYo7kzb3sfoB4H46H2?u=https%3A%2F%2Fwww.investmentbank.barclays.com%2Fdisclosures%2Fbarclays-global-markets-disclosures.html regarding our standard terms for the Investment Bank of Barclays where we trade with you in principal-to-principal wholesale markets transactions; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.”
>> _____________________________________________________________________
>> _____________________________________________________________________
>> _____________________________________________________________________
>> __________________ If you are incorporated or operating in Australia, 
>> please see https://clicktime.symantec.com/363JsWkoFWSnAzXKqMUkAxC6H2?u=https%3A%2F%2Fwww.home.barclays%2Fdisclosures%2Fimportantapacdisclosures.html for important disclosure.
>> _____________________________________________________________________
>> _____________________________________________________________________
>> _____________________________________________________________________
>> __________________ How we use personal information  see our privacy 
>> notice 
>> https://clicktime.symantec.com/3KcfyEdGC8pv5YeHZ23xfDv6H2?u=https%3A%
>> 2F%2Fwww.investmentbank.barclays.com%2Fdisclosures%2Fpersonalinformat
>> ionuse.html 
>> _____________________________________________________________________
>> _____________________________________________________________________
>> _____________________________________________________________________
>> __________________


_________________________________________________________________________________________________________________________________________________________________________________________________________________________________
“This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.

For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; https://www.investmentbank.barclays.com/disclosures/barclays-global-markets-disclosures.html regarding our standard terms for the Investment Bank of Barclays where we trade with you in principal-to-principal wholesale markets transactions; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.”  
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________
If you are incorporated or operating in Australia, please see https://www.home.barclays/disclosures/importantapacdisclosures.html for important disclosure.
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________
How we use personal information  see our privacy notice https://www.investmentbank.barclays.com/disclosures/personalinformationuse.html 
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________