From archie.cobbs at gmail.com Thu Dec 1 18:09:11 2022 From: archie.cobbs at gmail.com (Archie Cobbs) Date: Thu, 1 Dec 2022 12:09:11 -0600 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment Message-ID: This relates to JDK-8193904 (if amber-dev is the wrong place to ask about this please let me know; here's an earlier thread on compiler-dev). The bug asks the question, should we change the JLS so that "MyClass.this" is treated the same way as "this" with respect to definite assignment? Here's how the compiler currently behaves: // This class compiles without error public class Example1 { final int foo; public Example1() { System.err.println(Example1.this.foo); // no error generated here this.foo = 42; } } // This class fails to compile public class Example2 { private final int foo; public Example2() { Example2.this.foo = 42; // "cannot assign a value to final variable foo" } } Presumably the reason for the different treatment is because in general the qualifying expression could be arbitrarily complex and the compiler can't be expected to detect any possible qualification in front of "foo", so why even start down that road? For example, instead of Example1.this = 42; you could have: ((Example1)Function.identity().apply(this)).foo = 42; Currently JLS ?16 says: > An access to its value consists of the simple name of the variable (or, > for a field, the simple name of the field qualified by this) occurring > anywhere in an expression except as the left-hand operand of the simple > assignment operator = (?15.26.1 > ). > > Some options... 1. Deprecate the use of "MyClass.this" where "this" would work and start generating a warning 2. Update JLS ?16 to say "qualified by this or qualified by MyClass.this" 3. Do nothing and resolve JDK-8193904 as "Won't Fix" 4. ?? Thoughts? -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Thu Dec 1 19:02:18 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 1 Dec 2022 14:02:18 -0500 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: References: Message-ID: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> I can think of multiple explanations for why this wasn't done in 1.1 when the `MyClass.this` syntax was introduced, but I think there's a little more detangling to be done before we have a clear menu of options, because we have to consider that `Example1.this` only means `this` when there are no intervening class declarations between the use and the declaration of Example1. To come up with a sensible meaning of option (2), I think we have to assess the spec impact of threading the equivalence of `[implicit C].this` and `C.this` through the spec? On 12/1/2022 1:09 PM, Archie Cobbs wrote: > This relates to JDK-8193904 > (if amber-dev is the > wrong place to ask about this please let me know; here's an earlier > thread > > on compiler-dev). > > The bug asks the question, should we change the JLS so that > "MyClass.this" is treated the same way as "this" with respect to > definite assignment? > > Here's how the compiler currently behaves: > > // This class compiles without error > public class Example1 { > ? ? final int foo; > ? ? public Example1() { > System.err.println(Example1.this.foo); ?// no error generated here > ? ? ? ? this.foo = 42; > ? ? } > } > > // This class fails to compile > public class Example2 { > ? ? private final int foo; > ? ? public Example2() { > Example2.this.foo = 42; // "cannot assign a value to final variablefoo" > ? ? } > } > > Presumably the reason for the different treatment is because in > general the qualifying expression could be arbitrarily complex and the > compiler can't be expected to detect any possible qualification in > front of "foo", so why even start down that road? > > For example, instead of > > Example1.this = 42; > > you could have: > > ((Example1)Function.identity().apply(this)).foo = 42; > > Currently JLS ?16 says: > > An access to its value consists of the simple name of the variable > (or, for a field, the simple name of the field qualified by > |this|) occurring anywhere in an expression except as the > left-hand operand of the simple assignment operator |=| (?15.26.1 > ). > > > Some options... > > 1. Deprecate the use of "MyClass.this" where "this" would work and > start generating a warning > 2. Update JLS ?16 to say "qualified by this or qualified by MyClass.this" > 3. Do nothing and resolve JDK-8193904 as "Won't Fix" > 4. ?? > > Thoughts? > > -Archie > > -- > Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From archie.cobbs at gmail.com Thu Dec 1 21:29:18 2022 From: archie.cobbs at gmail.com (Archie Cobbs) Date: Thu, 1 Dec 2022 15:29:18 -0600 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> References: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> Message-ID: On Thu, Dec 1, 2022 at 1:03 PM Brian Goetz wrote: > To come up with a sensible meaning of option (2), I think we have to > assess the spec impact of threading the equivalence of `[implicit C].this` > and `C.this` through the spec? > I think it depends on how ambitious you want to be. At its narrowest, this issue is not about the equivalence of "MyClass.this" and "this", it's about the equivalence of "MyClass.this.foo" and "this.foo", and only as that relates to definite assignment, i.e., it's about the definition of the phrase "access to its value" in the first two sentences of JLS ?16. And of course this definition is a lexical one, not a semantic one. I agree it would be nicer if the entire JLS were refactored to make "MyClass.this" and "this" always mean the same thing (only when "this" has type MyClass of course), but that's a bigger change than is strictly necessary here. And I'm fuzzier on what that would even mean... what other sections are there besides ?16 that talk about "this" and/or "MyClass.this" in lexical terms instead of in terms of the object reference that they represent? I'm not familiar enough to know if there even are any, or which sections they are, off the top of my head. -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri Dec 2 17:13:50 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 2 Dec 2022 12:13:50 -0500 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: References: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> Message-ID: <002e37c5-e8c7-cc32-9339-62920b07d59f@oracle.com> Re-reading Ch16, you're right, this seems simpler than I was making it.? There are several sentences of the form > Such an assignment is defined to occur if and only if either the > simple name of the > variable (or, for a field, its simple name qualified by this) I think these can be refined to say something like (or for a field, its simple named qualified by `this` (both qualified and unqualified)) This language is already present in 16.1.10, but missing in other uses of "qualified by this" in Ch16.? We're not talking about `this` and `X.this`, but about different ways to name the same field. On 12/1/2022 4:29 PM, Archie Cobbs wrote: > On Thu, Dec 1, 2022 at 1:03 PM Brian Goetz wrote: > > To come up with a sensible meaning of option (2), I think we have > to assess the spec impact of threading the equivalence of > `[implicit C].this` and `C.this` through the spec? > > > I think it depends on how ambitious you want to be. > > At its narrowest, this issue is not about the equivalence of > "MyClass.this" and "this", it's about the equivalence of > "MyClass.this.foo" and "this.foo", and only as that relates to > definite assignment, i.e., it's about the definition of the phrase > "access to its value" in the first two sentences of JLS ?16. And of > course this definition is a lexical one, not a semantic one. > > I agree it would be nicer if the entire JLS were refactored to make > "MyClass.this" and "this" always mean the same thing (only when "this" > has type MyClass of course), but that's a bigger change than is > strictly necessary here. And I'm fuzzier on what that would even > mean... what other sections are there besides ?16 that talk about > "this" and/or "MyClass.this" in lexical terms instead of in terms of > the object reference that they represent? I'm not familiar enough to > know if there even are any, or which sections they are, off the top of > my head. > > -Archie > > -- > Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.r.rose at oracle.com Fri Dec 2 19:30:05 2022 From: john.r.rose at oracle.com (John Rose) Date: Fri, 02 Dec 2022 11:30:05 -0800 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: <002e37c5-e8c7-cc32-9339-62920b07d59f@oracle.com> References: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> <002e37c5-e8c7-cc32-9339-62920b07d59f@oracle.com> Message-ID: <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> I was there when these rules were written, so I added a comment to the bug explaining what?s up here. https://bugs.openjdk.org/browse/JDK-8193904?focusedCommentId=14542315&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14542315 In short, we have no good changes on the table here. The JLS is the way it is at this point intentionally for good reasons, summarized in the above comment. Since changing the JLS is super-expensive, even a neutral change would need to be highly beneficial to the whole ecosystem. Removing perceived inconsistencies (that have been unchanged for 25 years) is not highly beneficial to the whole ecosystem. I think Archie got at the heart of the problem in this comment here: > Presumably the reason for the different treatment is because in general the qualifying expression could be arbitrarily complex and the compiler can't be expected to detect any possible qualification in front of "foo", so why even start down that road? Exactly. Don?t start down that road, unless (1) you know where it goes, and (2) that it gets somewhere good, and (3) you can get there before you run out of gas. Don?t start just because you like the first billboard you see, or on a road which takes you to a tarpit, or with only enough supplies to travel on the first day of a week-long trip. I think we might have all three problems here. Regarding (2), better uninitialized variable control is I think the proper goal here, not some consistency of all the things. Dynamic checks are required to get the next level of value, IMO. Please see my comment on the bug for more details. Regarding the PR: It intentionally changes javac to deviate from the JLS. That?s a non-starter. (It got my attention but that?s a limited resource!) Before we change javac to make it compile a language we like better, we first have to decide to change the JLS. The JLS is not the Java Language Suggestions; it is the foundation of our whole ecosystem, and every word in it counts. Breaking those words breaks promises of compatibility we have made to literally millions of programmers over decades. Regarding tweaks to the Java language: It might be interesting to look up the history of Project Coin, where we explicitly solicited, discussed, and implemented small improvements to the JLS and (only then) the Java toolchain. Amber and Valhalla are the latest venues that we have created for making JLS changes, and this time around the changes tend to be larger in scale (than in Coin). But we constantly keep in mind the possibility of applying Amber or Valhalla feature work more widely or ?consistently? (for a reasonable sense of ?consistent?). By the way, Brian is the final authority on such work today, and the present note is my interpretation of how JLS evolution is working under his leadership. Here?s a current (not decades old) example of how static analysis in the JLS is evolving and how the rules are adjusted. The JLS have always constrained use of ?this? in a constructor; you have to call ?super? first, but then you have your own uninitialized fields you are responsible. (And the ?super? could have called one of your own overrides, which lets the super access the uninitialized value of one of your fields. Turing machines are wily beasts.) Now, Valhalla and Amber are adjusting the rules for initializing fields in constructors, so that they are different for classic Java classes, and for Valhalla value classes, and for Amber records. With Valhalla value classes in the current prototype, there is not ?super? call, but you cannot touch ?this? until every field of ?this? is definitely assigned. We could have done that from the beginning for all classes, but it would have been a hard restriction to live with. Imagine the objections: ?You mean I can?t call my logging helper function at the top of my constructor?? Still, though the designs are well-reasoned, the differences bug me a little. Since the differences apply to emerging parts of the language, we might consider spreading around the rules for value classes so they benefit other kinds of code. What I?m thinking of (Brian you know this already) is perhaps we could take some error conditions for premature use of ?this? in value classes and turn them into warnings for classic classes, so that code that circumvents the DU/DA rules, to obtain access to blank final fields uninitialized, will suddenly sprout lint warnings. (Or even turn the Valhalla errors into warnings, if we decide consistency wins at that point; it doesn?t always win, not by a long shot.) The extra warnings might be tolerable to the ecosystem, and it would be useful for some users (not all) to chase down the warnings and either fix code or suppress the warnings explicitly. But, for this idea, as for any other, would the benefits to the ecosystem be worth the development cost? A sharper question is, should we pause some other language feature before we enact some cleanups or generalizations that appeal to us? We can?t do everything at once, and just having a clever idea (like better warnings on incomplete ?this?) or a clever fix to javac code (like Archie?s) doesn?t help us decide what our priorities are. (And static checks only get us a subset of safety. I hope someday to add dynamic checks for uninitialized variables to the mix, to catch more user errors sooner.) If I?ve missed some other goal for this PR besides adjusting DA/DU rules and/or increasing ?consistency?, I apologize for missing it; it wasn?t clear to me. But the above is where my mind goes when the question arises ?what about the rules for this.x?? ? John From numeralnathan at gmail.com Fri Dec 2 20:29:58 2022 From: numeralnathan at gmail.com (Nathan Reynolds) Date: Fri, 2 Dec 2022 13:29:58 -0700 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> References: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> <002e37c5-e8c7-cc32-9339-62920b07d59f@oracle.com> <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> Message-ID: > code that circumvents the DU/DA rules, to obtain access to blank final fields uninitialized, will suddenly sprout lint warnings. PMD already flags a problem if the constructor calls a non-final member method. Is this sufficient? Are there holes in this rule? Of course, the non-final member method can be side-effect free but the rule doesn't care. If the PMD rule already flags all the problems, then we are really talking about adding that rule as a warning to javac. On Fri, Dec 2, 2022 at 12:30 PM John Rose wrote: > I was there when these rules were written, so I added a comment to the bug > explaining what?s up here. > > > https://bugs.openjdk.org/browse/JDK-8193904?focusedCommentId=14542315&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14542315 > > In short, we have no good changes on the table here. The JLS is the way > it is at this point intentionally for good reasons, summarized in the above > comment. > > Since changing the JLS is super-expensive, even a neutral change would > need to be highly beneficial to the whole ecosystem. Removing perceived > inconsistencies (that have been unchanged for 25 years) is not highly > beneficial to the whole ecosystem. > > I think Archie got at the heart of the problem in this comment here: > > > Presumably the reason for the different treatment is because in general > the qualifying expression could be arbitrarily complex and the compiler > can't be expected to detect any possible qualification in front of "foo", > so why even start down that road? > > Exactly. Don?t start down that road, unless (1) you know where it goes, > and (2) that it gets somewhere good, and (3) you can get there before you > run out of gas. Don?t start just because you like the first billboard you > see, or on a road which takes you to a tarpit, or with only enough supplies > to travel on the first day of a week-long trip. I think we might have all > three problems here. > > Regarding (2), better uninitialized variable control is I think the proper > goal here, not some consistency of all the things. Dynamic checks are > required to get the next level of value, IMO. Please see my comment on the > bug for more details. > > Regarding the PR: It intentionally changes javac to deviate from the > JLS. That?s a non-starter. (It got my attention but that?s a limited > resource!) Before we change javac to make it compile a language we like > better, we first have to decide to change the JLS. The JLS is not the Java > Language Suggestions; it is the foundation of our whole ecosystem, and > every word in it counts. Breaking those words breaks promises of > compatibility we have made to literally millions of programmers over > decades. > > Regarding tweaks to the Java language: It might be interesting to look up > the history of Project Coin, where we explicitly solicited, discussed, and > implemented small improvements to the JLS and (only then) the Java > toolchain. Amber and Valhalla are the latest venues that we have created > for making JLS changes, and this time around the changes tend to be larger > in scale (than in Coin). But we constantly keep in mind the possibility of > applying Amber or Valhalla feature work more widely or ?consistently? (for > a reasonable sense of ?consistent?). By the way, Brian is the final > authority on such work today, and the present note is my interpretation of > how JLS evolution is working under his leadership. > > Here?s a current (not decades old) example of how static analysis in the > JLS is evolving and how the rules are adjusted. The JLS have always > constrained use of ?this? in a constructor; you have to call ?super? first, > but then you have your own uninitialized fields you are responsible. (And > the ?super? could have called one of your own overrides, which lets the > super access the uninitialized value of one of your fields. Turing > machines are wily beasts.) > > Now, Valhalla and Amber are adjusting the rules for initializing fields in > constructors, so that they are different for classic Java classes, and for > Valhalla value classes, and for Amber records. With Valhalla value classes > in the current prototype, there is not ?super? call, but you cannot touch > ?this? until every field of ?this? is definitely assigned. We could have > done that from the beginning for all classes, but it would have been a hard > restriction to live with. Imagine the objections: ?You mean I can?t call > my logging helper function at the top of my constructor?? > > Still, though the designs are well-reasoned, the differences bug me a > little. Since the differences apply to emerging parts of the language, we > might consider spreading around the rules for value classes so they benefit > other kinds of code. What I?m thinking of (Brian you know this already) is > perhaps we could take some error conditions for premature use of ?this? in > value classes and turn them into warnings for classic classes, so that code > that circumvents the DU/DA rules, to obtain access to blank final fields > uninitialized, will suddenly sprout lint warnings. (Or even turn the > Valhalla errors into warnings, if we decide consistency wins at that point; > it doesn?t always win, not by a long shot.) The extra warnings might be > tolerable to the ecosystem, and it would be useful for some users (not all) > to chase down the warnings and either fix code or suppress the warnings > explicitly. > > But, for this idea, as for any other, would the benefits to the ecosystem > be worth the development cost? A sharper question is, should we pause some > other language feature before we enact some cleanups or generalizations > that appeal to us? We can?t do everything at once, and just having a > clever idea (like better warnings on incomplete ?this?) or a clever fix to > javac code (like Archie?s) doesn?t help us decide what our priorities are. > > (And static checks only get us a subset of safety. I hope someday to add > dynamic checks for uninitialized variables to the mix, to catch more user > errors sooner.) > > If I?ve missed some other goal for this PR besides adjusting DA/DU rules > and/or increasing ?consistency?, I apologize for missing it; it wasn?t > clear to me. But the above is where my mind goes when the question arises > ?what about the rules for this.x?? > > ? John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From archie.cobbs at gmail.com Fri Dec 2 20:32:40 2022 From: archie.cobbs at gmail.com (Archie Cobbs) Date: Fri, 2 Dec 2022 14:32:40 -0600 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> References: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> <002e37c5-e8c7-cc32-9339-62920b07d59f@oracle.com> <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> Message-ID: Hi John, Thanks for the great discussion. I should admit that I only came across this bug because I was trying to do a clean up of compiler bugs that were already fixed or "easy to fix" as a way of learning about the compiler code and making some low-level contributions (if anyone wants to help with the JIRA cleanup see this post and this post :) Of course at first I didn't appreciate that a JLS change was implied and instead charged ahead with "fixing" the bug and submitting a PR... After better understanding the implications, I agree making this change is questionable based on cost/benefit. Surely, uses of "MyClass.this" within MyClass constructors are uncommon. But it's an interesting philosophical question so to speak. I don't have strong opinions either way... I just hope we can decide one way or the other and resolve the bug :) The dynamic check idea is quite interesting... What I?m thinking of (Brian you know this already) is perhaps we could take > some error conditions for premature use of ?this? in value classes and turn > them into warnings for classic classes, so that code that circumvents the > DU/DA rules, to obtain access to blank final fields uninitialized, will > suddenly sprout lint warnings. > This relates to the "this escape" warning discussion of a few weeks ago. I'm (still) working on a prototype for such a warning & hopefully will have something soon. As you point out, the dynamic check idea would work for "this escape" as well... that would be nice down the road. Presumably with a dynamic scheme you could have "perfect" DU/DA analysis, albeit at runtime instead of compile time. -Archie On Fri, Dec 2, 2022 at 1:30 PM John Rose wrote: > I was there when these rules were written, so I added a comment to the bug > explaining what?s up here. > > > https://bugs.openjdk.org/browse/JDK-8193904?focusedCommentId=14542315&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14542315 > > In short, we have no good changes on the table here. The JLS is the way > it is at this point intentionally for good reasons, summarized in the above > comment. > > Since changing the JLS is super-expensive, even a neutral change would > need to be highly beneficial to the whole ecosystem. Removing perceived > inconsistencies (that have been unchanged for 25 years) is not highly > beneficial to the whole ecosystem. > > I think Archie got at the heart of the problem in this comment here: > > > Presumably the reason for the different treatment is because in general > the qualifying expression could be arbitrarily complex and the compiler > can't be expected to detect any possible qualification in front of "foo", > so why even start down that road? > > Exactly. Don?t start down that road, unless (1) you know where it goes, > and (2) that it gets somewhere good, and (3) you can get there before you > run out of gas. Don?t start just because you like the first billboard you > see, or on a road which takes you to a tarpit, or with only enough supplies > to travel on the first day of a week-long trip. I think we might have all > three problems here. > > Regarding (2), better uninitialized variable control is I think the proper > goal here, not some consistency of all the things. Dynamic checks are > required to get the next level of value, IMO. Please see my comment on the > bug for more details. > > Regarding the PR: It intentionally changes javac to deviate from the > JLS. That?s a non-starter. (It got my attention but that?s a limited > resource!) Before we change javac to make it compile a language we like > better, we first have to decide to change the JLS. The JLS is not the Java > Language Suggestions; it is the foundation of our whole ecosystem, and > every word in it counts. Breaking those words breaks promises of > compatibility we have made to literally millions of programmers over > decades. > > Regarding tweaks to the Java language: It might be interesting to look up > the history of Project Coin, where we explicitly solicited, discussed, and > implemented small improvements to the JLS and (only then) the Java > toolchain. Amber and Valhalla are the latest venues that we have created > for making JLS changes, and this time around the changes tend to be larger > in scale (than in Coin). But we constantly keep in mind the possibility of > applying Amber or Valhalla feature work more widely or ?consistently? (for > a reasonable sense of ?consistent?). By the way, Brian is the final > authority on such work today, and the present note is my interpretation of > how JLS evolution is working under his leadership. > > Here?s a current (not decades old) example of how static analysis in the > JLS is evolving and how the rules are adjusted. The JLS have always > constrained use of ?this? in a constructor; you have to call ?super? first, > but then you have your own uninitialized fields you are responsible. (And > the ?super? could have called one of your own overrides, which lets the > super access the uninitialized value of one of your fields. Turing > machines are wily beasts.) > > Now, Valhalla and Amber are adjusting the rules for initializing fields in > constructors, so that they are different for classic Java classes, and for > Valhalla value classes, and for Amber records. With Valhalla value classes > in the current prototype, there is not ?super? call, but you cannot touch > ?this? until every field of ?this? is definitely assigned. We could have > done that from the beginning for all classes, but it would have been a hard > restriction to live with. Imagine the objections: ?You mean I can?t call > my logging helper function at the top of my constructor?? > > Still, though the designs are well-reasoned, the differences bug me a > little. Since the differences apply to emerging parts of the language, we > might consider spreading around the rules for value classes so they benefit > other kinds of code. What I?m thinking of (Brian you know this already) is > perhaps we could take some error conditions for premature use of ?this? in > value classes and turn them into warnings for classic classes, so that code > that circumvents the DU/DA rules, to obtain access to blank final fields > uninitialized, will suddenly sprout lint warnings. (Or even turn the > Valhalla errors into warnings, if we decide consistency wins at that point; > it doesn?t always win, not by a long shot.) The extra warnings might be > tolerable to the ecosystem, and it would be useful for some users (not all) > to chase down the warnings and either fix code or suppress the warnings > explicitly. > > But, for this idea, as for any other, would the benefits to the ecosystem > be worth the development cost? A sharper question is, should we pause some > other language feature before we enact some cleanups or generalizations > that appeal to us? We can?t do everything at once, and just having a > clever idea (like better warnings on incomplete ?this?) or a clever fix to > javac code (like Archie?s) doesn?t help us decide what our priorities are. > > (And static checks only get us a subset of safety. I hope someday to add > dynamic checks for uninitialized variables to the mix, to catch more user > errors sooner.) > > If I?ve missed some other goal for this PR besides adjusting DA/DU rules > and/or increasing ?consistency?, I apologize for missing it; it wasn?t > clear to me. But the above is where my mind goes when the question arises > ?what about the rules for this.x?? > > ? John > -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri Dec 2 20:47:33 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 2 Dec 2022 15:47:33 -0500 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> References: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> <002e37c5-e8c7-cc32-9339-62920b07d59f@oracle.com> <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> Message-ID: <6aad98b1-80dd-fd86-6483-a67961de0f7e@oracle.com> > I think Archie got at the heart of the problem in this comment here: > >> Presumably the reason for the different treatment is because in general the qualifying expression could be arbitrarily complex and the compiler can't be expected to detect any possible qualification in front of "foo", so why even start down that road? > Exactly. Don?t start down that road, unless (1) you know where it goes, and (2) that it gets somewhere good, and (3) you can get there before you run out of gas. Don?t start just because you like the first billboard you see, or on a road which takes you to a tarpit, or with only enough supplies to travel on the first day of a week-long trip. I think we might have all three problems here. I think there are two different perspectives on `MyClass.this.x` that lead to two different views of the world. For right or wrong, my mental model has always been that a field name `f` is shorthand for `this.f` which in turn is shorthand for `MyClass.this.f`.? Under this view, it makes sense to treat all the *names* of f equally, and all three of these are names for f. The alternate view is that `MyClass.this.f` is, in fact, a reference to the *field* `MyClass.this` of an inner class (or some other expression that happens to contain `this`), and therefore is an arbitrary expression that we shouldn't try to analyze through, any more than `(this + 1 - 1).f` would be if we could do arithmetic on references. Under the view to which I baselessly subscribe, treating `MyClass.this.f` as being the fully qualified name of `f` is more consistent, because it is just as much a "name for f" as the bare field or `this.f` is. But I agree overall that this isn't worth spending a lot of spec-revision currency on. From forax at univ-mlv.fr Sat Dec 3 12:02:04 2022 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 3 Dec 2022 13:02:04 +0100 (CET) Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: <6aad98b1-80dd-fd86-6483-a67961de0f7e@oracle.com> References: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> <002e37c5-e8c7-cc32-9339-62920b07d59f@oracle.com> <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> <6aad98b1-80dd-fd86-6483-a67961de0f7e@oracle.com> Message-ID: <538861866.58578669.1670068924414.JavaMail.zimbra@u-pem.fr> ----- Original Message ----- > From: "Brian Goetz" > To: "John Rose" , "Archie Cobbs" > Cc: "amber-dev" > Sent: Friday, December 2, 2022 9:47:33 PM > Subject: Re: Different handing of "this" vs. "MyClass.this" in definite assignment >> I think Archie got at the heart of the problem in this comment here: >> >>> Presumably the reason for the different treatment is because in general the >>> qualifying expression could be arbitrarily complex and the compiler can't be >>> expected to detect any possible qualification in front of "foo", so why even >>> start down that road? >> Exactly. Don?t start down that road, unless (1) you know where it goes, and (2) >> that it gets somewhere good, and (3) you can get there before you run out of >> gas. Don?t start just because you like the first billboard you see, or on a >> road which takes you to a tarpit, or with only enough supplies to travel on the >> first day of a week-long trip. I think we might have all three problems here. > > I think there are two different perspectives on `MyClass.this.x` that > lead to two different views of the world. > > For right or wrong, my mental model has always been that a field name > `f` is shorthand for `this.f` which in turn is shorthand for > `MyClass.this.f`.? Under this view, it makes sense to treat all the > *names* of f equally, and all three of these are names for f. > > The alternate view is that `MyClass.this.f` is, in fact, a reference to > the *field* `MyClass.this` of an inner class (or some other expression > that happens to contain `this`), and therefore is an arbitrary > expression that we shouldn't try to analyze through, any more than > `(this + 1 - 1).f` would be if we could do arithmetic on references. > > Under the view to which I baselessly subscribe, treating > `MyClass.this.f` as being the fully qualified name of `f` is more > consistent, because it is just as much a "name for f" as the bare field > or `this.f` is. As a kind of side story, i've stopped, perhaps 10 years ago, to talk about "CurrentClass.this" as an alias for "this" because my students were confused about "this" being a parameter or a field. So i only show "OuterClass.this". > > But I agree overall that this isn't worth spending a lot of > spec-revision currency on. regards, R?mi From archie.cobbs at gmail.com Tue Dec 6 14:33:25 2022 From: archie.cobbs at gmail.com (Archie Cobbs) Date: Tue, 6 Dec 2022 08:33:25 -0600 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: <6aad98b1-80dd-fd86-6483-a67961de0f7e@oracle.com> References: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> <002e37c5-e8c7-cc32-9339-62920b07d59f@oracle.com> <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> <6aad98b1-80dd-fd86-6483-a67961de0f7e@oracle.com> Message-ID: On Fri, Dec 2, 2022 at 2:47 PM Brian Goetz wrote: > >> Presumably the reason for the different treatment is because in general > the qualifying expression could be arbitrarily complex and the compiler > can't be expected to detect any possible qualification in front of "foo", > so why even start down that road? > > Exactly. Don?t start down that road, unless (1) you know where it goes, > and (2) that it gets somewhere good, and (3) you can get there before you > run out of gas. Don?t start just because you like the first billboard you > see, or on a road which takes you to a tarpit, or with only enough supplies > to travel on the first day of a week-long trip. I think we might have all > three problems here. > > I think there are two different perspectives on `MyClass.this.x` that > lead to two different views of the world. > > .... > > But I agree overall that this isn't worth spending a lot of > spec-revision currency on. > It sounds like there is agreement to leave the spec alone for the time being. I'll retract my PR and leave it up to others to update and/or resolve JDK-8193904. Thanks again for all the good discussion. -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Tue Dec 6 15:49:19 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 6 Dec 2022 10:49:19 -0500 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: References: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> <002e37c5-e8c7-cc32-9339-62920b07d59f@oracle.com> <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> <6aad98b1-80dd-fd86-6483-a67961de0f7e@oracle.com> Message-ID: <19c82579-2c07-e30b-3308-4596533ac0b6@oracle.com> Are there some better warnings you could see coming out of this exploration? On 12/6/2022 9:33 AM, Archie Cobbs wrote: > On Fri, Dec 2, 2022 at 2:47 PM Brian Goetz wrote: > > >> Presumably the reason for the different treatment is because in > general the qualifying expression could be arbitrarily complex and > the compiler can't be expected to detect any possible > qualification in front of "foo", so why even start down that road? > > Exactly.? Don?t start down that road, unless (1) you know where > it goes, and (2) that it gets somewhere good, and (3) you can get > there before you run out of gas.? Don?t start just because you > like the first billboard you see, or on a road which takes you to > a tarpit, or with only enough supplies to travel on the first day > of a week-long trip.? I think we might have all three problems here. > > I think there are two different perspectives on `MyClass.this.x` that > lead to two different views of the world. > > .... > > But I agree overall that this isn't worth spending a lot of > spec-revision currency on. > > > It sounds like there is agreement to leave the spec alone for the time > being. > > I'll retract my PR and leave it up to others to update and/or resolve > JDK-8193904. > > Thanks again for all the good discussion. > > -Archie > > -- > Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From archie.cobbs at gmail.com Tue Dec 6 16:05:51 2022 From: archie.cobbs at gmail.com (Archie Cobbs) Date: Tue, 6 Dec 2022 10:05:51 -0600 Subject: Different handing of "this" vs. "MyClass.this" in definite assignment In-Reply-To: <19c82579-2c07-e30b-3308-4596533ac0b6@oracle.com> References: <6210dcf1-3ff6-4735-6f54-ad12dacb7433@oracle.com> <002e37c5-e8c7-cc32-9339-62920b07d59f@oracle.com> <8420808A-EC21-48E1-B2BF-C4FC2CCD1CA0@oracle.com> <6aad98b1-80dd-fd86-6483-a67961de0f7e@oracle.com> <19c82579-2c07-e30b-3308-4596533ac0b6@oracle.com> Message-ID: On Tue, Dec 6, 2022 at 9:49 AM Brian Goetz wrote: > Are there some better warnings you could see coming out of this > exploration? > Great idea. What about simply warning any time "MyClass.this" is used in a situation where it means the same thing as plain "this"? Might make for a nice "lint" warning: MyClass.java:4: warning: [this-qual] unnecessary qualification of 'this' MyClass.this.x = 12; ^ -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Wed Dec 7 18:55:22 2022 From: davidalayachew at gmail.com (David Alayachew) Date: Wed, 7 Dec 2022 13:55:22 -0500 Subject: Finding the right keyword for new features (was "Re: Could we change the syntax for when clauses?") In-Reply-To: References: Message-ID: I have one more question I'd like to ask. Aside from this, I think we've explored the idea well enough, so I'll leave the topic alone for good. Are we opposed to doing something like this? case Integer i #when i > 0 My goal here is to use the # symbol (or any other unused symbol) as sort of a springboard into a larger pool of potential signifiers. You get simplicity and expressiveness for the cost of one more character that is currently unused (to my knowledge). Furthermore, the introduction of the unused character can help with visually separating the pattern from the guard, but with minimal loudness. And of course, being unused means that it meets the needs for Java language parsing folks too. Thank you for your time and help! David Alayachew -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed Dec 7 19:04:31 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 7 Dec 2022 14:04:31 -0500 Subject: Finding the right keyword for new features (was "Re: Could we change the syntax for when clauses?") In-Reply-To: References: Message-ID: Imagine we did this, and we added a dozen or so new #keywords to the language over the next ten years.? Now imagine you're a new developer learning Java. In addition to all the struggling with learning to program and learning the language, you're now also guaranteed to be distracted with "how do I remember whether it's "if" or "#if, and why can't the stupid compiler just figure it out for me?". And the explanation "Oh, that was added after the Great Sharpification" will not be compelling. As a concrete example, this was tried in Java 5 with "@interface".? To this day, people complain about this, and with good reason: it is a gratuitous departure from the rest of the language. We should strive to make the features we add look like they were there all along. On 12/7/2022 1:55 PM, David Alayachew wrote: > I have one more question I'd like to ask. Aside from this, I think > we've explored the idea well enough, so I'll leave the topic alone for > good. > > Are we opposed to doing something like this? > > case Integer i #when i > 0 > > My goal here is to use the # symbol (or any other unused symbol) as > sort of a springboard into a larger pool of potential signifiers. You > get simplicity and expressiveness for the cost of one more character > that is currently unused (to my knowledge). Furthermore, the > introduction of the unused character can help with visually separating > the pattern from the guard, but with minimal loudness. And of course, > being unused means that it meets the needs for Java language parsing > folks too. > > Thank you for your time and help! > David Alayachew -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Wed Dec 7 19:22:22 2022 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 7 Dec 2022 20:22:22 +0100 (CET) Subject: Finding the right keyword for new features (was "Re: Could we change the syntax for when clauses?") In-Reply-To: References: Message-ID: <656397664.61863749.1670440942365.JavaMail.zimbra@u-pem.fr> > From: "Brian Goetz" > To: "David Alayachew" > Cc: "amber-dev" > Sent: Wednesday, December 7, 2022 8:04:31 PM > Subject: Re: Finding the right keyword for new features (was "Re: Could we > change the syntax for when clauses?") > Imagine we did this, and we added a dozen or so new #keywords to the language > over the next ten years. Now imagine you're a new developer learning Java. In > addition to all the struggling with learning to program and learning the > language, you're now also guaranteed to be distracted with "how do I remember > whether it's "if" or "#if, and why can't the stupid compiler just figure it out > for me?". And the explanation "Oh, that was added after the Great > Sharpification" will not be compelling. > As a concrete example, this was tried in Java 5 with "@interface". To this day, > people complain about this, and with good reason: it is a gratuitous departure > from the rest of the language. > We should strive to make the features we add look like they were there all > along. #yes, #It's #an #effect #that #we #can #already #see #with #the #APIs, #every #year, #i've #students #asking #me #why Map.get() #does #not #return #an Optional #or #why java.util.RandomAccess #is #not #an #annotation. #R?mi #PS: #to #make #my #answer #more #simple #and #expressive, #I've #prefixed #every #words #which #is #not #a #Java #class #or a #method #by #the #sign ##. > On 12/7/2022 1:55 PM, David Alayachew wrote: >> I have one more question I'd like to ask. Aside from this, I think we've >> explored the idea well enough, so I'll leave the topic alone for good. >> Are we opposed to doing something like this? >> case Integer i #when i > 0 >> My goal here is to use the # symbol (or any other unused symbol) as sort of a >> springboard into a larger pool of potential signifiers. You get simplicity and >> expressiveness for the cost of one more character that is currently unused (to >> my knowledge). Furthermore, the introduction of the unused character can help >> with visually separating the pattern from the guard, but with minimal loudness. >> And of course, being unused means that it meets the needs for Java language >> parsing folks too. >> Thank you for your time and help! >> David Alayachew -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidalayachew at gmail.com Wed Dec 7 19:47:23 2022 From: davidalayachew at gmail.com (David Alayachew) Date: Wed, 7 Dec 2022 14:47:23 -0500 Subject: Finding the right keyword for new features (was "Re: Could we change the syntax for when clauses?") In-Reply-To: <656397664.61863749.1670440942365.JavaMail.zimbra@u-pem.fr> References: <656397664.61863749.1670440942365.JavaMail.zimbra@u-pem.fr> Message-ID: Makes a lot of sense Brian, ty. And I can certainly agree with the @interface. Flexible as it is, it's one of the more confusing parts of the language to interact with. > Imagine you're a new developer learning Java. > We should strive to make the features we add > look like they were there all along. I definitely hear you now. You're saying that new language features should be as seamless as possible because that cohesiveness facilitates learning for beginners and experienced devs alike. ~ > #PS: #to #make #my #answer #more #simple > #and #expressive, #I've #prefixed #every > #words #which #is #not #a #Java #class > #or a #method #by #the #sign ## Lol, point taken #R?mi. Both of you are very much correct. I see the flaw in the logic and how it is certainly not a good fit for the language. I'll try and think my suggestions through more in the future. But I must say, your code does not compile. :P > RemiSnark.java:3: error: '#' expected > #or a #method #by #the #sign ## > ^ > 1 error Thank you both for your time and help! David Alayachew -------------- next part -------------- An HTML attachment was scrubbed... URL: From claytonwohl at gmail.com Fri Dec 9 20:57:54 2022 From: claytonwohl at gmail.com (Clayton Wohl) Date: Fri, 9 Dec 2022 14:57:54 -0600 Subject: Record Pattern Bug in Java 19 In-Reply-To: <8718c980-8f83-ec30-f208-c26a7e1bc240@oracle.com> References: <8718c980-8f83-ec30-f208-c26a7e1bc240@oracle.com> Message-ID: This is fixed in Java 20 build 27, 2022-12-09. Awesome! Thank you! On Wed, Jul 27, 2022 at 3:04 PM Brian Goetz wrote: > Thanks for the bug report! This is the right place for bug reports like > this. > > I believe this is the same as a bug we're currently working on; the way > we've specified exhaustiveness is almost, but not quite, right with respect > to generics. Stay tuned for an update. > > On 7/27/2022 2:26 PM, Clayton Wohl wrote: > > // The example given in Brian Goetz's article: > https://www.infoq.com/articles/data-oriented-programming-java/ > sealed interface Opt { > record Some(T value) implements Opt { } > record None() implements Opt { } > } > > // This works: Exhaustive switch without default case, but no record > pattern > public static void thisWorks1(int value) { > Opt optValue = doSomethingThatReturnsOpt(value); > switch (optValue) { > case Opt.Some some -> System.out.printf("got string: > %s%n", some.value()); > case Opt.None none -> System.out.println("got none"); > }; > } > > // This works: record pattern in a switch statement with a default case. > public static void thisWorks2(int value) { > Opt optValue = doSomethingThatReturnsOpt(value); > switch (optValue) { > case Opt.Some(String v) -> System.out.printf("got string: > %s%n", v); > case Opt.None none -> System.out.println("got none"); > default -> System.out.printf("default%n"); > }; > } > > // This does NOT compile: Exhaustive switch without default case + record > pattern > public static void thisDoesNotWork(int value) { > Opt optValue = doSomethingThatReturnsOpt(value); > switch (optValue) { > case Opt.Some(String v) -> System.out.printf("got string: > %s%n", v); > case Opt.None none -> System.out.println("got none"); > }; > } > > This is with the latest public JDK 19 build: build 19-ea+32-2220 > > I hope I'm posting to the correct list. If this list is for internal Java > developers only, I'm sorry. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From archie.cobbs at gmail.com Thu Dec 15 17:00:25 2022 From: archie.cobbs at gmail.com (Archie Cobbs) Date: Thu, 15 Dec 2022 11:00:25 -0600 Subject: Loosening requirements for super() invocation In-Reply-To: References: <140e133e-4c46-5d83-1ccf-32e40406a13a@oracle.com> <362d1d7b-97fe-dc2e-de02-1d0169bd2626@oracle.com> <75cc216c-12c1-d7bd-37f1-c73d4fc97143@oracle.com> <7242cdba-0b7d-2520-b159-b13f91113c18@oracle.com> <347917af-4663-6fa8-a6b3-6645a3e5f5ce@oracle.com> Message-ID: On Wed, Nov 9, 2022 at 12:06 PM Archie Cobbs wrote: > I'll try to come up with a concrete but simple starting point. > It's taken me a little while, but I finally have a prototype for 'this' escape warnings ready to play with. To review this discussion... Java suffers from a 'this' escape problem, which is when a constructor invokes a method that could be overridden in a subclass, in which case the method will execute before the subclass constructor has finished initializing the instance, possibly leading to chaos, or at least unexpected and difficult to debug behavior. Two related proposals are on the table to help address this: *1. JDK-8194743 Permit additional statements before this/super in constructors* If a subclass is allowed to do initialization prior to invoking super(), the possible damage from a superclass 'this' escape can be reduced or eliminated. A prototype is implemented and described in detail here . Of course this change would also require a JLS spec change. *2. Adding a compiler warning to help reveal situations where a 'this' escape can happen.* This is now prototyped here . The main logic is encapsulated in this class: ThisEscapeAnalyzer.java . Some notes on this analysis: - We "execute" constructors and track where the 'this' reference goes as the constructor executes. - We use a very simplified flow analysis that you might call a "flood analysis", where the union of every possible code branch is taken. - A "leak" is defined as the possible passing of a subclassed 'this' reference to code defined outside of the current compilation unit. - In other words, we don't try to protect the current compilation unit from itself. - We can ignore private constructors - they can never be used by external subclasses, etc. - If a constructor invokes a method defined in the same compilation unit, and that method cannot be overridden, then our analysis can safely recurse into the method. - When this occurs the warning displays each step in the stack trace to help in comprehension - The possible locations for a 'this' reference that we try to track are: - Current 'this' - Current outer 'this' - Local parameter/variable - Method return value - Current expression value (i.e. top of stack) - We don't try to track assignments to & from fields (for future study). - We don't try to follow super() invocations - We categorize tracked references as direct or indirect to add a tiny bit of nuance. Playing around with this, it seems to work pretty well. As you might expect, when you turn this on with existing code like java.base, you can get a lot of warnings. One fear with adding this warning is that developers who like to use -Xlint:all by default will suddenly see a zillion new warnings and freak out. So it's probably worth understanding how likely this warning is to fire with existing code out there. As one rough data point, when I build OpenJDK's 71 total modules, 33 of them have a 'this' escape somewhere and require me to add an exclusion for 'this-escape' from the -Xlint:all to compile again. So that's about 50% hit rate... maybe not so bad considering how much code this represents. At least the warnings that I've looked at so far are all indeed true positives, in the sense that the constructor does in fact pass a 'this' reference to a method that a subclass, someday, somewhere, could override. Mostly it's one of two cases: the constructor invokes an overridable instance method, or the constructor invokes a method with 'this' as a parameter (e.g. registering a listener). Some of the warnings come from package-private classes. This is consistent with the current design, but it brings up the notion that one could draw the "external" line differently... i.e., instead of at the compilation unit boundary, at the Java package boundary, or even the Java module boundary. There are some potential refinements, e.g., noticing when all permitted subclasses of a sealed class are in the same compilation unit. Any thoughts or comments greatly appreciated. -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From hjohn at xs4all.nl Thu Dec 22 12:37:29 2022 From: hjohn at xs4all.nl (John Hendrikx) Date: Thu, 22 Dec 2022 12:37:29 +0000 Subject: Transient fields in records Message-ID: Hi, I have tried googling this topic and looking through the amber-dev and amber-spec-experts archives, but was unable to locate anything that mentioned transient fields in combination with records. The JEP itself also makes no mention of transient fields. The word transient seems to never be mentioned in combination with records anywhere... I often enough design classes for serialization in some fashion or other. One of the tools that one can use here is to use transient fields to store information that can be completely derived from the other fields. These are used to store information that may be too large to store or too expensive to (re)calculate on every access. All major frameworks that do serialization recognize such fields and act accordingly (Java serialization, Jackson, Hibernate). However, records do not allow such fields, even though I think one of their primary use cases is data storage, which often will sooner or later involve serialization. Now I can't imagine this was an oversight, so I'm curious to know if transient fields were simply left out for now, or that they are not allowed for other reasons. Javac simply complains: TransientRecord.java:4: error: field declaration must be static Here is the use case I had when I encountered this problem: public record Message( A a, B b, C c, D d) { public enum Type { A, B, C, D; } private final transient List types; // <<< not allowed public Message { List types = new ArrayList<>(); if(a != null) { types.add(Type.A); } if(b != null) { types.add(Type.B); } if(c != null) { types.add(Type.C); } if(d != null) { types.add(Type.D); } this.types = Collections.unmodifiableList(types); } public List getTypes() { return types; // use transient field to avoid recalculation } } --John -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Thu Dec 22 14:25:45 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 22 Dec 2022 14:25:45 +0000 Subject: Transient fields in records In-Reply-To: References: Message-ID: <356374D9-C39C-4E61-9B36-FE96D2D804CB@oracle.com> The problem you are confronting is not that the record can?t have transient fields; it can?t have fields *at all* that are not part of the state description. A record is the state, the whole state, and nothing but the state. On Dec 22, 2022, at 7:37 AM, John Hendrikx > wrote: Hi, I have tried googling this topic and looking through the amber-dev and amber-spec-experts archives, but was unable to locate anything that mentioned transient fields in combination with records. The JEP itself also makes no mention of transient fields. The word transient seems to never be mentioned in combination with records anywhere... I often enough design classes for serialization in some fashion or other. One of the tools that one can use here is to use transient fields to store information that can be completely derived from the other fields. These are used to store information that may be too large to store or too expensive to (re)calculate on every access. All major frameworks that do serialization recognize such fields and act accordingly (Java serialization, Jackson, Hibernate). However, records do not allow such fields, even though I think one of their primary use cases is data storage, which often will sooner or later involve serialization. Now I can't imagine this was an oversight, so I'm curious to know if transient fields were simply left out for now, or that they are not allowed for other reasons. Javac simply complains: TransientRecord.java:4: error: field declaration must be static Here is the use case I had when I encountered this problem: public record Message( A a, B b, C c, D d) { public enum Type { A, B, C, D; } private final transient List types; // <<< not allowed public Message { List types = new ArrayList<>(); if(a != null) { types.add(Type.A); } if(b != null) { types.add(Type.B); } if(c != null) { types.add(Type.C); } if(d != null) { types.add(Type.D); } this.types = Collections.unmodifiableList(types); } public List getTypes() { return types; // use transient field to avoid recalculation } } --John -------------- next part -------------- An HTML attachment was scrubbed... URL: From redio.development at gmail.com Tue Dec 27 13:49:13 2022 From: redio.development at gmail.com (Red IO) Date: Tue, 27 Dec 2022 14:49:13 +0100 Subject: Function level try recourse directives. Message-ID: I was once again writing closing heavy code in a java method and was wondering if I could reduce the clutter of the multi line try block: try (var foo1 = new Foo1() ; var foo2 = new Foo2() ; ....) { ... } Sometimes we need to initialize something between the resources and we get multiple levels of nesting. And I was wondering if we could not just lose the block and auto close at the end of a method like this : void foo() { try (var foo1 = new Foo1()); //init some stuff try (var foo2 = new Foo2()); // do some stuff // auto close at end of method } I think method level try recourse would improve readability and unnecessary nesting significantly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From org.openjdk at io7m.com Tue Dec 27 14:07:26 2022 From: org.openjdk at io7m.com (Mark Raynsford) Date: Tue, 27 Dec 2022 14:07:26 +0000 Subject: Function level try recourse directives. In-Reply-To: References: Message-ID: <20221227140726.04f72129@sunflower.int.arc7.info> On 2022-12-27T14:49:13 +0100 Red IO wrote: > > Sometimes we need to initialize something between the resources and we get > multiple levels of nesting. > And I was wondering if we could not just lose the block and auto close at > the end of a method like this : I work with Vulkan a lot, and that involves a metric ton of closeable resources that also have to be closed in stack order. I have a library-level solution for this: https://www.io7m.com/software/jmulticlose/ -- Mark Raynsford | https://www.io7m.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 228 bytes Desc: OpenPGP digital signature URL: From redio.development at gmail.com Tue Dec 27 14:16:47 2022 From: redio.development at gmail.com (Red IO) Date: Tue, 27 Dec 2022 15:16:47 +0100 Subject: Function level try recourse directives. In-Reply-To: <20221227140726.04f72129@sunflower.int.arc7.info> References: <20221227140726.04f72129@sunflower.int.arc7.info> Message-ID: This is a nice workaround but allocates additional resources and adds clutter in form of (still) a level of nesting and an unnecessary collection. Considering that try resource is only a syntactic sugar. It shouldn't be difficult to add this simple addition of syntax to provide this enormous ergonomic advantage. Also I think it's unlikely that try (... ); would find another useful use. On Tue, Dec 27, 2022, 15:07 Mark Raynsford wrote: > On 2022-12-27T14:49:13 +0100 > Red IO wrote: > > > > Sometimes we need to initialize something between the resources and we > get > > multiple levels of nesting. > > And I was wondering if we could not just lose the block and auto close at > > the end of a method like this : > > I work with Vulkan a lot, and that involves a metric ton of closeable > resources that also have to be closed in stack order. I have a > library-level solution for this: > > https://www.io7m.com/software/jmulticlose/ > > -- > Mark Raynsford | https://www.io7m.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Tue Dec 27 16:25:40 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 27 Dec 2022 16:25:40 +0000 Subject: Function level try recourse directives. In-Reply-To: References: <20221227140726.04f72129@sunflower.int.arc7.info> Message-ID: I think Mark?s solution is quite practical. I think you?re greatly overestimating the benefits (?enormous ergonomic advantage?) of your idea (a common trap to fall into) and enormously^2 underestimating where the bar is for ?add features to everyone?s Java.? On Dec 27, 2022, at 9:16 AM, Red IO > wrote: This is a nice workaround but allocates additional resources and adds clutter in form of (still) a level of nesting and an unnecessary collection. Considering that try resource is only a syntactic sugar. It shouldn't be difficult to add this simple addition of syntax to provide this enormous ergonomic advantage. Also I think it's unlikely that try (... ); would find another useful use. On Tue, Dec 27, 2022, 15:07 Mark Raynsford > wrote: On 2022-12-27T14:49:13 +0100 Red IO > wrote: > > Sometimes we need to initialize something between the resources and we get > multiple levels of nesting. > And I was wondering if we could not just lose the block and auto close at > the end of a method like this : I work with Vulkan a lot, and that involves a metric ton of closeable resources that also have to be closed in stack order. I have a library-level solution for this: https://www.io7m.com/software/jmulticlose/ -- Mark Raynsford | https://www.io7m.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From numeralnathan at gmail.com Tue Dec 27 17:42:53 2022 From: numeralnathan at gmail.com (Nathan Reynolds) Date: Tue, 27 Dec 2022 09:42:53 -0800 Subject: Function level try recourse directives. In-Reply-To: References: <20221227140726.04f72129@sunflower.int.arc7.info> Message-ID: If the CloseableCollection doesn't escape (likely), then JIT will get rid of the object allocation. The internal Closeable[] (?) may still stick around... unless JIT can determine it doesn't escape too. Can JIT escape analysis get rid of small arrays? If so, the CloseableCollection's footprint disappears. The only thing remaining is the logic. That's exactly what an enhanced try-with-resources would do. So, instead of putting effort into a risky Java language change, I world suggest we put the effort into JIT to get rid of the allocations. This will benefit not only CloseableCollection but a whole lot more. Switching topics... Please do a code analysis of where multiple try with resources are used. Figure out the typical number of resources that are closed. Then, set the default size of the internal array (?) to that size. The default size may be too large and many elements in the array will be null. By setting the default size smaller, there will be less unused space allocated and hence GC won't happen as often. On Tue, Dec 27, 2022, 8:25 AM Brian Goetz wrote: > I think Mark?s solution is quite practical. I think you?re greatly > overestimating the benefits (?enormous ergonomic advantage?) of your idea > (a common trap to fall into) and enormously^2 underestimating where the bar > is for ?add features to everyone?s Java.? > > > > On Dec 27, 2022, at 9:16 AM, Red IO wrote: > > This is a nice workaround but allocates additional resources and adds > clutter in form of (still) a level of nesting and an unnecessary > collection. Considering that try resource is only a syntactic sugar. It > shouldn't be difficult to add this simple addition of syntax to provide > this enormous ergonomic advantage. Also I think it's unlikely that try (... > ); would find another useful use. > > On Tue, Dec 27, 2022, 15:07 Mark Raynsford wrote: > >> On 2022-12-27T14:49:13 +0100 >> Red IO wrote: >> > >> > Sometimes we need to initialize something between the resources and we >> get >> > multiple levels of nesting. >> > And I was wondering if we could not just lose the block and auto close >> at >> > the end of a method like this : >> >> I work with Vulkan a lot, and that involves a metric ton of closeable >> resources that also have to be closed in stack order. I have a >> library-level solution for this: >> >> https://www.io7m.com/software/jmulticlose/ >> >> -- >> Mark Raynsford | https://www.io7m.com >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From org.openjdk at io7m.com Tue Dec 27 18:53:17 2022 From: org.openjdk at io7m.com (Mark Raynsford) Date: Tue, 27 Dec 2022 18:53:17 +0000 Subject: Function level try recourse directives. In-Reply-To: References: <20221227140726.04f72129@sunflower.int.arc7.info> Message-ID: <20221227185317.4acbd1c5@sunflower.int.arc7.info> On 2022-12-27T09:42:53 -0800 Nathan Reynolds wrote: > If the CloseableCollection doesn't escape (likely), then JIT will get rid > of the object allocation. The internal Closeable[] (?) may still stick > around... unless JIT can determine it doesn't escape too. Can JIT escape > analysis get rid of small arrays? If so, the CloseableCollection's > footprint disappears. The only thing remaining is the logic. That's > exactly what an enhanced try-with-resources would do. I haven't been able to spot any instances of the closeable collection in heap samples, or the internal ArrayDeque instances. Doing soft-realtime work, you can be certain I'm looking at heap samples. :) It hasn't even been necessary so far to use anything other than the default array size. Those instances are either gone in a single method, or they live for a very long time (and we're talking about less than a couple of hundred bytes per collection, typically). -- Mark Raynsford | https://www.io7m.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 228 bytes Desc: OpenPGP digital signature URL: From reinier at zwitserloot.com Tue Dec 27 19:15:02 2022 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 27 Dec 2022 11:15:02 -0800 Subject: Function level try recourse directives. In-Reply-To: References: Message-ID: Lombok?s @Cleanup (https://projectlombok.org/features/Cleanup) works exactly like that: @Cleanup var foo1 = new Foo1(); @Cleanup var foo2 = new Foo2(); Does what you want. However, it runs to the end of the scope that foo1 is defined in. An exotic java feature is that you can just write braces whenever you feel like, so you can use that to scope your resources as desired: void example() { @Cleanup var db = ...; // scoped for the method, so closed at the end String acctName; { // note, just an opening brace here, on its own. Legal. @Cleanup var rs = db.query(...); acctName = rs.getString(1); } System.out.println(acctName); } It feels like the road not taken: That looks fine, is as flexible as you need it to be (in that you control exactly where the resource should be closed) and works great in daily use. It?s just? not what the try-with-resources construct ended up doing. It feels a little odd to add it *now*, and ?just write some braces to scope some variables? is not a thing your average java programmer ever does. I?m not quite sure why not (it?s quite useful given that java doesn?t make it easy to put helper methods inside methods), but, it is what it is: Introducing this feature more or less demands of the community that they get into the habit of writing braces solely to define a scope. This is a big ordeal; for example, most autoformatters will make this unduly ugly by forcibly trying to let the opening brace start on its own line (sometimes, it?s own entire paragraph, i.e. 2 newlines are injected), even though often it is more convenient and easier to read to open it *after* the relevant variable declaration, or a comment or label explaining what the point of this block is. A casual analysis indicates that going all out leads to nice results and anything in between would incur significant costs. In other words, we should have *either* a java where everybody is used to using braces to create scoped blocks and the *only* variant of try-with is the stand-alone one as you propose - *or* we have the java we have today, where almost nobody knows it?s legal to just write braces to create a scoped block, auto-formatters don?t know what to do with them, and try-with comes with its own mandatory block. Given that at this juncture it?s not possible to travel back in time and get rid of the existing try-with (with the mandatory block), I?d say: This idea doesn?t ?work? for java. And that?s coming from someone who is so convinced your way is better, he programmed it into lombok about a year before try-with was introduced :) In other words, better is great, but ?does not demand significant cultural adjustments and does not lead to endless style wars? is better. The costs incurred are high cultural friction, style wars, and likely, that this alternate ?form? of try-with will be so rarely used, it will end up being a member of that list of ?java lang features most programmers don?t *ever* use and cannot recognize?. It?s not a good idea for that list to grow any longer than it already is. --Reinier Zwitserloot On 27 Dec 2022 at 14:49:13, Red IO wrote: > I was once again writing closing heavy code in a java method and was > wondering if I could reduce the clutter of the multi line try block: > try (var foo1 = new Foo1() ; > var foo2 = new Foo2() ; > ....) { > ... > } > Sometimes we need to initialize something between the resources and we get > multiple levels of nesting. > And I was wondering if we could not just lose the block and auto close at > the end of a method like this : > > void foo() { > try (var foo1 = new Foo1()); > //init some stuff > try (var foo2 = new Foo2()); > // do some stuff > // auto close at end of method > } > > I think method level try recourse would improve readability and > unnecessary nesting significantly. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From redio.development at gmail.com Wed Dec 28 10:43:50 2022 From: redio.development at gmail.com (Red IO) Date: Wed, 28 Dec 2022 11:43:50 +0100 Subject: Function level try recourse directives. In-Reply-To: References: Message-ID: I think there where many valid points against it mentioned. But I disagree with your scope thesis. The "open try resource" is not meant to be used like this: void foo() { .. { try (..); .. } .. } Which is what I understand you mean by opening a scope just for the scope itself. It is meant primarily for use cases where your resource live the lifetime of the function, which is pretty common in well refractored code. Also adding the scope defeats the idea of the feature entirely. You could write: void foo() { .. try (..) { .. } .. } Which is exactly what the current existing try resource is. The motivation of feature is to remove redundant nesting. You often encounter situations like this: void foo() { try (..) { //the entire code of the function } } In this example the layer of nesting the try resource introduces is totally unnecessary since the scope is the same as the function scope. I think you underestimate the occurrences of the code above. Also with closable resources increasing in numbers the amount of nesting in Java code will be greatly increased. My estimate is that the scopeless try resource is as often applicable in Java as the scopeless if is. I proposed adding this feature not only because I think it's a great feature in general but also since it already has a track record of being useful and used by a community very similar to the java community. C# has the exact same feature as java's try resource just with a different keyword used (using). It has the scopeless version and if you look at some c# code that uses closable resources the scopeless variant is used 70+% of the time. It removes a lot of redundant scoping while not really adding much complexity nor using syntax that is likely to be used by some other feature in the future. I'm not sure how difficult the addition of a feature like this is since I still have no idea how changes in syntax are done in Java by any means. But if it's not to difficult to implement why not try it and ask the community what they think. I think asking the community whether they would use it or not is way more effective than speculating how popular it might be. It wouldn't even have to be implemented to ask the community what they think. I don't have the audience to make such surveys by any means. Otherwise I would do a post, video or poll myself. On Tue, Dec 27, 2022, 20:15 Reinier Zwitserloot wrote: > Lombok?s @Cleanup (https://projectlombok.org/features/Cleanup) works > exactly like that: > > @Cleanup var foo1 = new Foo1(); > @Cleanup var foo2 = new Foo2(); > > > Does what you want. However, it runs to the end of the scope that foo1 is > defined in. An exotic java feature is that you can just write braces > whenever you feel like, so you can use that to scope your resources as > desired: > > void example() { > @Cleanup var db = ...; // scoped for the method, so closed at the end > > String acctName; { // note, just an opening brace here, on its own. Legal. > @Cleanup var rs = db.query(...); > acctName = rs.getString(1); > } > > System.out.println(acctName); > } > > > It feels like the road not taken: That looks fine, is as flexible as you > need it to be (in that you control exactly where the resource should be > closed) and works great in daily use. It?s just? not what the > try-with-resources construct ended up doing. It feels a little odd to add > it *now*, and ?just write some braces to scope some variables? is not a > thing your average java programmer ever does. I?m not quite sure why not > (it?s quite useful given that java doesn?t make it easy to put helper > methods inside methods), but, it is what it is: Introducing this feature > more or less demands of the community that they get into the habit of > writing braces solely to define a scope. This is a big ordeal; for example, > most autoformatters will make this unduly ugly by forcibly trying to let > the opening brace start on its own line (sometimes, it?s own entire > paragraph, i.e. 2 newlines are injected), even though often it is more > convenient and easier to read to open it *after* the relevant variable > declaration, or a comment or label explaining what the point of this block > is. > > A casual analysis indicates that going all out leads to nice results and > anything in between would incur significant costs. In other words, we > should have *either* a java where everybody is used to using braces to > create scoped blocks and the *only* variant of try-with is the > stand-alone one as you propose - *or* we have the java we have today, > where almost nobody knows it?s legal to just write braces to create a > scoped block, auto-formatters don?t know what to do with them, and try-with > comes with its own mandatory block. > > Given that at this juncture it?s not possible to travel back in time and > get rid of the existing try-with (with the mandatory block), I?d say: This > idea doesn?t ?work? for java. And that?s coming from someone who is so > convinced your way is better, he programmed it into lombok about a year > before try-with was introduced :) > > In other words, better is great, but ?does not demand significant cultural > adjustments and does not lead to endless style wars? is better. > > The costs incurred are high cultural friction, style wars, and likely, > that this alternate ?form? of try-with will be so rarely used, it will end > up being a member of that list of ?java lang features most programmers > don?t *ever* use and cannot recognize?. It?s not a good idea for that > list to grow any longer than it already is. > > --Reinier Zwitserloot > > > On 27 Dec 2022 at 14:49:13, Red IO wrote: > >> I was once again writing closing heavy code in a java method and was >> wondering if I could reduce the clutter of the multi line try block: >> try (var foo1 = new Foo1() ; >> var foo2 = new Foo2() ; >> ....) { >> ... >> } >> Sometimes we need to initialize something between the resources and we >> get multiple levels of nesting. >> And I was wondering if we could not just lose the block and auto close at >> the end of a method like this : >> >> void foo() { >> try (var foo1 = new Foo1()); >> //init some stuff >> try (var foo2 = new Foo2()); >> // do some stuff >> // auto close at end of method >> } >> >> I think method level try recourse would improve readability and >> unnecessary nesting significantly. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Wed Dec 28 17:06:51 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 28 Dec 2022 17:06:51 +0000 Subject: Function level try recourse directives. In-Reply-To: References: Message-ID: <474079CC-E4DF-4111-927E-02233CFBB47C@oracle.com> > I think there where many valid points against it mentioned. But I disagree with your scope thesis. The "open try resource" is not meant to be used like this: > void foo() { > .. > { > try (..); > .. > } > .. > } Which is what I understand you mean by opening a scope just for the scope itself. It is meant primarily for use cases where your resource live the lifetime of the function, which is pretty common in well refractored code. What you are advocating for is something like Golang?s `defer` statement, which queues up cleanup operations to run at function exit. But, the `defer` mechanism is a weak bit of language design; it forces developers to align method boundaries with resource management boundaries, even when the two don?t agree. Yes, the two coincide often enough to make it seem harmless and even clever, but such couplings often end up being impediments to refactoring and the source of subtle bugs. (For example, an ?extract method? refactor may change the order in which resources are released.). Ultimately, it has many of the same deficiencies as global static state or nonlocal control flow ? it interferes with modular reasoning. Adding more of this into the language isn?t necessarily helpful. > Which is exactly what the current existing try resource is. The motivation of feature is to remove redundant nesting. I think this is the root of the problem. This is not a very strong motivation for introducing a new and different form of scoping into the language. From brian.goetz at oracle.com Wed Dec 28 17:16:46 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 28 Dec 2022 17:16:46 +0000 Subject: Function level try recourse directives. In-Reply-To: <474079CC-E4DF-4111-927E-02233CFBB47C@oracle.com> References: <474079CC-E4DF-4111-927E-02233CFBB47C@oracle.com> Message-ID: <5DB9800A-B1B1-4A80-ABFD-0925F5994383@oracle.com> > On Dec 28, 2022, at 12:06 PM, Brian Goetz wrote: > >> I think there where many valid points against it mentioned. But I disagree with your scope thesis. The "open try resource" is not meant to be used like this: >> void foo() { >> .. >> { >> try (..); >> .. >> } >> .. >> } > > Which is what I understand you mean by opening a scope just for the scope itself. It is meant primarily for use cases where your resource live the lifetime of the function, which is pretty common in well refractored code. There was some sort of editing error here, the above sentence was part of Red?s mail, not my reply. My reply begins here: > What you are advocating for is something like Golang?s `defer` statement, which queues up cleanup operations to run at function exit. > > But, the `defer` mechanism is a weak bit of language design; it forces developers to align method boundaries with resource management boundaries, even when the two don?t agree. Yes, the two coincide often enough to make it seem harmless and even clever, but such couplings often end up being impediments to refactoring and the source of subtle bugs. (For example, an ?extract method? refactor may change the order in which resources are released.). Ultimately, it has many of the same deficiencies as global static state or nonlocal control flow ? it interferes with modular reasoning. Adding more of this into the language isn?t necessarily helpful. > > >> Which is exactly what the current existing try resource is. The motivation of feature is to remove redundant nesting. > > I think this is the root of the problem. This is not a very strong motivation for introducing a new and different form of scoping into the language. > > From numeralnathan at gmail.com Wed Dec 28 20:15:03 2022 From: numeralnathan at gmail.com (Nathan Reynolds) Date: Wed, 28 Dec 2022 12:15:03 -0800 Subject: Function level try recourse directives. In-Reply-To: <20221227185317.4acbd1c5@sunflower.int.arc7.info> References: <20221227140726.04f72129@sunflower.int.arc7.info> <20221227185317.4acbd1c5@sunflower.int.arc7.info> Message-ID: CloseableCollection is a class of the library. It won't show up in heap dumps because probably very few people are using it. You would need to look at JIT optimized assembly to see if JIT is eliding the CloseableCollection and it's internal data structure (array?). On Tue, Dec 27, 2022 at 10:53 AM Mark Raynsford wrote: > On 2022-12-27T09:42:53 -0800 > Nathan Reynolds wrote: > > > If the CloseableCollection doesn't escape (likely), then JIT will get rid > > of the object allocation. The internal Closeable[] (?) may still stick > > around... unless JIT can determine it doesn't escape too. Can JIT escape > > analysis get rid of small arrays? If so, the CloseableCollection's > > footprint disappears. The only thing remaining is the logic. That's > > exactly what an enhanced try-with-resources would do. > > I haven't been able to spot any instances of the closeable collection > in heap samples, or the internal ArrayDeque instances. Doing > soft-realtime work, you can be certain I'm looking at heap samples. :) > > It hasn't even been necessary so far to use anything other than the > default array size. Those instances are either gone in a single method, > or they live for a very long time (and we're talking about less than a > couple of hundred bytes per collection, typically). > > -- > Mark Raynsford | https://www.io7m.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From archie.cobbs at gmail.com Wed Dec 28 22:31:28 2022 From: archie.cobbs at gmail.com (Archie Cobbs) Date: Wed, 28 Dec 2022 16:31:28 -0600 Subject: Loosening requirements for super() invocation In-Reply-To: References: <140e133e-4c46-5d83-1ccf-32e40406a13a@oracle.com> <362d1d7b-97fe-dc2e-de02-1d0169bd2626@oracle.com> <75cc216c-12c1-d7bd-37f1-c73d4fc97143@oracle.com> <7242cdba-0b7d-2520-b159-b13f91113c18@oracle.com> <347917af-4663-6fa8-a6b3-6645a3e5f5ce@oracle.com> Message-ID: Following up on the discussion below with some more concrete data... On Thu, Dec 15, 2022 at 11:00 AM Archie Cobbs wrote: > On Wed, Nov 9, 2022 at 12:06 PM Archie Cobbs > wrote: > >> I'll try to come up with a concrete but simple starting point. >> > > It's taken me a little while, but I finally have a prototype for 'this' > escape warnings ready to play with. > Here are some more precise stats on what happens when you enable the 'this' escape warning for the openjdk build. The table below shows the number of new 'this' escape warnings generated, and the total number of Java files, in each module: #WARNINGS #FILES MODULE 161 3168 java.base 0 127 java.compiler 0 18 java.datatransfer 62 1848 java.xml 0 18 java.prefs 160 2740 java.desktop 0 10 java.instrument 0 35 java.logging 25 330 java.management 0 30 java.security.sasl 14 106 java.rmi 5 199 java.naming 0 16 java.management.rmi 21 142 java.net.http 0 15 java.scripting 0 5 java.transaction.xa 0 212 java.security.jgss 0 271 java.xml.crypto 0 77 java.sql 7 56 java.sql.rowset 0 1 java.se 0 22 java.smartcardio 3 18 jdk.accessibility 0 60 jdk.internal.jvmstat 0 15 jdk.attach 0 147 jdk.charsets 2 14 jdk.zipfs 55 355 jdk.compiler 0 35 jdk.crypto.ec 0 76 jdk.crypto.cryptoki 4 68 jdk.dynalink 0 3 jdk.internal.ed 0 3 jdk.editpad 19 940 jdk.hotspot.agent 7 59 jdk.httpserver 0 5 jdk.incubator.concurrent 0 50 jdk.incubator.vector 3 104 jdk.internal.le 0 52 jdk.internal.opt 11 209 jdk.internal.vm.ci 0 1 jdk.internal.vm.compiler 0 1 jdk.internal.vm.compiler.management 0 30 jdk.jartool 5 245 jdk.javadoc 0 40 jdk.jcmd 0 24 jdk.management 0 30 jdk.management.agent 30 64 jdk.jconsole 10 139 jdk.jdeps 0 1 jdk.jdwp.agent 24 254 jdk.jdi 0 263 jdk.jfr 0 89 jdk.jlink 0 56 jdk.jpackage 6 97 jdk.jshell 0 4 jdk.jsobject 3 11 jdk.jstatd 0 1873 jdk.localedata 0 15 jdk.management.jfr 0 16 jdk.naming.dns 0 8 jdk.naming.rmi 0 9 jdk.net 0 2 jdk.nio.mapmode 0 11 jdk.random 0 26 jdk.sctp 0 30 jdk.security.auth 0 16 jdk.security.jgss 0 9 jdk.unsupported 0 8 jdk.unsupported.desktop 0 94 jdk.xml.dom So fortunately it looks feasible to address these with @SuppressWarnings("this-escape") annotations. Adding them is probably a good thing in its own right, because they would serve as new "Heads up for possible 'this' escape" markers in the code where there were none before, especially because this gotcha can be pretty hard to spot. I'll work on refining the 'this' escape patch to use @SuppressWarnings instead of build flags, and then update this thread when that's done. -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From redio.development at gmail.com Wed Dec 28 21:27:16 2022 From: redio.development at gmail.com (Red IO) Date: Wed, 28 Dec 2022 22:27:16 +0100 Subject: Function level try recourse directives. In-Reply-To: <5DB9800A-B1B1-4A80-ABFD-0925F5994383@oracle.com> References: <474079CC-E4DF-4111-927E-02233CFBB47C@oracle.com> <5DB9800A-B1B1-4A80-ABFD-0925F5994383@oracle.com> Message-ID: I agree that it seems like a minor improvement to remove 1 level if nesting. But it removes 1 level of nesting per individualy managed recourse (taking into consideration that you can group resources in 1 try block). I would argue that it isn't really a completely new feature but an ergonomic addition to the existing of try resource. It would be handled completely by the preprocessor (or however syntactic sugar is implemented). Regarding the coupling of object and resource scope. I would argue that resource scopes naturally align with method scopes since the single responsibility of the method would be something regarding the closable resource. Regarding refactoring I don't think tools would have a hard time refactoring: void foo() { try (var foo = new Foo()) { foo.bar(); } } To: void foo() { try (var foo = new Foo()); foo.bar(); } And back. This refactoring can also often easily done by hand. Regarding methd extraction the tool would simply need to unwind (turn the lower example into the one above it). This is always a trivial refactoring since the implicit scope of the foo variable lasts from the try block to the end of the method. Even if you have multiple resources: void foo() { try (var foo = new Foo()); try (var foo2 = new Foo()); foo.bar(foo2); } It is trivially refactored to: void foo() { try (var foo = new Foo()) { try (var foo2 = new Foo()) { foo.bar(foo2); } } } If you look at this simple example alone without any logic like an if statement or a loop the amount of nesting really makes things hard to read. In contrary the version with the scopeless try resource still highlights the special treatment of the object while looking similar to a normal object creation the whole thing really is. The implementation effort and risk of the feature is low while the amount of clarity it adds is high. There is also a data that shows the impact of the feature since it already exists in c# which is synthatically nearly equivalent to java. I also don't think it makes the language harder to learn or to understand. For people who already know the try with resources feature only need to "learn" that you can now use try resource with a method level scope. If you are new to try with resources you learn that you can automatically manage closable resources either at method level or with its own confined scope. The c# reference site of the feature: https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/using-statement On Wed, Dec 28, 2022, 18:16 Brian Goetz wrote: > > > > On Dec 28, 2022, at 12:06 PM, Brian Goetz > wrote: > > > >> I think there where many valid points against it mentioned. But I > disagree with your scope thesis. The "open try resource" is not meant to be > used like this: > >> void foo() { > >> .. > >> { > >> try (..); > >> .. > >> } > >> .. > >> } > > > > Which is what I understand you mean by opening a scope just for the > scope itself. It is meant primarily for use cases where your resource live > the lifetime of the function, which is pretty common in well refractored > code. > > There was some sort of editing error here, the above sentence was part of > Red?s mail, not my reply. My reply begins here: > > > What you are advocating for is something like Golang?s `defer` > statement, which queues up cleanup operations to run at function exit. > > > > But, the `defer` mechanism is a weak bit of language design; it forces > developers to align method boundaries with resource management boundaries, > even when the two don?t agree. Yes, the two coincide often enough to make > it seem harmless and even clever, but such couplings often end up being > impediments to refactoring and the source of subtle bugs. (For example, an > ?extract method? refactor may change the order in which resources are > released.). Ultimately, it has many of the same deficiencies as global > static state or nonlocal control flow ? it interferes with modular > reasoning. Adding more of this into the language isn?t necessarily > helpful. > > > > > >> Which is exactly what the current existing try resource is. The > motivation of feature is to remove redundant nesting. > > > > I think this is the root of the problem. This is not a very strong > motivation for introducing a new and different form of scoping into the > language. > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From org.openjdk at io7m.com Thu Dec 29 10:28:21 2022 From: org.openjdk at io7m.com (Mark Raynsford) Date: Thu, 29 Dec 2022 10:28:21 +0000 Subject: Function level try recourse directives. In-Reply-To: References: <20221227140726.04f72129@sunflower.int.arc7.info> <20221227185317.4acbd1c5@sunflower.int.arc7.info> Message-ID: <20221229102821.0cb20fed@sunflower.int.arc7.info> On 2022-12-28T12:15:03 -0800 Nathan Reynolds wrote: > CloseableCollection is a class of the library. It won't show up in heap > dumps because probably very few people are using it. You would need to > look at JIT optimized assembly to see if JIT is eliding the > CloseableCollection and it's internal data structure (array?). To be clear, I'm referring to the heap sampling feature of tools like VisualVM: https://herongyang.com/Java-Tools/jvisualvm-Heap-Dump-Class-List.jpg The CloseableCollection will show up even if there's only a single instance. The internal array of the ArrayDeque does get mixed into the samples for Object[], though. -- Mark Raynsford | https://www.io7m.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 228 bytes Desc: OpenPGP digital signature URL: From markus at headcrashing.eu Thu Dec 29 15:46:41 2022 From: markus at headcrashing.eu (Markus Karg) Date: Thu, 29 Dec 2022 16:46:41 +0100 Subject: Proposal: Optimizing Efficiency using Read-only Arrays Message-ID: <000701d91b9c$bef3ff10$3cdbfd30$@eu> Proposal: Optimizing Efficiency using Read-only Arrays TL;DR: Read-only Arrays will improve speed, reduce memory and power consumption, provide security by default, and make programming and reviews easier and quicker. Looking at the profile of any average real-world application, it is apparent that a lot of memory activity stems from allocating byte arrays. Byte arrays are a core building block of several APIs in OpenJDK. Just to name two of them: First and foremost Strings, as they are ubiquitous, but also I/O, as byte arrays are the buckets which carry all data through any InputStream/OutputStream. While I was authoring several java.io optimizations in the past months, the latter became the driver for me write down this propsal. Nevertheless, the proposal is focusing on a general solution, applicable to all Java APIs, beyond I/O. To perform any I/O in Java, all data MUST pass one or multiple byte arrays, each and every day. As it is easy to imagine, we can easily talk about multiple Gigabytes per day for an average server product. Once this array reference is passed to a custom method, it leaves the safe harbor of the JDK while entering possibly evil outside world - it becomes compromised. The called custom method ("Mr Evil") could either read privata data sitting in the array beyond passed read lower and upper limits, or could write poisoned data into the passed array, picked up afterwards by the JDK code (hence is treated as "safe" data). To mitigate these risks, typically byte arrays are duplicated (at least within limits) before forwarded to the outer world, so the "evil" receiver will only see a temporary / trimmed copy of the array. Just due to that single safety means alone, each day tens of thousands of Java servers are squandering precious memory and power, producing considerable amounts of carbon dioxide in turn. While copying buffers is effective, it also is inefficient. "Inefficiency" is definitively not a term we want Java to be recognized as in the age of climate change. N.B.: As soon as we omit explicit creation of an array copy, either due to a human programming fault, or due to an unexpected technical failure, security is ineffective! Hence relying on explicit copies is also a suboptimal ("flaky") safety means. Due to that risk, reviews of I/O code often become complex, lenghty and exhausting, making them rather expensive. This is just one single example. You could easily find lots more in the JDK. If the Java language would have a means to mark arrays as "read-only" to the Compiler / JVM (just like it alrady has for final variables), then no more need for an explicit copy exists. Several benefits would arise from the fact that no copy of the array is created (and removed) in turn: * Speed is improved. While System.arraycopy() is quick, not calling it at all is quicker. * GC pressure is reduced. While it might be low already, not creating a copy of an array makes it zero. * Security by default. As the JVM cannot write "read-only" arrays, there is no harm when an explicit copy is omitted. * Reduced memory consumption. No copy at all means literally zero additional memory. * Reduced power consumption. No power to invest into squandered CPU cycles. * Easier programming. No need to remind explicit creation of copies. * Simpler code. No copies means no code to create them, making the reminder simpler to understand. * Quicker reviews. Reviewer does not have to take care to check for compromised buffers, which is easily forgotten. While each single effect might be small, remind that all these effects will happen all together at once, and are massively applied each and every day, as arrays are building blocks of the JDK. To sum up, I'd like to propose to add a means to the Java language which turns arrays into "read-only" arrays. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kasperni at gmail.com Thu Dec 29 15:58:35 2022 From: kasperni at gmail.com (Kasper Nielsen) Date: Thu, 29 Dec 2022 16:58:35 +0100 Subject: Proposal: Optimizing Efficiency using Read-only Arrays In-Reply-To: <000701d91b9c$bef3ff10$3cdbfd30$@eu> References: <000701d91b9c$bef3ff10$3cdbfd30$@eu> Message-ID: On Thu, 29 Dec 2022 at 16:46, Markus Karg wrote: > > Proposal: Optimizing Efficiency using Read-only Arrays > There are already two draft JEPs for this: https://openjdk.org/jeps/8261099 https://openjdk.org/jeps/8261007 /Kasper /Kasper From markus at headcrashing.eu Thu Dec 29 16:08:39 2022 From: markus at headcrashing.eu (Markus Karg) Date: Thu, 29 Dec 2022 17:08:39 +0100 Subject: AW: Proposal: Optimizing Efficiency using Read-only Arrays In-Reply-To: References: <000701d91b9c$bef3ff10$3cdbfd30$@eu> Message-ID: <000f01d91b9f$d087a1c0$7196e540$@eu> Thank you, Kasper. Actually this is not exactly the same but nearly related: Those two JEPs provide just one specific, possible solution ("freezing", i. e. turning previously writable array into read-only one) to the overall problem. There could be other solutions these JEPs explicitly do not aim: For example, another solution for my proposal is directly creating a read-only constant instead of creating a writable array and then "freezing" it afterwards (which, IIUC, will hold two objects in memory, a still writable one, and a read-only view). The aim of my PR is to discuss the final solution we actually want to go, and when. -Markus -----Urspr?ngliche Nachricht----- Von: Kasper Nielsen [mailto:kasperni at gmail.com] Gesendet: Donnerstag, 29. Dezember 2022 16:59 An: Markus Karg Cc: amber-dev at openjdk.org Betreff: Re: Proposal: Optimizing Efficiency using Read-only Arrays On Thu, 29 Dec 2022 at 16:46, Markus Karg wrote: > > Proposal: Optimizing Efficiency using Read-only Arrays > There are already two draft JEPs for this: https://openjdk.org/jeps/8261099 https://openjdk.org/jeps/8261007 /Kasper /Kasper From redio.development at gmail.com Thu Dec 29 16:21:32 2022 From: redio.development at gmail.com (Red IO) Date: Thu, 29 Dec 2022 17:21:32 +0100 Subject: Proposal: Optimizing Efficiency using Read-only Arrays In-Reply-To: References: <000701d91b9c$bef3ff10$3cdbfd30$@eu> Message-ID: What about a user oriented general View class? A class that can base its data on memory owned by another data structure. You could create a view from a list an array or with a bit of integration with Panama from a memory segment. The view could be mutable or immutable could be subranged without copy and would provide a good return type to pass to untrusted code. The biggest downside would be primitives which would suffer boxing or would require hand specialized versions. On Thu, Dec 29, 2022, 16:58 Kasper Nielsen wrote: > On Thu, 29 Dec 2022 at 16:46, Markus Karg wrote: > > > > Proposal: Optimizing Efficiency using Read-only Arrays > > > > There are already two draft JEPs for this: > > https://openjdk.org/jeps/8261099 > https://openjdk.org/jeps/8261007 > > /Kasper > > > /Kasper > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Thu Dec 29 16:42:28 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 29 Dec 2022 16:42:28 +0000 Subject: Proposal: Optimizing Efficiency using Read-only Arrays In-Reply-To: <000f01d91b9f$d087a1c0$7196e540$@eu> References: <000701d91b9c$bef3ff10$3cdbfd30$@eu> <000f01d91b9f$d087a1c0$7196e540$@eu> Message-ID: <531B5B49-A3A9-4FE0-91AC-B2232A958536@oracle.com> Freezing addresses nearly all of the concerns you raised, except one: that one might only want to share a part of any array with a callee, but once you share the reference, they can read outside the bounds of what you?ve shared. There are known techniques for addressing this as well, but probably best to get the big part of it done first (eliminating the inherent mutability of array references) and then come back for slices/spans. > On Dec 29, 2022, at 11:08 AM, Markus Karg wrote: > > Thank you, Kasper. > > Actually this is not exactly the same but nearly related: Those two JEPs provide just one specific, possible solution ("freezing", i. e. turning previously writable array into read-only one) to the overall problem. There could be other solutions these JEPs explicitly do not aim: For example, another solution for my proposal is directly creating a read-only constant instead of creating a writable array and then "freezing" it afterwards (which, IIUC, will hold two objects in memory, a still writable one, and a read-only view). The aim of my PR is to discuss the final solution we actually want to go, and when. > > -Markus > > -----Urspr?ngliche Nachricht----- > Von: Kasper Nielsen [mailto:kasperni at gmail.com] > Gesendet: Donnerstag, 29. Dezember 2022 16:59 > An: Markus Karg > Cc: amber-dev at openjdk.org > Betreff: Re: Proposal: Optimizing Efficiency using Read-only Arrays > > On Thu, 29 Dec 2022 at 16:46, Markus Karg wrote: >> >> Proposal: Optimizing Efficiency using Read-only Arrays >> > > There are already two draft JEPs for this: > > https://openjdk.org/jeps/8261099 > https://openjdk.org/jeps/8261007 > > /Kasper > > > /Kasper > From numeralnathan at gmail.com Thu Dec 29 16:55:25 2022 From: numeralnathan at gmail.com (Nathan Reynolds) Date: Thu, 29 Dec 2022 08:55:25 -0800 Subject: Function level try recourse directives. In-Reply-To: <20221229102821.0cb20fed@sunflower.int.arc7.info> References: <20221227140726.04f72129@sunflower.int.arc7.info> <20221227185317.4acbd1c5@sunflower.int.arc7.info> <20221229102821.0cb20fed@sunflower.int.arc7.info> Message-ID: I understand. However, CloseableCollection is not part of the JDK. It is in a separate library that is probably brand new. It probably is used only by a handful of projects at this time. On Thu, Dec 29, 2022 at 2:28 AM Mark Raynsford wrote: > On 2022-12-28T12:15:03 -0800 > Nathan Reynolds wrote: > > > CloseableCollection is a class of the library. It won't show up in heap > > dumps because probably very few people are using it. You would need to > > look at JIT optimized assembly to see if JIT is eliding the > > CloseableCollection and it's internal data structure (array?). > > To be clear, I'm referring to the heap sampling feature of tools like > VisualVM: > > https://herongyang.com/Java-Tools/jvisualvm-Heap-Dump-Class-List.jpg > > The CloseableCollection will show up even if there's only a single > instance. The internal array of the ArrayDeque does get mixed into the > samples for Object[], though. > > -- > Mark Raynsford | https://www.io7m.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From numeralnathan at gmail.com Thu Dec 29 17:04:11 2022 From: numeralnathan at gmail.com (Nathan Reynolds) Date: Thu, 29 Dec 2022 09:04:11 -0800 Subject: Proposal: Optimizing Efficiency using Read-only Arrays In-Reply-To: <000701d91b9c$bef3ff10$3cdbfd30$@eu> References: <000701d91b9c$bef3ff10$3cdbfd30$@eu> Message-ID: In one project, we changed the code to be zero copy I/O. This dramatically improved the performance of I/O intensive applications. So, getting to zero copy I/O is a tremendous win. How would the Java language prevent a rogue custom method from altering the array? Someone could write a method that receives a read-only array and then change the bytecode in the .class to say it receives a writable array. The enforcement would have to be at the JVM level. Furthermore, reflection would have to deal with enforcing the read-only attribute. On Thu, Dec 29, 2022 at 7:46 AM Markus Karg wrote: > Proposal: Optimizing Efficiency using Read-only Arrays > > > > TL;DR: Read-only Arrays will improve speed, reduce memory and power > consumption, provide security by default, and make programming and reviews > easier and quicker. > > > > Looking at the profile of any average real-world application, it is > apparent that a lot of memory activity stems from allocating byte arrays. > > Byte arrays are a core building block of several APIs in OpenJDK. > > Just to name two of them: First and foremost Strings, as they are > ubiquitous, but also I/O, as byte arrays are the buckets which carry all > data through any InputStream/OutputStream. > > While I was authoring several java.io optimizations in the past months, > the latter became the driver for me write down this propsal. > > Nevertheless, the proposal is focusing on a general solution, applicable > to all Java APIs, beyond I/O. > > > > To perform any I/O in Java, all data MUST pass one or multiple byte > arrays, each and every day. > > As it is easy to imagine, we can easily talk about multiple Gigabytes per > day for an average server product. > > Once this array reference is passed to a custom method, it leaves the safe > harbor of the JDK while entering possibly evil outside world - it becomes > compromised. > > The called custom method ("Mr Evil") could either read privata data > sitting in the array beyond passed read lower and upper limits, or could > write poisoned data into the passed array, picked up afterwards by the JDK > code (hence is treated as "safe" data). > > To mitigate these risks, typically byte arrays are duplicated (at least > within limits) before forwarded to the outer world, so the "evil" receiver > will only see a temporary / trimmed copy of the array. > > Just due to that single safety means alone, each day tens of thousands of > Java servers are squandering precious memory and power, producing > considerable amounts of carbon dioxide in turn. > > While copying buffers is effective, it also is inefficient. > > "Inefficiency" is definitively not a term we want Java to be recognized as > in the age of climate change. > > N.B.: As soon as we omit explicit creation of an array copy, either due to > a human programming fault, or due to an unexpected technical failure, > security is ineffective! Hence relying on explicit copies is also a > suboptimal ("flaky") safety means. Due to that risk, reviews of I/O code > often become complex, lenghty and exhausting, making them rather expensive. > > This is just one single example. You could easily find lots more in the > JDK. > > > > If the Java language would have a means to mark arrays as "read-only" to > the Compiler / JVM (just like it alrady has for final variables), then no > more need for an explicit copy exists. > > Several benefits would arise from the fact that no copy of the array is > created (and removed) in turn: > > * Speed is improved. While System.arraycopy() is quick, not calling it at > all is quicker. > > * GC pressure is reduced. While it might be low already, not creating a > copy of an array makes it zero. > > * Security by default. As the JVM cannot write "read-only" arrays, there > is no harm when an explicit copy is omitted. > > * Reduced memory consumption. No copy at all means literally zero > additional memory. > > * Reduced power consumption. No power to invest into squandered CPU cycles. > > * Easier programming. No need to remind explicit creation of copies. > > * Simpler code. No copies means no code to create them, making the > reminder simpler to understand. > > * Quicker reviews. Reviewer does not have to take care to check for > compromised buffers, which is easily forgotten. > > > > While each single effect might be small, remind that all these effects > will happen all together at once, and are massively applied each and every > day, as arrays are building blocks of the JDK. > > > > To sum up, I'd like to propose to add a means to the Java language which > turns arrays into "read-only" arrays. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From org.openjdk at io7m.com Thu Dec 29 17:08:06 2022 From: org.openjdk at io7m.com (Mark Raynsford) Date: Thu, 29 Dec 2022 17:08:06 +0000 Subject: Function level try recourse directives. In-Reply-To: References: <20221227140726.04f72129@sunflower.int.arc7.info> <20221227185317.4acbd1c5@sunflower.int.arc7.info> <20221229102821.0cb20fed@sunflower.int.arc7.info> Message-ID: <20221229170806.4b226b36@sunflower.int.arc7.info> On 2022-12-29T08:55:25 -0800 Nathan Reynolds wrote: > I understand. However, CloseableCollection is not part of the JDK. It is > in a separate library that is probably brand new. It probably is used only > by a handful of projects at this time. I'm sorry, I think I missed your point (or got my wires crossed with the replies here). -- Mark Raynsford | https://www.io7m.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 228 bytes Desc: OpenPGP digital signature URL: From brian.goetz at oracle.com Thu Dec 29 17:10:43 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 29 Dec 2022 17:10:43 +0000 Subject: Proposal: Optimizing Efficiency using Read-only Arrays In-Reply-To: References: <000701d91b9c$bef3ff10$3cdbfd30$@eu> Message-ID: <3F306965-638A-4461-9F9A-4E520F988C1A@oracle.com> This speaks to an important design decision underlying the freezing proposal. The ecosystem is full of API points that take arrays; we wouldn?t want to bifurcate those into those who take mutable array and those who take read-only arrays. So a new bytecode descriptor for readonly arrays would be problematic not from a platform perspective, but from an ecosystem perspective. What we want is to reuse the world of existing non-evil, array-consuming code without disruption. So it makes sense that writability be a dynamic property enforced by the VM. Fortunately, the VM already has to validate array writes (for reference arrays), because you can cast a String[] to an Object[] and then try to put an Integer in it ? and you?ll get an ArrayStoreException. For a readonly array, you?d always get ASE. On Dec 29, 2022, at 12:04 PM, Nathan Reynolds > wrote: In one project, we changed the code to be zero copy I/O. This dramatically improved the performance of I/O intensive applications. So, getting to zero copy I/O is a tremendous win. How would the Java language prevent a rogue custom method from altering the array? Someone could write a method that receives a read-only array and then change the bytecode in the .class to say it receives a writable array. The enforcement would have to be at the JVM level. Furthermore, reflection would have to deal with enforcing the read-only attribute. On Thu, Dec 29, 2022 at 7:46 AM Markus Karg > wrote: Proposal: Optimizing Efficiency using Read-only Arrays TL;DR: Read-only Arrays will improve speed, reduce memory and power consumption, provide security by default, and make programming and reviews easier and quicker. Looking at the profile of any average real-world application, it is apparent that a lot of memory activity stems from allocating byte arrays. Byte arrays are a core building block of several APIs in OpenJDK. Just to name two of them: First and foremost Strings, as they are ubiquitous, but also I/O, as byte arrays are the buckets which carry all data through any InputStream/OutputStream. While I was authoring several java.io optimizations in the past months, the latter became the driver for me write down this propsal. Nevertheless, the proposal is focusing on a general solution, applicable to all Java APIs, beyond I/O. To perform any I/O in Java, all data MUST pass one or multiple byte arrays, each and every day. As it is easy to imagine, we can easily talk about multiple Gigabytes per day for an average server product. Once this array reference is passed to a custom method, it leaves the safe harbor of the JDK while entering possibly evil outside world - it becomes compromised. The called custom method ("Mr Evil") could either read privata data sitting in the array beyond passed read lower and upper limits, or could write poisoned data into the passed array, picked up afterwards by the JDK code (hence is treated as "safe" data). To mitigate these risks, typically byte arrays are duplicated (at least within limits) before forwarded to the outer world, so the "evil" receiver will only see a temporary / trimmed copy of the array. Just due to that single safety means alone, each day tens of thousands of Java servers are squandering precious memory and power, producing considerable amounts of carbon dioxide in turn. While copying buffers is effective, it also is inefficient. "Inefficiency" is definitively not a term we want Java to be recognized as in the age of climate change. N.B.: As soon as we omit explicit creation of an array copy, either due to a human programming fault, or due to an unexpected technical failure, security is ineffective! Hence relying on explicit copies is also a suboptimal ("flaky") safety means. Due to that risk, reviews of I/O code often become complex, lenghty and exhausting, making them rather expensive. This is just one single example. You could easily find lots more in the JDK. If the Java language would have a means to mark arrays as "read-only" to the Compiler / JVM (just like it alrady has for final variables), then no more need for an explicit copy exists. Several benefits would arise from the fact that no copy of the array is created (and removed) in turn: * Speed is improved. While System.arraycopy() is quick, not calling it at all is quicker. * GC pressure is reduced. While it might be low already, not creating a copy of an array makes it zero. * Security by default. As the JVM cannot write "read-only" arrays, there is no harm when an explicit copy is omitted. * Reduced memory consumption. No copy at all means literally zero additional memory. * Reduced power consumption. No power to invest into squandered CPU cycles. * Easier programming. No need to remind explicit creation of copies. * Simpler code. No copies means no code to create them, making the reminder simpler to understand. * Quicker reviews. Reviewer does not have to take care to check for compromised buffers, which is easily forgotten. While each single effect might be small, remind that all these effects will happen all together at once, and are massively applied each and every day, as arrays are building blocks of the JDK. To sum up, I'd like to propose to add a means to the Java language which turns arrays into "read-only" arrays. -------------- next part -------------- An HTML attachment was scrubbed... URL: From archie.cobbs at gmail.com Fri Dec 30 04:10:50 2022 From: archie.cobbs at gmail.com (Archie Cobbs) Date: Thu, 29 Dec 2022 22:10:50 -0600 Subject: Loosening requirements for super() invocation In-Reply-To: References: <140e133e-4c46-5d83-1ccf-32e40406a13a@oracle.com> <362d1d7b-97fe-dc2e-de02-1d0169bd2626@oracle.com> <75cc216c-12c1-d7bd-37f1-c73d4fc97143@oracle.com> <7242cdba-0b7d-2520-b159-b13f91113c18@oracle.com> <347917af-4663-6fa8-a6b3-6645a3e5f5ce@oracle.com> Message-ID: On Wed, Dec 28, 2022 at 4:31 PM Archie Cobbs wrote: > So fortunately it looks feasible to address these with > @SuppressWarnings("this-escape") annotations. > > Adding them is probably a good thing in its own right, because they would > serve as new "Heads up for possible 'this' escape" markers in the code > where there were none before, especially because this gotcha can be pretty > hard to spot. > > I'll work on refining the 'this' escape patch to use @SuppressWarnings > instead of build flags, and then update this thread when that's done. > OK I have a draft of this. It's still a "draft" because I don't have the ability to build locally for every platform, so I'm having trouble provoking all the warnings from Java code that is platform-specific. And the automated github builds take a long time and then only report one more new warning, making for an impossibly slow feedback loop (if anyone is interested in helping out, that would be awesome and it's very easy). In any case, the good news is that fewer @SupppressWarnings annotations than I thought are required. In part this is because there were some redundant warnings being generated, which is now fixed. Here are the updated stats. These numbers are promisingly low. The first column is the number of @SupppressWarnings annotations added to eliminate all of the new 'this' escape warnings I'm able to provoke. #SUPPRESS #FILES MODULE 47 3077 java.base 0 128 java.compiler 0 18 java.datatransfer 86 2899 java.desktop 0 10 java.instrument 0 23 java.logging 4 330 java.management 0 16 java.management.rmi 5 199 java.naming 15 142 java.net.http 0 20 java.prefs 4 106 java.rmi 0 15 java.scripting 0 1 java.se 7 216 java.security.jgss 0 30 java.security.sasl 0 23 java.smartcardio 1 77 java.sql 0 56 java.sql.rowset 0 5 java.transaction.xa 39 1848 java.xml 32 271 java.xml.crypto 3 20 jdk.accessibility 0 21 jdk.attach 0 18 jdk.charsets 53 333 jdk.compiler 1 76 jdk.crypto.cryptoki 0 35 jdk.crypto.ec 0 11 jdk.crypto.mscapi 4 68 jdk.dynalink 0 3 jdk.editpad 10 942 jdk.hotspot.agent 3 56 jdk.httpserver 0 5 jdk.incubator.concurrent 0 50 jdk.incubator.vector 0 3 jdk.internal.ed 1 61 jdk.internal.jvmstat 1 113 jdk.internal.le 1 52 jdk.internal.opt 5 209 jdk.internal.vm.ci 0 1 jdk.internal.vm.compiler 0 1 jdk.internal.vm.compiler.management 0 18 jdk.jartool 5 228 jdk.javadoc 0 41 jdk.jcmd 18 64 jdk.jconsole 10 124 jdk.jdeps 25 254 jdk.jdi 0 1 jdk.jdwp.agent 0 263 jdk.jfr 0 77 jdk.jlink 1 80 jdk.jpackage 6 88 jdk.jshell 0 4 jdk.jsobject 1 11 jdk.jstatd 0 281 jdk.localedata 1 25 jdk.management 0 19 jdk.management.agent 0 15 jdk.management.jfr 0 16 jdk.naming.dns 0 8 jdk.naming.rmi 0 11 jdk.net 0 2 jdk.nio.mapmode 0 11 jdk.random 0 37 jdk.sctp 0 30 jdk.security.auth 0 16 jdk.security.jgss 0 9 jdk.unsupported 0 8 jdk.unsupported.desktop 0 94 jdk.xml.dom 1 14 jdk.zipfs Code available here . -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri Dec 30 16:48:14 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 30 Dec 2022 16:48:14 +0000 Subject: Loosening requirements for super() invocation In-Reply-To: References: <140e133e-4c46-5d83-1ccf-32e40406a13a@oracle.com> <362d1d7b-97fe-dc2e-de02-1d0169bd2626@oracle.com> <75cc216c-12c1-d7bd-37f1-c73d4fc97143@oracle.com> <7242cdba-0b7d-2520-b159-b13f91113c18@oracle.com> <347917af-4663-6fa8-a6b3-6645a3e5f5ce@oracle.com> Message-ID: Nice. One thing that makes me a little uncomfortable is that @SW usually is used to say ?the compiler can?t prove this is safe, but I am asserting it is.? But this is different; the compiler has discovered the class is unsafe, and @SW is saying ?I know, I know, stop telling me.? So I think there?s an additional documentation opportunity here where, for the classes we had to annotate with @SW, we include an additional @apiNote of the form ?This class behaves badly, oops?, with a link to a common doc page explaining the problem. (While we don?t have to do this necessarily immediately, the chance of it falling on the floor if we don?t are higher.). Basically, what you?ve identified is that there are a number of classes in the JDK which fail to follow best practices with regard to self-use from constructors. We can?t turn back the clock, but we can mark these as ?don?t code like my brother?, so people see these patterns of coding and realize they are not to be emulated. I also see in the code that there are two separate categories, LeakingThisInConstructor and this-escape; what?s the difference? On Dec 29, 2022, at 11:10 PM, Archie Cobbs > wrote: On Wed, Dec 28, 2022 at 4:31 PM Archie Cobbs > wrote: So fortunately it looks feasible to address these with @SuppressWarnings("this-escape") annotations. Adding them is probably a good thing in its own right, because they would serve as new "Heads up for possible 'this' escape" markers in the code where there were none before, especially because this gotcha can be pretty hard to spot. I'll work on refining the 'this' escape patch to use @SuppressWarnings instead of build flags, and then update this thread when that's done. OK I have a draft of this. It's still a "draft" because I don't have the ability to build locally for every platform, so I'm having trouble provoking all the warnings from Java code that is platform-specific. And the automated github builds take a long time and then only report one more new warning, making for an impossibly slow feedback loop (if anyone is interested in helping out, that would be awesome and it's very easy). In any case, the good news is that fewer @SupppressWarnings annotations than I thought are required. In part this is because there were some redundant warnings being generated, which is now fixed. Here are the updated stats. These numbers are promisingly low. The first column is the number of @SupppressWarnings annotations added to eliminate all of the new 'this' escape warnings I'm able to provoke. #SUPPRESS #FILES MODULE 47 3077 java.base 0 128 java.compiler 0 18 java.datatransfer 86 2899 java.desktop 0 10 java.instrument 0 23 java.logging 4 330 java.management 0 16 java.management.rmi 5 199 java.naming 15 142 java.net.http 0 20 java.prefs 4 106 java.rmi 0 15 java.scripting 0 1 java.se 7 216 java.security.jgss 0 30 java.security.sasl 0 23 java.smartcardio 1 77 java.sql 0 56 java.sql.rowset 0 5 java.transaction.xa 39 1848 java.xml 32 271 java.xml.crypto 3 20 jdk.accessibility 0 21 jdk.attach 0 18 jdk.charsets 53 333 jdk.compiler 1 76 jdk.crypto.cryptoki 0 35 jdk.crypto.ec 0 11 jdk.crypto.mscapi 4 68 jdk.dynalink 0 3 jdk.editpad 10 942 jdk.hotspot.agent 3 56 jdk.httpserver 0 5 jdk.incubator.concurrent 0 50 jdk.incubator.vector 0 3 jdk.internal.ed 1 61 jdk.internal.jvmstat 1 113 jdk.internal.le 1 52 jdk.internal.opt 5 209 jdk.internal.vm.ci 0 1 jdk.internal.vm.compiler 0 1 jdk.internal.vm.compiler.management 0 18 jdk.jartool 5 228 jdk.javadoc 0 41 jdk.jcmd 18 64 jdk.jconsole 10 124 jdk.jdeps 25 254 jdk.jdi 0 1 jdk.jdwp.agent 0 263 jdk.jfr 0 77 jdk.jlink 1 80 jdk.jpackage 6 88 jdk.jshell 0 4 jdk.jsobject 1 11 jdk.jstatd 0 281 jdk.localedata 1 25 jdk.management 0 19 jdk.management.agent 0 15 jdk.management.jfr 0 16 jdk.naming.dns 0 8 jdk.naming.rmi 0 11 jdk.net 0 2 jdk.nio.mapmode 0 11 jdk.random 0 37 jdk.sctp 0 30 jdk.security.auth 0 16 jdk.security.jgss 0 9 jdk.unsupported 0 8 jdk.unsupported.desktop 0 94 jdk.xml.dom 1 14 jdk.zipfs Code available here. -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From archie.cobbs at gmail.com Fri Dec 30 17:20:47 2022 From: archie.cobbs at gmail.com (Archie Cobbs) Date: Fri, 30 Dec 2022 11:20:47 -0600 Subject: Loosening requirements for super() invocation In-Reply-To: References: <140e133e-4c46-5d83-1ccf-32e40406a13a@oracle.com> <362d1d7b-97fe-dc2e-de02-1d0169bd2626@oracle.com> <75cc216c-12c1-d7bd-37f1-c73d4fc97143@oracle.com> <7242cdba-0b7d-2520-b159-b13f91113c18@oracle.com> <347917af-4663-6fa8-a6b3-6645a3e5f5ce@oracle.com> Message-ID: On Fri, Dec 30, 2022 at 10:48 AM Brian Goetz wrote: > One thing that makes me a little uncomfortable is that @SW usually is used > to say ?the compiler can?t prove this is safe, but I am asserting it is.? > But this is different; the compiler has discovered the class is unsafe, > and @SW is saying ?I know, I know, stop telling me.? So I think there?s an > additional documentation opportunity here where, for the classes we had to > annotate with @SW, we include an additional @apiNote of the form ?This > class behaves badly, oops?, with a link to a common doc page explaining the > problem. (While we don?t have to do this necessarily immediately, the > chance of it falling on the floor if we don?t are higher.). Basically, what > you?ve identified is that there are a number of classes in the JDK which > fail to follow best practices with regard to self-use from constructors. > We can?t turn back the clock, but we can mark these as ?don?t code like my > brother?, so people see these patterns of coding and realize they are not > to be emulated. > Good point. I think it's also worth considering which scenarios require such documentation. We have drawn the "boundary lines" at the compilation unit boundary, but this means (for example) there are package-private classes that generate warnings. Because these classes will end up in a module, which requires special effort to crack open, the warnings only really apply to JDK developers working in that same package. Might be worth restricting this to classes that a "normal" person could "normally" extend, for some suitable definitions of "normal" and "normally". I also see in the code that there are two separate categories, > LeakingThisInConstructor and this-escape; what?s the difference? > Great question. I was surprised to stumble across existing code with @SuppressWarnings("LeakingThisInConstructor") annotations. >From what I can tell, these are leftovers from about 10 years ago when some developers were using NetBeans, which had just such a warning. I don't think they are relevant any more but I didn't want to presume anything so I left them in there. Thinking about them more, we should probably just replace them because otherwise they will just continue to confuse... -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Fri Dec 30 19:00:42 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 30 Dec 2022 19:00:42 +0000 Subject: Loosening requirements for super() invocation In-Reply-To: References: <140e133e-4c46-5d83-1ccf-32e40406a13a@oracle.com> <362d1d7b-97fe-dc2e-de02-1d0169bd2626@oracle.com> <75cc216c-12c1-d7bd-37f1-c73d4fc97143@oracle.com> <7242cdba-0b7d-2520-b159-b13f91113c18@oracle.com> <347917af-4663-6fa8-a6b3-6645a3e5f5ce@oracle.com> Message-ID: <91E15764-ECBF-43DB-831F-0FB31CD0ECE9@oracle.com> On Dec 30, 2022, at 12:20 PM, Archie Cobbs > wrote: On Fri, Dec 30, 2022 at 10:48 AM Brian Goetz > wrote: One thing that makes me a little uncomfortable is that @SW usually is used to say ?the compiler can?t prove this is safe, but I am asserting it is.? But this is different; the compiler has discovered the class is unsafe, and @SW is saying ?I know, I know, stop telling me.? So I think there?s an additional documentation opportunity here where, for the classes we had to annotate with @SW, we include an additional @apiNote of the form ?This class behaves badly, oops?, with a link to a common doc page explaining the problem. (While we don?t have to do this necessarily immediately, the chance of it falling on the floor if we don?t are higher.). Basically, what you?ve identified is that there are a number of classes in the JDK which fail to follow best practices with regard to self-use from constructors. We can?t turn back the clock, but we can mark these as ?don?t code like my brother?, so people see these patterns of coding and realize they are not to be emulated. Good point. I think it's also worth considering which scenarios require such documentation. We have drawn the "boundary lines" at the compilation unit boundary, but this means (for example) there are package-private classes that generate warnings. Because these classes will end up in a module, which requires special effort to crack open, the warnings only really apply to JDK developers working in that same package. For package-private classes, we should know all the subclasses, and therefore may have the opportunity to seal it. Then we are not subject to ?arbitrary subclass might do X?, but only ?these specific subclasses do X.? So perhaps this is an opportunity to make final / seal some package-private classes that don?t need extensibility. I also see in the code that there are two separate categories, LeakingThisInConstructor and this-escape; what?s the difference? Great question. I was surprised to stumble across existing code with @SuppressWarnings("LeakingThisInConstructor") annotations. From what I can tell, these are leftovers from about 10 years ago when some developers were using NetBeans, which had just such a warning. I don't think they are relevant any more but I didn't want to presume anything so I left them in there. So, LTIC is not a lint category of javac? When Jan gets back from break, maybe he can shed some more light on this. But unless we have a good reason to keep the LTICs, we should consider garbage-collecting them. Thinking about them more, we should probably just replace them because otherwise they will just continue to confuse... -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From numeralnathan at gmail.com Fri Dec 30 19:04:23 2022 From: numeralnathan at gmail.com (Nathan Reynolds) Date: Fri, 30 Dec 2022 11:04:23 -0800 Subject: Loosening requirements for super() invocation In-Reply-To: References: <140e133e-4c46-5d83-1ccf-32e40406a13a@oracle.com> <362d1d7b-97fe-dc2e-de02-1d0169bd2626@oracle.com> <75cc216c-12c1-d7bd-37f1-c73d4fc97143@oracle.com> <7242cdba-0b7d-2520-b159-b13f91113c18@oracle.com> <347917af-4663-6fa8-a6b3-6645a3e5f5ce@oracle.com> Message-ID: Let's say we have a constructor that calls final methods (or the class is final) that belong to the class and don't leak "this" (or call other final methods that don't leak "this"). Would this be flagged? If so, why? This doesn't seem to be a leak. I use this when the setter has some complex logic and would rather re-use than duplicate code. (I suppose the same could be said if the constructor calls a static final method in the class and passes "this"). Let's say we have a constructor that assigns all the member fields and then leaks "this". Would this be flagged? If so, why? This doesn't seem to be a leak any more since the object is fully constructed. On Fri, Dec 30, 2022 at 9:21 AM Archie Cobbs wrote: > On Fri, Dec 30, 2022 at 10:48 AM Brian Goetz > wrote: > >> One thing that makes me a little uncomfortable is that @SW usually is >> used to say ?the compiler can?t prove this is safe, but I am asserting it >> is.? But this is different; the compiler has discovered the class is >> unsafe, and @SW is saying ?I know, I know, stop telling me.? So I think >> there?s an additional documentation opportunity here where, for the classes >> we had to annotate with @SW, we include an additional @apiNote of the form >> ?This class behaves badly, oops?, with a link to a common doc page >> explaining the problem. (While we don?t have to do this necessarily >> immediately, the chance of it falling on the floor if we don?t are >> higher.). Basically, what you?ve identified is that there are a number of >> classes in the JDK which fail to follow best practices with regard to >> self-use from constructors. We can?t turn back the clock, but we can mark >> these as ?don?t code like my brother?, so people see these patterns of >> coding and realize they are not to be emulated. >> > > Good point. > > I think it's also worth considering which scenarios require such > documentation. We have drawn the "boundary lines" at the compilation unit > boundary, but this means (for example) there are package-private classes > that generate warnings. Because these classes will end up in a module, > which requires special effort to crack open, the warnings only really apply > to JDK developers working in that same package. > > Might be worth restricting this to classes that a "normal" person could > "normally" extend, for some suitable definitions of "normal" and "normally". > > I also see in the code that there are two separate categories, >> LeakingThisInConstructor and this-escape; what?s the difference? >> > > Great question. I was surprised to stumble across existing code with > @SuppressWarnings("LeakingThisInConstructor") annotations. > > From what I can tell, these are leftovers from about 10 years ago when > some developers were using NetBeans, which had just such a warning. I don't > think they are relevant any more but I didn't want to presume anything so I > left them in there. > > Thinking about them more, we should probably just replace them because > otherwise they will just continue to confuse... > > -Archie > > -- > Archie L. Cobbs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From archie.cobbs at gmail.com Fri Dec 30 23:47:56 2022 From: archie.cobbs at gmail.com (Archie Cobbs) Date: Fri, 30 Dec 2022 17:47:56 -0600 Subject: Loosening requirements for super() invocation In-Reply-To: References: <140e133e-4c46-5d83-1ccf-32e40406a13a@oracle.com> <362d1d7b-97fe-dc2e-de02-1d0169bd2626@oracle.com> <75cc216c-12c1-d7bd-37f1-c73d4fc97143@oracle.com> <7242cdba-0b7d-2520-b159-b13f91113c18@oracle.com> <347917af-4663-6fa8-a6b3-6645a3e5f5ce@oracle.com> Message-ID: Hi Nathan, On Fri, Dec 30, 2022 at 1:04 PM Nathan Reynolds wrote: > Let's say we have a constructor that calls final methods (or the class is > final) that belong to the class and don't leak "this" (or call other final > methods that don't leak "this"). Would this be flagged? > No, this would not be reported as a leak. It has to be possible that the class being analyzed could, somehow, somewhere, someday, be subclassed in some other compilation unit. Also, the constructor would have to be invokable by such a subclass. So private constructors cannot leak directly. They can leak indirectly, however, if invoked from other constructors. > Let's say we have a constructor that assigns all the member fields and > then leaks "this". Would this be flagged? If so, why? This doesn't seem > to be a leak any more since the object is fully constructed. > Yes, this would be flagged. The reason is that the "leak" is defined not in terms of the class being analyzed, but in terms of some theoretical subclass. In your example, even if the fields in the class being analyzed are fully initialized before the leak, no fields in a subclass could be, so from the subclass' perspective the leak still represents a threat. -Archie -- Archie L. Cobbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Sat Dec 31 19:02:27 2022 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 31 Dec 2022 19:02:27 +0000 Subject: Loosening requirements for super() invocation In-Reply-To: References: <140e133e-4c46-5d83-1ccf-32e40406a13a@oracle.com> <362d1d7b-97fe-dc2e-de02-1d0169bd2626@oracle.com> <75cc216c-12c1-d7bd-37f1-c73d4fc97143@oracle.com> <7242cdba-0b7d-2520-b159-b13f91113c18@oracle.com> <347917af-4663-6fa8-a6b3-6645a3e5f5ce@oracle.com> Message-ID: <5504CAA4-0815-4353-9154-6BCA24A430C4@oracle.com> > Let's say we have a constructor that assigns all the member fields and then leaks "this". Would this be flagged? If so, why? This doesn't seem to be a leak any more since the object is fully constructed. This is a common misconception, but there are at least two serious problems with this theory. First, if the class is not final, then it may not be fully constructed. Consider: class A { final int a; A(int a) { this.a = a; // Am I fully constructed? leak(); } void leak() { } } This looks harmless, right? Well, now try: class B extends A { String b; B(int a, String b) { super(a); this.b = b; } @Override void leak() { System.out.println(b.length()); } } The leak() call will be made when the *A* state is fully initialized, but the B state has not yet been initialized. B innocently overrides leak() and is surprised to see a not-fully-initialized B. Whoops. Moral: the ?end? of a constructor is not the end of a constructor. Moreover, even if you assume A was final, you still lose. Because the final field initialization safety semantics offered by the JMM are dependent on the constructor not leaking `this`. So if you leak `this`, your final field guarantees are voided, and another thread could see the wrong value for `a`. Whoops. Moral: this stuff is harder than it looks. ?Clever? arguments like ?but the object must be fully initialized at this point? are stepping out onto some thin ice that requires very, very careful reasoning. From jens.lidestrom at fripost.org Fri Dec 30 21:37:38 2022 From: jens.lidestrom at fripost.org (=?UTF-8?Q?Jens_Lidestr=c3=b6m?=) Date: Fri, 30 Dec 2022 21:37:38 -0000 Subject: Loosening requirements for super() invocation In-Reply-To: References: <75cc216c-12c1-d7bd-37f1-c73d4fc97143@oracle.com> <7242cdba-0b7d-2520-b159-b13f91113c18@oracle.com> <347917af-4663-6fa8-a6b3-6645a3e5f5ce@oracle.com> Message-ID: On 2022-12-30 17:48, Brian Goetz wrote: > One thing that makes me a little uncomfortable is that @SW usually is used to say ?the compiler can?t prove this is safe, but I am asserting it is.? But this is different; the compiler has discovered the class is unsafe, Is an escaping this-reference really always an unacceptable problem? To me it seems like an escaping this-reference is safe if the following condition is fulfilled: One can make sure that the this-reference is only stored by the method to which it is passed as argument, it is not used in any other way before the constructor has completed. Looking at the diff in Archie's branch [1], it seems like most cases are not problematic. [1]: https://github.com/openjdk/jdk/compare/master...archiecobbs:jdk:ThisEscape In most cases in the diff the this-reference escapes to be able to do the following: * Set up a circular relationship with another object. * Register a Cleaner. * Register a listener. Sometimes an object might not be considered to be properly constructed without doing such things. It seems useful to be able to rely on the constructor to complete them. Regards, Jens Lidestr?m On 2022-12-30 17:48, Brian Goetz wrote: > Nice. > > One thing that makes me a little uncomfortable is that @SW usually is used to say ?the compiler can?t prove this is safe, but I am asserting it is.? ?But this is different; the compiler has discovered the class is unsafe, and @SW is saying ?I know, I know, stop telling me.? ?So I think there?s an additional documentation opportunity here where, for the classes we had to annotate with @SW, we include an additional @apiNote of the form ?This class behaves badly, oops?, with a link to a common doc page explaining the problem. ?(While we don?t have to do this necessarily immediately, the chance of it falling on the floor if we don?t are higher.). Basically, what you?ve identified is that there are a ?number of classes in the JDK which fail to follow best practices with regard to self-use from constructors. ?We can?t turn back the clock, but we can mark these as ?don?t code like my brother?, so people see these patterns of coding and realize they are not to be emulated. > > I also see in the code that there are two separate categories, LeakingThisInConstructor and this-escape; what?s the difference? > > > >> On Dec 29, 2022, at 11:10 PM, Archie Cobbs > wrote: >> >> On Wed, Dec 28, 2022 at 4:31 PM Archie Cobbs > wrote: >> >> So fortunately it looks feasible to address these with @SuppressWarnings("this-escape") annotations. >> >> Adding them is probably a good thing in its own right, because they would serve as new "Heads up for possible 'this' escape" markers in the code where there were none before, especially because this gotcha can be pretty hard to spot. >> >> I'll work on refining the 'this' escape patch to use @SuppressWarnings instead of build flags, and then update this thread when that's done. >> >> >> OK I have a draft of this. It's still a "draft" because I don't have the ability to build locally for every platform, so I'm having trouble provoking all the warnings from Java code that is platform-specific. And the automated github builds take a long time and then only report one more new warning, making for an impossibly slow feedback loop (if anyone is interested in helping out, that would be awesome and it's very easy). >> >> In any case, the good news is that fewer @SupppressWarnings annotations than I thought are required. In part this is because there were some redundant warnings being generated, which is now fixed. >> >> Here are the updated stats. These numbers are promisingly low. >> >> The first column is the number of @SupppressWarnings annotations added to eliminate all of the new 'this' escape warnings I'm able to provoke. >> >> ?#SUPPRESS ? ?#FILES MODULE >> ? ? ? ? 47 ? ? ?3077 java.base >> ? ? ? ? ?0 ? ? ? 128 java.compiler >> ? ? ? ? ?0 ? ? ? ?18 java.datatransfer >> ? ? ? ? 86 ? ? ?2899 java.desktop >> ? ? ? ? ?0 ? ? ? ?10 java.instrument >> ? ? ? ? ?0 ? ? ? ?23 java.logging >> ? ? ? ? ?4 ? ? ? 330 java.management >> ? ? ? ? ?0 ? ? ? ?16 java.management.rmi >> ? ? ? ? ?5 ? ? ? 199 java.naming >> ? ? ? ? 15 ? ? ? 142 java.net.http >> ? ? ? ? ?0 ? ? ? ?20 java.prefs >> ? ? ? ? ?4 ? ? ? 106 java.rmi >> ? ? ? ? ?0 ? ? ? ?15 java.scripting >> ? ? ? ? ?0 ? ? ? ? 1 java.se >> ? ? ? ? ?7 ? ? ? 216 java.security.jgss >> ? ? ? ? ?0 ? ? ? ?30 java.security.sasl >> ? ? ? ? ?0 ? ? ? ?23 java.smartcardio >> ? ? ? ? ?1 ? ? ? ?77 java.sql >> ? ? ? ? ?0 ? ? ? ?56 java.sql.rowset >> ? ? ? ? ?0 ? ? ? ? 5 java.transaction.xa >> ? ? ? ? 39 ? ? ?1848 java.xml >> ? ? ? ? 32 ? ? ? 271 java.xml.crypto >> ? ? ? ? ?3 ? ? ? ?20 jdk.accessibility >> ? ? ? ? ?0 ? ? ? ?21 jdk.attach >> ? ? ? ? ?0 ? ? ? ?18 jdk.charsets >> ? ? ? ? 53 ? ? ? 333 jdk.compiler >> ? ? ? ? ?1 ? ? ? ?76 jdk.crypto.cryptoki >> ? ? ? ? ?0 ? ? ? ?35 jdk.crypto.ec >> ? ? ? ? ?0 ? ? ? ?11 jdk.crypto.mscapi >> ? ? ? ? ?4 ? ? ? ?68 jdk.dynalink >> ? ? ? ? ?0 ? ? ? ? 3 jdk.editpad >> ? ? ? ? 10 ? ? ? 942 jdk.hotspot.agent >> ? ? ? ? ?3 ? ? ? ?56 jdk.httpserver >> ? ? ? ? ?0 ? ? ? ? 5 jdk.incubator.concurrent >> ? ? ? ? ?0 ? ? ? ?50 jdk.incubator.vector >> ? ? ? ? ?0 ? ? ? ? 3 jdk.internal.ed >> ? ? ? ? ?1 ? ? ? ?61 jdk.internal.jvmstat >> ? ? ? ? ?1 ? ? ? 113 jdk.internal.le >> ? ? ? ? ?1 ? ? ? ?52 jdk.internal.opt >> ? ? ? ? ?5 ? ? ? 209 jdk.internal.vm.ci >> ? ? ? ? ?0 ? ? ? ? 1 jdk.internal.vm.compiler >> ? ? ? ? ?0 ? ? ? ? 1 jdk.internal.vm.compiler.management >> ? ? ? ? ?0 ? ? ? ?18 jdk.jartool >> ? ? ? ? ?5 ? ? ? 228 jdk.javadoc >> ? ? ? ? ?0 ? ? ? ?41 jdk.jcmd >> ? ? ? ? 18 ? ? ? ?64 jdk.jconsole >> ? ? ? ? 10 ? ? ? 124 jdk.jdeps >> ? ? ? ? 25 ? ? ? 254 jdk.jdi >> ? ? ? ? ?0 ? ? ? ? 1 jdk.jdwp.agent >> ? ? ? ? ?0 ? ? ? 263 jdk.jfr >> ? ? ? ? ?0 ? ? ? ?77 jdk.jlink >> ? ? ? ? ?1 ? ? ? ?80 jdk.jpackage >> ? ? ? ? ?6 ? ? ? ?88 jdk.jshell >> ? ? ? ? ?0 ? ? ? ? 4 jdk.jsobject >> ? ? ? ? ?1 ? ? ? ?11 jdk.jstatd >> ? ? ? ? ?0 ? ? ? 281 jdk.localedata >> ? ? ? ? ?1 ? ? ? ?25 jdk.management >> ? ? ? ? ?0 ? ? ? ?19 jdk.management.agent >> ? ? ? ? ?0 ? ? ? ?15 jdk.management.jfr >> ? ? ? ? ?0 ? ? ? ?16 jdk.naming.dns >> ? ? ? ? ?0 ? ? ? ? 8 jdk.naming.rmi >> ? ? ? ? ?0 ? ? ? ?11 jdk.net >> ? ? ? ? ?0 ? ? ? ? 2 jdk.nio.mapmode >> ? ? ? ? ?0 ? ? ? ?11 jdk.random >> ? ? ? ? ?0 ? ? ? ?37 jdk.sctp >> ? ? ? ? ?0 ? ? ? ?30 jdk.security.auth >> ? ? ? ? ?0 ? ? ? ?16 jdk.security.jgss >> ? ? ? ? ?0 ? ? ? ? 9 jdk.unsupported >> ? ? ? ? ?0 ? ? ? ? 8 jdk.unsupported.desktop >> ? ? ? ? ?0 ? ? ? ?94 jdk.xml.dom >> ? ? ? ? ?1 ? ? ? ?14 jdk.zipfs >> >> Code available here . >> >> -Archie >> >> -- >> Archie L. Cobbs >