From steff.nicolas at gmail.com Thu Mar 3 02:48:04 2016 From: steff.nicolas at gmail.com (=?UTF-8?Q?St=C3=A9phane_NICOLAS?=) Date: Wed, 2 Mar 2016 18:48:04 -0800 Subject: Invoking code generated by an annotation processor In-Reply-To: References: <56BB9250.20607@oracle.com> <56BBAAD5.7080807@oracle.com> <56BCDAFB.3040107@oracle.com> <56C4C36B.1090506@oracle.com> <56C7B01B.3030609@oracle.com> Message-ID: @jonathan, is there any chance this will get fixed in a future release of javac ? (7 &/or 8+, we actually don't really mind about java 7 on our side). Thx in advance, Stephane 2016-02-19 19:50 GMT-08:00 St?phane NICOLAS : > Thx Joe for this historical precision, I appreciate it. > > I read carefully, and when the specs says : > > If the source code is not syntactically well-formed or has some other > irrecoverable error that could not be removed by the generation of new > types, a model may or may not be provided as a quality of implementation > issue. > > In the 2 cases we highlighted : a static import error or an erroneous > annotation value, it's hard to say here if a source with those errors is > syntactically valid. > > *If we consider it syntactically valid : *then those errors are not > recoverable, as they prevent the annotation processor from running. So > those errors should be reported. And javac does that. But it also reports > all cals to generated code as errors, which is our concern. > > Later, the specs says : > > If a program is syntactically valid but erroneous in some other fashion, > any returned model must have no less information than if all the method > bodies in the program were replaced by "throw new RuntimeException();" > > Here, javac seems to fail in our case as the source code is syntactically > valid, but erroneous in some other fashion, but our generated code is not > assumed to have the form of empty classes, methods etc.. The annotation > processor is simply not run and those errors are reported. > > So, if such errors are not considered syntax errors, javac doesn't follow > the specs. > > *If we consider the program is not syntactically valid : *then we fall in > a loophole of fuzzyness of the specs : a model may or may not be provided > as a quality of implementation issue. It looks like it's not possible to > generate a model that could be passed to annotation processors because of > those errors. Then only those should be reported. > > I tend to think there is something missing on this matter in the specs you > mention : an intermediary class of error. > > Unrecoverable errors : > > - syntax errors are about the form of the program (missing commas, > braces, etc. name it). Those are reported well by javac, and only them are > reported. Perfect > - model building errors : static imports and annotation values errors. > Those are also reported well by javac but they drag along with them error > to generated code, the recoverable errors below. > > Recoverable errors : > > - unknown classes or members of known classes : those are not > considered as error before annotation processing takes place. Which is OK, > but they are reported in case of model building errors, which is our > concern. > > So we can say here, to summarize the problem differently that right after > parsing a source and considering it valid, javac open a list of errors it > finds in the source. It adds all recoverable errors it finds to save them > for a next phase of processing after annotation processing. But at that > very precise moment, it also adds to the error list the non recoverable > model building errors. And if one is found, then all errors are reported, > which is not correct. > > The solution would be to separate model building errors and recoverable > errors right after having considered a program syntactically valid. > Maybe there is an intermediary phase needed to identify strictly non > recoverable model building errors and list them, before building the model > and detecting recoverable errors, which javac handles well. > > S. > > > 2016-02-19 16:15 GMT-08:00 Joseph D. Darcy : > >> Hello, >> >> On 2/17/2016 11:00 AM, Jonathan Gibbons wrote: >> >> St?phane, >> >> Internally, during the period before anno processing, javac categorizes >> errors as recoverable or not, where "recoverable" means "could this error >> conceivably be fixed by running annotation processors, which may generate >> code." >> >> It may be that we have to examine the scenario you describe to decide if >> the error should be marked as recoverable. >> >> >> To provide some additional historical context here, the first generation >> annotation processing of apt in JDK 5 as well as the standardized successor >> of javax.lang.model and javax.annotation.processing in JDK 6 and later >> shared a requirement of having to be able to process incomplete programs. >> In particular, to allow processors to generated missing subclasses and even >> superclasses of the the initial set of sources. >> >> In general, it is impractical to describe exactly how a compiler should >> model an incorrect program. The JSR 269 expert group took some some pains >> to craft a specification would balance the need for predictability while >> also allowing some room for implementation-specific behavior: current >> version of this specification with some refinements [1] over the years: >> >> During annotation processing, operating on incomplete or erroneous >> programs is necessary; however, there are fewer guarantees about the nature >> of the resulting model. If the source code is not syntactically well-formed >> or has some other irrecoverable error that could not be removed by the >> generation of new types, a model may or may not be provided as a quality of >> implementation issue. If a program is syntactically valid but erroneous in >> some other fashion, any returned model must have no less information than >> if all the method bodies in the program were replaced by "throw new >> RuntimeException();". If a program refers to a missing type XYZ, the >> returned model must contain no less information than if the declaration of >> type XYZ were assumed to be "class XYZ {}", "interface XYZ {}", "enum XYZ >> {}", or "@interface XYZ {}". If a program refers to a missing type XYZ> ... ,Kn>, the returned model must contain no less information than if the >> declaration of XYZ were assumed to be "class XYZ {}" or >> "interface XYZ {}" >> >> >> http://docs.oracle.com/javase/8/docs/api/javax/lang/model/element/package-summary.html >> >> As Jon alluded to earlier in this thread, how javac implements this >> aspect of the spec has evolved over time with the current intention to >> forge ahead when generating a missing type could allow the whole set of >> sources to compile successfully. >> >> Cheers, >> >> -Joe >> >> [1] https://bugs.openjdk.java.net/browse/JDK-7003550 >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at oracle.com Thu Mar 3 15:37:45 2016 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 03 Mar 2016 07:37:45 -0800 Subject: Invoking code generated by an annotation processor In-Reply-To: References: <56BB9250.20607@oracle.com> <56BBAAD5.7080807@oracle.com> <56BCDAFB.3040107@oracle.com> <56C4C36B.1090506@oracle.com> <56C7B01B.3030609@oracle.com> Message-ID: <56D85A49.50406@oracle.com> Stephane, I will create an issue to track this. I cannot make promises about fixing this, but if we do, it would be fixed in 9 first, and then maybe backported to 8. -- Jon On 03/02/2016 06:48 PM, St?phane NICOLAS wrote: > @jonathan, is there any chance this will get fixed in a future release > of javac ? (7 &/or 8+, we actually don't really mind about java 7 on > our side). > > Thx in advance, > Stephane > > 2016-02-19 19:50 GMT-08:00 St?phane NICOLAS >: > > Thx Joe for this historical precision, I appreciate it. > > I read carefully, and when the specs says : >> If the source code is not syntactically well-formed or has some >> other irrecoverable error that could not be removed by the >> generation of new types, a model may or may not be provided as a >> quality of implementation issue. > In the 2 cases we highlighted : a static import error or an > erroneous annotation value, it's hard to say here if a source with > those errors is syntactically valid. > > _If we consider it syntactically valid : _then those errors are > not recoverable, as they prevent the annotation processor from > running. So those errors should be reported. And javac does that. > But it also reports all cals to generated code as errors, which is > our concern. > > Later, the specs says : >> If a program is syntactically valid but erroneous in some other >> fashion, any returned model must have no less information than if >> all the method bodies in the program were replaced by "throw new >> RuntimeException();" > Here, javac seems to fail in our case as the source code is > syntactically valid, but erroneous in some other fashion, but our > generated code is not assumed to have the form of empty classes, > methods etc.. The annotation processor is simply not run and those > errors are reported. > > So, if such errors are not considered syntax errors, javac doesn't > follow the specs. > > _If we consider the program is not syntactically valid : _then we > fall in a loophole of fuzzyness of the specs : a model may or may > not be provided as a quality of implementation issue. It looks > like it's not possible to generate a model that could be passed to > annotation processors because of those errors. Then only those > should be reported. > > I tend to think there is something missing on this matter in the > specs you mention : an intermediary class of error. > > Unrecoverable errors : > > * syntax errors are about the form of the program (missing > commas, braces, etc. name it). Those are reported well by > javac, and only them are reported. Perfect > * model building errors : static imports and annotation values > errors. Those are also reported well by javac but they drag > along with them error to generated code, the recoverable > errors below. > > Recoverable errors : > > * unknown classes or members of known classes : those are not > considered as error before annotation processing takes place. > Which is OK, but they are reported in case of model building > errors, which is our concern. > > So we can say here, to summarize the problem differently that > right after parsing a source and considering it valid, javac open > a list of errors it finds in the source. It adds all recoverable > errors it finds to save them for a next phase of processing after > annotation processing. But at that very precise moment, it also > adds to the error list the non recoverable model building errors. > And if one is found, then all errors are reported, which is not > correct. > > The solution would be to separate model building errors and > recoverable errors right after having considered a program > syntactically valid. > Maybe there is an intermediary phase needed to identify strictly > non recoverable model building errors and list them, before > building the model and detecting recoverable errors, which javac > handles well. > > S. > > > 2016-02-19 16:15 GMT-08:00 Joseph D. Darcy >: > > Hello, > > On 2/17/2016 11:00 AM, Jonathan Gibbons wrote: >> St?phane, >> >> Internally, during the period before anno processing, javac >> categorizes errors as recoverable or not, where "recoverable" >> means "could this error conceivably be fixed by running >> annotation processors, which may generate code." >> >> It may be that we have to examine the scenario you describe >> to decide if the error should be marked as recoverable. >> > > To provide some additional historical context here, the first > generation annotation processing of apt in JDK 5 as well as > the standardized successor of javax.lang.model and > javax.annotation.processing in JDK 6 and later shared a > requirement of having to be able to process incomplete > programs. In particular, to allow processors to generated > missing subclasses and even superclasses of the the initial > set of sources. > > In general, it is impractical to describe exactly how a > compiler should model an incorrect program. The JSR 269 expert > group took some some pains to craft a specification would > balance the need for predictability while also allowing some > room for implementation-specific behavior: current version of > this specification with some refinements [1] over the years: > >> During annotation processing, operating on incomplete or >> erroneous programs is necessary; however, there are fewer >> guarantees about the nature of the resulting model. If the >> source code is not syntactically well-formed or has some >> other irrecoverable error that could not be removed by the >> generation of new types, a model may or may not be provided >> as a quality of implementation issue. If a program is >> syntactically valid but erroneous in some other fashion, any >> returned model must have no less information than if all the >> method bodies in the program were replaced by "throw new >> RuntimeException();". If a program refers to a missing type >> XYZ, the returned model must contain no less information than >> if the declaration of type XYZ were assumed to be "class XYZ >> {}", "interface XYZ {}", "enum XYZ {}", or "@interface XYZ >> {}". If a program refers to a missing type XYZ, >> the returned model must contain no less information than if >> the declaration of XYZ were assumed to be "class XYZ> ,Tn> {}" or "interface XYZ {}" > http://docs.oracle.com/javase/8/docs/api/javax/lang/model/element/package-summary.html > > As Jon alluded to earlier in this thread, how javac implements > this aspect of the spec has evolved over time with the current > intention to forge ahead when generating a missing type could > allow the whole set of sources to compile successfully. > > Cheers, > > -Joe > > [1] https://bugs.openjdk.java.net/browse/JDK-7003550 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steff.nicolas at gmail.com Thu Mar 3 15:48:46 2016 From: steff.nicolas at gmail.com (=?UTF-8?Q?St=C3=A9phane_NICOLAS?=) Date: Thu, 3 Mar 2016 07:48:46 -0800 Subject: Invoking code generated by an annotation processor In-Reply-To: <56D85A49.50406@oracle.com> References: <56BB9250.20607@oracle.com> <56BBAAD5.7080807@oracle.com> <56BCDAFB.3040107@oracle.com> <56C4C36B.1090506@oracle.com> <56C7B01B.3030609@oracle.com> <56D85A49.50406@oracle.com> Message-ID: Thx a lot Jon, it's perfect like this. We wil soon pass to java 9 in our android development anyway. If you could link the issue here once you create it, so that we can follow it up, that would be great. S. 2016-03-03 7:37 GMT-08:00 Jonathan Gibbons : > Stephane, > > I will create an issue to track this. I cannot make promises about fixing > this, but if we do, it would be fixed in 9 first, and then maybe backported > to 8. > > -- Jon > > > > On 03/02/2016 06:48 PM, St?phane NICOLAS wrote: > > @jonathan, is there any chance this will get fixed in a future release of > javac ? (7 &/or 8+, we actually don't really mind about java 7 on our > side). > > Thx in advance, > Stephane > > 2016-02-19 19:50 GMT-08:00 St?phane NICOLAS : > >> Thx Joe for this historical precision, I appreciate it. >> >> I read carefully, and when the specs says : >> >> If the source code is not syntactically well-formed or has some other >> irrecoverable error that could not be removed by the generation of new >> types, a model may or may not be provided as a quality of implementation >> issue. >> >> In the 2 cases we highlighted : a static import error or an erroneous >> annotation value, it's hard to say here if a source with those errors is >> syntactically valid. >> >> *If we consider it syntactically valid : *then those errors are not >> recoverable, as they prevent the annotation processor from running. So >> those errors should be reported. And javac does that. But it also reports >> all cals to generated code as errors, which is our concern. >> >> Later, the specs says : >> >> If a program is syntactically valid but erroneous in some other fashion, >> any returned model must have no less information than if all the method >> bodies in the program were replaced by "throw new RuntimeException();" >> >> Here, javac seems to fail in our case as the source code is syntactically >> valid, but erroneous in some other fashion, but our generated code is not >> assumed to have the form of empty classes, methods etc.. The annotation >> processor is simply not run and those errors are reported. >> >> So, if such errors are not considered syntax errors, javac doesn't follow >> the specs. >> >> *If we consider the program is not syntactically valid : *then we fall >> in a loophole of fuzzyness of the specs : a model may or may not be >> provided as a quality of implementation issue. It looks like it's not >> possible to generate a model that could be passed to annotation processors >> because of those errors. Then only those should be reported. >> >> I tend to think there is something missing on this matter in the specs >> you mention : an intermediary class of error. >> >> Unrecoverable errors : >> >> - syntax errors are about the form of the program (missing commas, >> braces, etc. name it). Those are reported well by javac, and only them are >> reported. Perfect >> - model building errors : static imports and annotation values >> errors. Those are also reported well by javac but they drag along with them >> error to generated code, the recoverable errors below. >> >> Recoverable errors : >> >> - unknown classes or members of known classes : those are not >> considered as error before annotation processing takes place. Which is OK, >> but they are reported in case of model building errors, which is our >> concern. >> >> So we can say here, to summarize the problem differently that right after >> parsing a source and considering it valid, javac open a list of errors it >> finds in the source. It adds all recoverable errors it finds to save them >> for a next phase of processing after annotation processing. But at that >> very precise moment, it also adds to the error list the non recoverable >> model building errors. And if one is found, then all errors are reported, >> which is not correct. >> >> The solution would be to separate model building errors and recoverable >> errors right after having considered a program syntactically valid. >> Maybe there is an intermediary phase needed to identify strictly non >> recoverable model building errors and list them, before building the model >> and detecting recoverable errors, which javac handles well. >> >> S. >> >> >> 2016-02-19 16:15 GMT-08:00 Joseph D. Darcy : >> >>> Hello, >>> >>> On 2/17/2016 11:00 AM, Jonathan Gibbons wrote: >>> >>> St?phane, >>> >>> Internally, during the period before anno processing, javac categorizes >>> errors as recoverable or not, where "recoverable" means "could this error >>> conceivably be fixed by running annotation processors, which may generate >>> code." >>> >>> It may be that we have to examine the scenario you describe to decide if >>> the error should be marked as recoverable. >>> >>> >>> To provide some additional historical context here, the first generation >>> annotation processing of apt in JDK 5 as well as the standardized successor >>> of javax.lang.model and javax.annotation.processing in JDK 6 and later >>> shared a requirement of having to be able to process incomplete programs. >>> In particular, to allow processors to generated missing subclasses and even >>> superclasses of the the initial set of sources. >>> >>> In general, it is impractical to describe exactly how a compiler should >>> model an incorrect program. The JSR 269 expert group took some some pains >>> to craft a specification would balance the need for predictability while >>> also allowing some room for implementation-specific behavior: current >>> version of this specification with some refinements [1] over the years: >>> >>> During annotation processing, operating on incomplete or erroneous >>> programs is necessary; however, there are fewer guarantees about the nature >>> of the resulting model. If the source code is not syntactically well-formed >>> or has some other irrecoverable error that could not be removed by the >>> generation of new types, a model may or may not be provided as a quality of >>> implementation issue. If a program is syntactically valid but erroneous in >>> some other fashion, any returned model must have no less information than >>> if all the method bodies in the program were replaced by "throw new >>> RuntimeException();". If a program refers to a missing type XYZ, the >>> returned model must contain no less information than if the declaration of >>> type XYZ were assumed to be "class XYZ {}", "interface XYZ {}", "enum XYZ >>> {}", or "@interface XYZ {}". If a program refers to a missing type XYZ>> ... ,Kn>, the returned model must contain no less information than if the >>> declaration of XYZ were assumed to be "class XYZ {}" or >>> "interface XYZ {}" >>> >>> >>> http://docs.oracle.com/javase/8/docs/api/javax/lang/model/element/package-summary.html >>> >>> As Jon alluded to earlier in this thread, how javac implements this >>> aspect of the spec has evolved over time with the current intention to >>> forge ahead when generating a missing type could allow the whole set of >>> sources to compile successfully. >>> >>> Cheers, >>> >>> -Joe >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-7003550 >>> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at oracle.com Thu Mar 3 16:24:07 2016 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 03 Mar 2016 08:24:07 -0800 Subject: Invoking code generated by an annotation processor In-Reply-To: References: <56BB9250.20607@oracle.com> <56BBAAD5.7080807@oracle.com> <56BCDAFB.3040107@oracle.com> <56C4C36B.1090506@oracle.com> <56C7B01B.3030609@oracle.com> Message-ID: <56D86527.2070709@oracle.com> To pick up on this, briefly, to clarify, > it's hard to say here if a source with those errors is syntactically > valid. No, it's not hard. "Syntactically valid" simply means that javac can parse the source code and build a syntax tree. Informally, you can read this as saying, "can javac build a structural representation of the source code, such that it is reasonable to proceed with further analysis." If javac can't even get to first base, then it will report errors and give up. -- Jon On 02/19/2016 07:50 PM, St?phane NICOLAS wrote: > Thx Joe for this historical precision, I appreciate it. > > I read carefully, and when the specs says : >> If the source code is not syntactically well-formed or has some other >> irrecoverable error that could not be removed by the generation of >> new types, a model may or may not be provided as a quality of >> implementation issue. > In the 2 cases we highlighted : a static import error or an erroneous > annotation value, it's hard to say here if a source with those errors > is syntactically valid. > > _If we consider it syntactically valid : _then those errors are not > recoverable, as they prevent the annotation processor from running. So > those errors should be reported. And javac does that. But it also > reports all cals to generated code as errors, which is our concern. > > Later, the specs says : >> If a program is syntactically valid but erroneous in some other >> fashion, any returned model must have no less information than if all >> the method bodies in the program were replaced by "throw new >> RuntimeException();" > Here, javac seems to fail in our case as the source code is > syntactically valid, but erroneous in some other fashion, but our > generated code is not assumed to have the form of empty classes, > methods etc.. The annotation processor is simply not run and those > errors are reported. > > So, if such errors are not considered syntax errors, javac doesn't > follow the specs. > > _If we consider the program is not syntactically valid : _then we fall > in a loophole of fuzzyness of the specs : a model may or may not be > provided as a quality of implementation issue. It looks like it's not > possible to generate a model that could be passed to annotation > processors because of those errors. Then only those should be reported. > > I tend to think there is something missing on this matter in the specs > you mention : an intermediary class of error. > > Unrecoverable errors : > > * syntax errors are about the form of the program (missing commas, > braces, etc. name it). Those are reported well by javac, and only > them are reported. Perfect > * model building errors : static imports and annotation values > errors. Those are also reported well by javac but they drag along > with them error to generated code, the recoverable errors below. > > Recoverable errors : > > * unknown classes or members of known classes : those are not > considered as error before annotation processing takes place. > Which is OK, but they are reported in case of model building > errors, which is our concern. > > So we can say here, to summarize the problem differently that right > after parsing a source and considering it valid, javac open a list of > errors it finds in the source. It adds all recoverable errors it finds > to save them for a next phase of processing after annotation > processing. But at that very precise moment, it also adds to the error > list the non recoverable model building errors. And if one is found, > then all errors are reported, which is not correct. > > The solution would be to separate model building errors and > recoverable errors right after having considered a program > syntactically valid. > Maybe there is an intermediary phase needed to identify strictly non > recoverable model building errors and list them, before building the > model and detecting recoverable errors, which javac handles well. > > S. > > > 2016-02-19 16:15 GMT-08:00 Joseph D. Darcy >: > > Hello, > > On 2/17/2016 11:00 AM, Jonathan Gibbons wrote: >> St?phane, >> >> Internally, during the period before anno processing, javac >> categorizes errors as recoverable or not, where "recoverable" >> means "could this error conceivably be fixed by running >> annotation processors, which may generate code." >> >> It may be that we have to examine the scenario you describe to >> decide if the error should be marked as recoverable. >> > > To provide some additional historical context here, the first > generation annotation processing of apt in JDK 5 as well as the > standardized successor of javax.lang.model and > javax.annotation.processing in JDK 6 and later shared a > requirement of having to be able to process incomplete programs. > In particular, to allow processors to generated missing subclasses > and even superclasses of the the initial set of sources. > > In general, it is impractical to describe exactly how a compiler > should model an incorrect program. The JSR 269 expert group took > some some pains to craft a specification would balance the need > for predictability while also allowing some room for > implementation-specific behavior: current version of this > specification with some refinements [1] over the years: > >> During annotation processing, operating on incomplete or >> erroneous programs is necessary; however, there are fewer >> guarantees about the nature of the resulting model. If the source >> code is not syntactically well-formed or has some other >> irrecoverable error that could not be removed by the generation >> of new types, a model may or may not be provided as a quality of >> implementation issue. If a program is syntactically valid but >> erroneous in some other fashion, any returned model must have no >> less information than if all the method bodies in the program >> were replaced by "throw new RuntimeException();". If a program >> refers to a missing type XYZ, the returned model must contain no >> less information than if the declaration of type XYZ were assumed >> to be "class XYZ {}", "interface XYZ {}", "enum XYZ {}", or >> "@interface XYZ {}". If a program refers to a missing type >> XYZ, the returned model must contain no less >> information than if the declaration of XYZ were assumed to be >> "class XYZ {}" or "interface XYZ {}" > http://docs.oracle.com/javase/8/docs/api/javax/lang/model/element/package-summary.html > > As Jon alluded to earlier in this thread, how javac implements > this aspect of the spec has evolved over time with the current > intention to forge ahead when generating a missing type could > allow the whole set of sources to compile successfully. > > Cheers, > > -Joe > > [1] https://bugs.openjdk.java.net/browse/JDK-7003550 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at oracle.com Thu Mar 3 16:25:07 2016 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 03 Mar 2016 08:25:07 -0800 Subject: Invoking code generated by an annotation processor In-Reply-To: References: <56BB9250.20607@oracle.com> <56BBAAD5.7080807@oracle.com> <56BCDAFB.3040107@oracle.com> <56C4C36B.1090506@oracle.com> <56C7B01B.3030609@oracle.com> <56D85A49.50406@oracle.com> Message-ID: <56D86563.4050000@oracle.com> JDK-8151194 https://bugs.openjdk.java.net/browse/JDK-8151194 -- Jon On 03/03/2016 07:48 AM, St?phane NICOLAS wrote: > Thx a lot Jon, it's perfect like this. We wil soon pass to java 9 in > our android development anyway. > If you could link the issue here once you create it, so that we can > follow it up, that would be great. > > S. > > 2016-03-03 7:37 GMT-08:00 Jonathan Gibbons > >: > > Stephane, > > I will create an issue to track this. I cannot make promises > about fixing this, but if we do, it would be fixed in 9 first, and > then maybe backported to 8. > > -- Jon > > > > On 03/02/2016 06:48 PM, St?phane NICOLAS wrote: >> @jonathan, is there any chance this will get fixed in a future >> release of javac ? (7 &/or 8+, we actually don't really mind >> about java 7 on our side). >> >> Thx in advance, >> Stephane >> >> 2016-02-19 19:50 GMT-08:00 St?phane NICOLAS >> >: >> >> Thx Joe for this historical precision, I appreciate it. >> >> I read carefully, and when the specs says : >>> If the source code is not syntactically well-formed or has >>> some other irrecoverable error that could not be removed by >>> the generation of new types, a model may or may not be >>> provided as a quality of implementation issue. >> In the 2 cases we highlighted : a static import error or an >> erroneous annotation value, it's hard to say here if a source >> with those errors is syntactically valid. >> >> _If we consider it syntactically valid : _then those errors >> are not recoverable, as they prevent the annotation processor >> from running. So those errors should be reported. And javac >> does that. But it also reports all cals to generated code as >> errors, which is our concern. >> >> Later, the specs says : >>> If a program is syntactically valid but erroneous in some >>> other fashion, any returned model must have no less >>> information than if all the method bodies in the program >>> were replaced by "throw new RuntimeException();" >> Here, javac seems to fail in our case as the source code is >> syntactically valid, but erroneous in some other fashion, but >> our generated code is not assumed to have the form of empty >> classes, methods etc.. The annotation processor is simply not >> run and those errors are reported. >> >> So, if such errors are not considered syntax errors, javac >> doesn't follow the specs. >> >> _If we consider the program is not syntactically valid : >> _then we fall in a loophole of fuzzyness of the specs : a >> model may or may not be provided as a quality of >> implementation issue. It looks like it's not possible to >> generate a model that could be passed to annotation >> processors because of those errors. Then only those should be >> reported. >> >> I tend to think there is something missing on this matter in >> the specs you mention : an intermediary class of error. >> >> Unrecoverable errors : >> >> * syntax errors are about the form of the program (missing >> commas, braces, etc. name it). Those are reported well by >> javac, and only them are reported. Perfect >> * model building errors : static imports and annotation >> values errors. Those are also reported well by javac but >> they drag along with them error to generated code, the >> recoverable errors below. >> >> Recoverable errors : >> >> * unknown classes or members of known classes : those are >> not considered as error before annotation processing >> takes place. Which is OK, but they are reported in case >> of model building errors, which is our concern. >> >> So we can say here, to summarize the problem differently that >> right after parsing a source and considering it valid, javac >> open a list of errors it finds in the source. It adds all >> recoverable errors it finds to save them for a next phase of >> processing after annotation processing. But at that very >> precise moment, it also adds to the error list the non >> recoverable model building errors. And if one is found, then >> all errors are reported, which is not correct. >> >> The solution would be to separate model building errors and >> recoverable errors right after having considered a program >> syntactically valid. >> Maybe there is an intermediary phase needed to identify >> strictly non recoverable model building errors and list them, >> before building the model and detecting recoverable errors, >> which javac handles well. >> >> S. >> >> >> 2016-02-19 16:15 GMT-08:00 Joseph D. Darcy >> >: >> >> Hello, >> >> On 2/17/2016 11:00 AM, Jonathan Gibbons wrote: >>> St?phane, >>> >>> Internally, during the period before anno processing, >>> javac categorizes errors as recoverable or not, where >>> "recoverable" means "could this error conceivably be >>> fixed by running annotation processors, which may >>> generate code." >>> >>> It may be that we have to examine the scenario you >>> describe to decide if the error should be marked as >>> recoverable. >>> >> >> To provide some additional historical context here, the >> first generation annotation processing of apt in JDK 5 as >> well as the standardized successor of javax.lang.model >> and javax.annotation.processing in JDK 6 and later shared >> a requirement of having to be able to process incomplete >> programs. In particular, to allow processors to generated >> missing subclasses and even superclasses of the the >> initial set of sources. >> >> In general, it is impractical to describe exactly how a >> compiler should model an incorrect program. The JSR 269 >> expert group took some some pains to craft a >> specification would balance the need for predictability >> while also allowing some room for implementation-specific >> behavior: current version of this specification with some >> refinements [1] over the years: >> >>> During annotation processing, operating on incomplete or >>> erroneous programs is necessary; however, there are >>> fewer guarantees about the nature of the resulting >>> model. If the source code is not syntactically >>> well-formed or has some other irrecoverable error that >>> could not be removed by the generation of new types, a >>> model may or may not be provided as a quality of >>> implementation issue. If a program is syntactically >>> valid but erroneous in some other fashion, any returned >>> model must have no less information than if all the >>> method bodies in the program were replaced by "throw new >>> RuntimeException();". If a program refers to a missing >>> type XYZ, the returned model must contain no less >>> information than if the declaration of type XYZ were >>> assumed to be "class XYZ {}", "interface XYZ {}", "enum >>> XYZ {}", or "@interface XYZ {}". If a program refers to >>> a missing type XYZ, the returned model must >>> contain no less information than if the declaration of >>> XYZ were assumed to be "class XYZ {}" or >>> "interface XYZ {}" >> http://docs.oracle.com/javase/8/docs/api/javax/lang/model/element/package-summary.html >> >> As Jon alluded to earlier in this thread, how javac >> implements this aspect of the spec has evolved over time >> with the current intention to forge ahead when generating >> a missing type could allow the whole set of sources to >> compile successfully. >> >> Cheers, >> >> -Joe >> >> [1] https://bugs.openjdk.java.net/browse/JDK-7003550 >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleksey.shipilev at oracle.com Mon Mar 7 18:46:27 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 7 Mar 2016 21:46:27 +0300 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class Message-ID: <56DDCC83.8010601@oracle.com> Hi, There seems to be a corner case in our current String concatenation handling: we are recording the exact argument types to call StringConcatFactory with, to allow JDK to use that information for optimization. This works well, until we meet a private class. Then, we get an IllegalAccessError while trying to execute a bootstrap method referencing an inaccessible class: https://bugs.openjdk.java.net/browse/JDK-8151223 This does not happen with "old" concat flavor, because we are calling StringBuilder.append(Object) there, which does not leak away the private type. Luckily, we are not mandated to always call StringConcatFactory with an exact type. We can detect inaccessible classes during compilation, and use the closest accessible super-type instead. We could just strip to java.lang.Object in those cases, but it seems better to sharpen the type to the first accessible/non-ambiguous class/interface up the hierarchy. Webrev: http://cr.openjdk.java.net/~shade/8151223/webrev.01 Testing: failing test, new regression test; JDK java/lang/String; Langtools com/,jdk/,tools/ Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From maurizio.cimadamore at oracle.com Mon Mar 7 19:19:11 2016 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 7 Mar 2016 19:19:11 +0000 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DDCC83.8010601@oracle.com> References: <56DDCC83.8010601@oracle.com> Message-ID: <56DDD42F.2060707@oracle.com> Hi Alexm thanks for the patch; the idea is generally sound - I have few questions about the implementation of the 'sharpestAccessible' method when it comes to some javac quirks: * note that when you recur on supertypes, you will end up with Type.noType, a type whose tag is NONE. In other words, supertype(Object) == Type.noType. This means that the current patch is (i) risking a bit when calling isAccessible on Type.noType - I'm not 100% sure if that's supported - might just work accidentallly and (ii) the subsequent test: if (!sharpestClass.equals(syms.objectType)) { + return sharpestClass; + } Is redundant (and wrong) - in other words, you will always exit the loop with some type that is != Object (I think), which makes the interface part useless. * when it comes to interface sharpening, your code prefers D to B when calling the sharpening routine on InaccessibleA: class InaccessibleA extends InaccessibeB implements InaccessibleC { } class InaccessibleB implements B { } interface InaccessibleC implements InaccessibleD { } interface InaccessibleD extends D { } interface B { } interface D { } In other words, the routine does a depth-first search on superclasses, followed by a depth-first search on interfaces. Would it make sense to have a routine in which all direct supertypes are treated equally? I.e. you look for an accessible supertype in the list of direct supertypes (Types.directSupertypes) - if one is found you return it, if none is found you repeat the check on the superclass. * I'm not sure what happens when the input of the routine is not a classtype? I.e. an array or a type-var? Now, it seems like type-vars are probably erased before this point (or you'd have issue in the current impl too) - but arrays can definitively come up. Now, testing an array type doesn't make a lot of sense, since the array class is fictional; and array supertypes are not meaningful - i.e. Serializable and Cloneable - you don't want to end up with such types in the indy. I think array types should be special cased, so that the sharpening logic works on the element type. This means that this routine probably needs to become a type visitor (Types.SimpleTypeVisitor) - which has different behaviours for different type shapes. The rest looks good. Maurizio On 07/03/16 18:46, Aleksey Shipilev wrote: > Hi, > > There seems to be a corner case in our current String concatenation > handling: we are recording the exact argument types to call > StringConcatFactory with, to allow JDK to use that information for > optimization. This works well, until we meet a private class. Then, we > get an IllegalAccessError while trying to execute a bootstrap method > referencing an inaccessible class: > https://bugs.openjdk.java.net/browse/JDK-8151223 > > This does not happen with "old" concat flavor, because we are calling > StringBuilder.append(Object) there, which does not leak away the private > type. > > Luckily, we are not mandated to always call StringConcatFactory with an > exact type. We can detect inaccessible classes during compilation, and > use the closest accessible super-type instead. We could just strip to > java.lang.Object in those cases, but it seems better to sharpen the type > to the first accessible/non-ambiguous class/interface up the hierarchy. > > Webrev: > http://cr.openjdk.java.net/~shade/8151223/webrev.01 > > Testing: failing test, new regression test; JDK java/lang/String; > Langtools com/,jdk/,tools/ > > Thanks, > -Aleksey > From forax at univ-mlv.fr Mon Mar 7 21:33:28 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 7 Mar 2016 22:33:28 +0100 (CET) Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DDCC83.8010601@oracle.com> References: <56DDCC83.8010601@oracle.com> Message-ID: <1311109652.872339.1457386408692.JavaMail.zimbra@u-pem.fr> Hi Aleksey, I think that trying to get the most specific type in that case is a good idea but i think that trying to find the most specific interface(s) is useless. Calling a method on an interface will always be resolved at runtime, moreover, it's not possible to create a default method overriding Object.toString() so i believe that calling toString() on an interface is equivalent to calling toString on Object in term of type information. cheers, R?mi ----- Mail original ----- > De: "Aleksey Shipilev" > ?: compiler-dev at openjdk.java.net > Envoy?: Lundi 7 Mars 2016 19:46:27 > Objet: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class > > Hi, > > There seems to be a corner case in our current String concatenation > handling: we are recording the exact argument types to call > StringConcatFactory with, to allow JDK to use that information for > optimization. This works well, until we meet a private class. Then, we > get an IllegalAccessError while trying to execute a bootstrap method > referencing an inaccessible class: > https://bugs.openjdk.java.net/browse/JDK-8151223 > > This does not happen with "old" concat flavor, because we are calling > StringBuilder.append(Object) there, which does not leak away the private > type. > > Luckily, we are not mandated to always call StringConcatFactory with an > exact type. We can detect inaccessible classes during compilation, and > use the closest accessible super-type instead. We could just strip to > java.lang.Object in those cases, but it seems better to sharpen the type > to the first accessible/non-ambiguous class/interface up the hierarchy. > > Webrev: > http://cr.openjdk.java.net/~shade/8151223/webrev.01 > > Testing: failing test, new regression test; JDK java/lang/String; > Langtools com/,jdk/,tools/ > > Thanks, > -Aleksey > > From aleksey.shipilev at oracle.com Mon Mar 7 22:45:50 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 8 Mar 2016 01:45:50 +0300 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DDD42F.2060707@oracle.com> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> Message-ID: <56DE049E.7030601@oracle.com> Hi Maurizio (and Remy)! Thanks for reviews, see the comments below, and webrev link at the bottom: On 03/07/2016 10:19 PM, Maurizio Cimadamore wrote: > * note that when you recur on supertypes, you will end up with > Type.noType, a type whose tag is NONE. In other words, supertype(Object) > == Type.noType. This means that the current patch is (i) risking a bit > when calling isAccessible on Type.noType - I'm not 100% sure if that's > supported - might just work accidentallly and (ii) the subsequent test: > > if (!sharpestClass.equals(syms.objectType)) { > + return sharpestClass; > + } > > > Is redundant (and wrong) - in other words, you will always exit the loop > with some type that is != Object (I think), which makes the interface > part useless. Type sharpestClass = originalType; while (!isAccessible(sharpestClass)) { sharpestClass = types.supertype(sharpestClass); } if (!sharpestClass.equals(syms.objectType)) { return sharpestClass; } I don't think it was redundant: Object.class passes isAccessible check, and we exit the loop either with a concrete subclass of Object, or Object itself. But I agree we may want to check for Tag.NONE too. > * when it comes to interface sharpening, your code prefers D to B when > calling the sharpening routine on InaccessibleA: > > class InaccessibleA extends InaccessibeB implements InaccessibleC { } > class InaccessibleB implements B { } > interface InaccessibleC implements InaccessibleD { } > interface InaccessibleD extends D { } > interface B { } > interface D { } > > > In other words, the routine does a depth-first search on superclasses, > followed by a depth-first search on interfaces. Would it make sense to > have a routine in which all direct supertypes are treated equally? I.e. > you look for an accessible supertype in the list of direct supertypes > (Types.directSupertypes) - if one is found you return it, if none is > found you repeat the check on the superclass. I think it returning either B or D is plausible: it is rather ambiguous what is the sharpest interface here. If you define "sharpness" as the number of extend/implement chains from the type to accessible interface, then B is sharpest. But, I think this steps into an odd territory when a difference in "nesting" depth over two hierarchy branches defines the outcome. Current routine was more stable in this regard: we only walk interfaces from the leaf type, and bail if no unique accessible interface is met. > * I'm not sure what happens when the input of the routine is not a > classtype? I.e. an array or a type-var? Now, it seems like type-vars are > probably erased before this point (or you'd have issue in the current > impl too) - but arrays can definitively come up. Good catch! I extended the test to arrays, and it failed roughly the same way. Well, I think interface sharpening backfired (and the benefit of having it is slim, as Remi noted), so let's make a step back, and bail to java.lang.Object when private class is detected; and only make attempt to sharpen the class, not interfaces: http://cr.openjdk.java.net/~shade/8151223/webrev.02/ Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From john.r.rose at oracle.com Tue Mar 8 02:34:46 2016 From: john.r.rose at oracle.com (John Rose) Date: Mon, 7 Mar 2016 18:34:46 -0800 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <1311109652.872339.1457386408692.JavaMail.zimbra@u-pem.fr> References: <56DDCC83.8010601@oracle.com> <1311109652.872339.1457386408692.JavaMail.zimbra@u-pem.fr> Message-ID: <9230308D-98FB-4EE1-A283-B4F12EA0F5A1@oracle.com> The more cleverly we use static information when generating bytecodes, the more murky will be our binary compatibility story. I suggest you rely mainly on locally available metadata (caller class and its nest) and on stable packages (java.lang). Any uncertainty should send the argument type all the way to Object. Dynamic type profiling will pick up some of the dropped bits and we won't have binary compat. puzzles to solve down the road. Like the present bug. ? John > On Mar 7, 2016, at 1:33 PM, Remi Forax wrote: > > Hi Aleksey, > I think that trying to get the most specific type in that case is a good idea but > i think that trying to find the most specific interface(s) is useless. > > Calling a method on an interface will always be resolved at runtime, > moreover, it's not possible to create a default method overriding Object.toString() > so i believe that calling toString() on an interface is equivalent to calling toString on Object in term of type information. > > cheers, > R?mi > > ----- Mail original ----- >> De: "Aleksey Shipilev" >> ?: compiler-dev at openjdk.java.net >> Envoy?: Lundi 7 Mars 2016 19:46:27 >> Objet: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class >> >> Hi, >> >> There seems to be a corner case in our current String concatenation >> handling: we are recording the exact argument types to call >> StringConcatFactory with, to allow JDK to use that information for >> optimization. This works well, until we meet a private class. Then, we >> get an IllegalAccessError while trying to execute a bootstrap method >> referencing an inaccessible class: >> https://bugs.openjdk.java.net/browse/JDK-8151223 >> >> This does not happen with "old" concat flavor, because we are calling >> StringBuilder.append(Object) there, which does not leak away the private >> type. >> >> Luckily, we are not mandated to always call StringConcatFactory with an >> exact type. We can detect inaccessible classes during compilation, and >> use the closest accessible super-type instead. We could just strip to >> java.lang.Object in those cases, but it seems better to sharpen the type >> to the first accessible/non-ambiguous class/interface up the hierarchy. >> >> Webrev: >> http://cr.openjdk.java.net/~shade/8151223/webrev.01 >> >> Testing: failing test, new regression test; JDK java/lang/String; >> Langtools com/,jdk/,tools/ >> >> Thanks, >> -Aleksey >> >> From forax at univ-mlv.fr Tue Mar 8 08:29:14 2016 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Tue, 8 Mar 2016 09:29:14 +0100 (CET) Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <9230308D-98FB-4EE1-A283-B4F12EA0F5A1@oracle.com> References: <56DDCC83.8010601@oracle.com> <1311109652.872339.1457386408692.JavaMail.zimbra@u-pem.fr> <9230308D-98FB-4EE1-A283-B4F12EA0F5A1@oracle.com> Message-ID: <145895390.1199126.1457425754516.JavaMail.zimbra@u-pem.fr> Hi John, ----- Mail original ----- > De: "John Rose" > ?: "Remi Forax" > Cc: "Aleksey Shipilev" , compiler-dev at openjdk.java.net > Envoy?: Mardi 8 Mars 2016 03:34:46 > Objet: Re: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class > > The more cleverly we use static information when generating bytecodes, the > more murky will be our binary compatibility story. I suggest you rely mainly > on locally available metadata (caller class and its nest) and on stable > packages (java.lang). Any uncertainty should send the argument type all the > way to Object. why java.lang and not org.jruby, groovy.lang, etc. In an ideal world (let's forget security just for now) every classes should be considered as the same by the compiler/JDK/JIT. While i can agree that instead of trying to find the most specific type in the hierarchy, if the class is not public, it should be replaced by Object. I can not agree that a public type should be erased to Object just because its package is not java.lang. if it's a public type, it's a public type after all. so IMO, the rule should be, only non accessible classes should be erased to Object. I would also erased all interfaces, even the public ones because having an interface as type doesn't provide any benefit compared to using Object. > Dynamic type profiling will pick up some of the dropped bits > and we won't have binary compat. puzzles to solve down the road. Like the > present bug. Dynamic type profiling may have the cost of a dynamic typecheck. trying to not expose developpers to their responsibility (maintaining backward compat of public types) is not solving any puzzles, and BTW, talking about puzzles, i believe that jigsaw also goes in that direction by making the dependencies more explicit (my secret hope is to have a build tool that will not let you make a backward incompatible change without bumping the major version of a module). > > ? John > R?mi > > On Mar 7, 2016, at 1:33 PM, Remi Forax wrote: > > > > Hi Aleksey, > > I think that trying to get the most specific type in that case is a good > > idea but > > i think that trying to find the most specific interface(s) is useless. > > > > Calling a method on an interface will always be resolved at runtime, > > moreover, it's not possible to create a default method overriding > > Object.toString() > > so i believe that calling toString() on an interface is equivalent to > > calling toString on Object in term of type information. > > > > cheers, > > R?mi > > > > ----- Mail original ----- > >> De: "Aleksey Shipilev" > >> ?: compiler-dev at openjdk.java.net > >> Envoy?: Lundi 7 Mars 2016 19:46:27 > >> Objet: RFR (S) 8151223: String concatenation fails with implicit > >> toString() on package-private class > >> > >> Hi, > >> > >> There seems to be a corner case in our current String concatenation > >> handling: we are recording the exact argument types to call > >> StringConcatFactory with, to allow JDK to use that information for > >> optimization. This works well, until we meet a private class. Then, we > >> get an IllegalAccessError while trying to execute a bootstrap method > >> referencing an inaccessible class: > >> https://bugs.openjdk.java.net/browse/JDK-8151223 > >> > >> This does not happen with "old" concat flavor, because we are calling > >> StringBuilder.append(Object) there, which does not leak away the private > >> type. > >> > >> Luckily, we are not mandated to always call StringConcatFactory with an > >> exact type. We can detect inaccessible classes during compilation, and > >> use the closest accessible super-type instead. We could just strip to > >> java.lang.Object in those cases, but it seems better to sharpen the type > >> to the first accessible/non-ambiguous class/interface up the hierarchy. > >> > >> Webrev: > >> http://cr.openjdk.java.net/~shade/8151223/webrev.01 > >> > >> Testing: failing test, new regression test; JDK java/lang/String; > >> Langtools com/,jdk/,tools/ > >> > >> Thanks, > >> -Aleksey > >> > >> > From aleksey.shipilev at oracle.com Tue Mar 8 08:53:06 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 8 Mar 2016 11:53:06 +0300 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <9230308D-98FB-4EE1-A283-B4F12EA0F5A1@oracle.com> References: <56DDCC83.8010601@oracle.com> <1311109652.872339.1457386408692.JavaMail.zimbra@u-pem.fr> <9230308D-98FB-4EE1-A283-B4F12EA0F5A1@oracle.com> Message-ID: <56DE92F2.6030701@oracle.com> On 03/08/2016 05:34 AM, John Rose wrote: > The more cleverly we use static information when generating > bytecodes, the more murky will be our binary compatibility story. I > suggest you rely mainly on locally available metadata (caller class > and its nest) and on stable packages (java.lang). Any uncertainty > should send the argument type all the way to Object. Dynamic type > profiling will pick up some of the dropped bits and we won't have > binary compat. puzzles to solve down the road. Like the present bug. Not really sure what you are suggesting to do right now, John. I think sharpening to the most specific accessible class is okay, because JLS Chapter 13 "Binary Compatibility" clearly points out that changing the visibility of classes/members to less access may be binary incompatible. IOW, once you compiled against an accessible class, it is a maintainers' headache to preserve compatibility. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From maurizio.cimadamore at oracle.com Tue Mar 8 11:12:37 2016 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 8 Mar 2016 11:12:37 +0000 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DE049E.7030601@oracle.com> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> <56DE049E.7030601@oracle.com> Message-ID: <56DEB3A5.3060003@oracle.com> On 07/03/16 22:45, Aleksey Shipilev wrote: > I don't think it was redundant: Object.class passes isAccessible check, > and we exit the loop either with a concrete subclass of Object, or > Object itself. But I agree we may want to check for Tag.NONE too. Sorry - you are obviously right here - Object is always accessible, so your code was indeed correct. Feel free to change it back if you like! Maurizio From maurizio.cimadamore at oracle.com Tue Mar 8 11:14:43 2016 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 8 Mar 2016 11:14:43 +0000 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DEB3A5.3060003@oracle.com> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> <56DE049E.7030601@oracle.com> <56DEB3A5.3060003@oracle.com> Message-ID: <56DEB423.8060503@oracle.com> On 08/03/16 11:12, Maurizio Cimadamore wrote: > > > On 07/03/16 22:45, Aleksey Shipilev wrote: >> I don't think it was redundant: Object.class passes isAccessible check, >> and we exit the loop either with a concrete subclass of Object, or >> Object itself. But I agree we may want to check for Tag.NONE too. > Sorry - you are obviously right here - Object is always accessible, so > your code was indeed correct. Feel free to change it back if you like! > > Maurizio Also one last nit: Type componentType = ((Type.ArrayType) originalType).getComponentType(); getComponentType is an API method - there's an internal routine: Types.elemtype which works on any type, and thus you could avoid the cast. Maurizio From maurizio.cimadamore at oracle.com Tue Mar 8 11:25:16 2016 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 8 Mar 2016 11:25:16 +0000 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DE92F2.6030701@oracle.com> References: <56DDCC83.8010601@oracle.com> <1311109652.872339.1457386408692.JavaMail.zimbra@u-pem.fr> <9230308D-98FB-4EE1-A283-B4F12EA0F5A1@oracle.com> <56DE92F2.6030701@oracle.com> Message-ID: <56DEB69C.3030305@oracle.com> Here's an extreme case where sharpening can end up in some weird case in a binary compatible scenario; quoting chapter 13: "Changing the direct superclass or the set of direct superinterfaces of a class type will not break compatibility with pre-existing binaries, provided that the total set of superclasses or superinterfaces, respectively, of the class type loses no members." So, let's start with: public class A { } class Inaccessible extends A { } Here, the sharpening method would lift 'Inaccessible' to 'A'. Now, let's change that as follows: public class A { } public class B extends A { } class Inaccessible extends B { } Now, the sharpening method will lift 'Inaccessible' to 'B'. Is this an issue? While I don't think it's a big issue, it also means that the runtime cannot totally rely on the static information put there by javac, as this information might as well be stale (as in the above case, where a classfile says 'A' where in fact the sharpest type is 'B'). Of course this can happen even w/o accessibility woes, so we should probably not worry about this? Maurizio On 08/03/16 08:53, Aleksey Shipilev wrote: > On 03/08/2016 05:34 AM, John Rose wrote: >> The more cleverly we use static information when generating >> bytecodes, the more murky will be our binary compatibility story. I >> suggest you rely mainly on locally available metadata (caller class >> and its nest) and on stable packages (java.lang). Any uncertainty >> should send the argument type all the way to Object. Dynamic type >> profiling will pick up some of the dropped bits and we won't have >> binary compat. puzzles to solve down the road. Like the present bug. > Not really sure what you are suggesting to do right now, John. > > I think sharpening to the most specific accessible class is okay, > because JLS Chapter 13 "Binary Compatibility" clearly points out that > changing the visibility of classes/members to less access may be binary > incompatible. IOW, once you compiled against an accessible class, it is > a maintainers' headache to preserve compatibility. > > -Aleksey > From aleksey.shipilev at oracle.com Tue Mar 8 13:09:44 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 8 Mar 2016 16:09:44 +0300 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DEB69C.3030305@oracle.com> References: <56DDCC83.8010601@oracle.com> <1311109652.872339.1457386408692.JavaMail.zimbra@u-pem.fr> <9230308D-98FB-4EE1-A283-B4F12EA0F5A1@oracle.com> <56DE92F2.6030701@oracle.com> <56DEB69C.3030305@oracle.com> Message-ID: <56DECF18.1030203@oracle.com> On 03/08/2016 02:25 PM, Maurizio Cimadamore wrote: > Here's an extreme case where sharpening can end up in some weird case in > a binary compatible scenario; quoting chapter 13: > > "Changing the direct superclass or the set of direct superinterfaces of > a class type will not break compatibility with pre-existing binaries, > provided that the total set of superclasses or superinterfaces, > respectively, of the class type loses no members." > > So, let's start with: > > public class A { } > > class Inaccessible extends A { } > > Here, the sharpening method would lift 'Inaccessible' to 'A'. > > Now, let's change that as follows: > > public class A { } > public class B extends A { } > > class Inaccessible extends B { } > > Now, the sharpening method will lift 'Inaccessible' to 'B'. > > Is this an issue? While I don't think it's a big issue, it also means > that the runtime cannot totally rely on the static information put there > by javac, as this information might as well be stale (as in the above > case, where a classfile says 'A' where in fact the sharpest type is > 'B'). Of course this can happen even w/o accessibility woes, so we > should probably not worry about this? To be fair, lifting from the inaccessible class to some other class already loses "sharpness". There are shades of gray in what's the exact class chosen between java.lang.Object and the leaf inaccessible class. IOW, we are talking about heuristics here, and a heuristic still blindly choosing A after B had appeared is well-behaved. In yet another words, the type information in BSM call is already a guess -- still a useful one. Current patch basically says that *some* type information is still better than the absolute absence of it. The ISC code would still happily work with A. Recompilation would resharpen to B. Now, the design goal for ISC was to come up with a stable bytecode that would not force users to recompile their Java programs when we change the String concat strategy. Whether that should be extended to binary-compatible changes in concat inputs, like in the example above, is debatable. I'd say no; you face the entire realm of new performance issues when you extend the class hierarchy in a binary compatible manner (think new cases in type profiling), and this is yet another little thing. Of course, since we are facing this issue with a inaccessible classes only (which are, arguably, very rare in string concats), we might as well strip them to Object. But, I am reluctant of giving up type information without being forced to do so. Aside: spelling out the actual types before calling toString on them in the concat stub is actually better for optimizers, because it bounds the set of possible classes there at the .toString() invocation site. Ultimately, for public leaf classes, you may skip type profiling and/or inline caching altogether (CHA does it already). The "smart compiler card" has less power here, because ISC is basically an attempt to downplay the importance of complicated optimizing compiler, in favor of smarter handling in JDK. Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From aleksey.shipilev at oracle.com Tue Mar 8 13:59:22 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 8 Mar 2016 16:59:22 +0300 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DEB423.8060503@oracle.com> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> <56DE049E.7030601@oracle.com> <56DEB3A5.3060003@oracle.com> <56DEB423.8060503@oracle.com> Message-ID: <56DEDABA.8090703@oracle.com> On 03/08/2016 02:14 PM, Maurizio Cimadamore wrote: > On 08/03/16 11:12, Maurizio Cimadamore wrote: >> >> >> On 07/03/16 22:45, Aleksey Shipilev wrote: >>> I don't think it was redundant: Object.class passes isAccessible check, >>> and we exit the loop either with a concrete subclass of Object, or >>> Object itself. But I agree we may want to check for Tag.NONE too. >> Sorry - you are obviously right here - Object is always accessible, so >> your code was indeed correct. Feel free to change it back if you like! >> >> Maurizio > Also one last nit: > > Type componentType = ((Type.ArrayType) originalType).getComponentType(); > > > getComponentType is an API method - there's an internal routine: > Types.elemtype which works on any type, and thus you could avoid the cast. Thanks Maurizio, fixed both nits: http://cr.openjdk.java.net/~shade/8151223/webrev.03/ -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From forax at univ-mlv.fr Tue Mar 8 15:09:13 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 8 Mar 2016 16:09:13 +0100 (CET) Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DEDABA.8090703@oracle.com> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> <56DE049E.7030601@oracle.com> <56DEB3A5.3060003@oracle.com> <56DEB423.8060503@oracle.com> <56DEDABA.8090703@oracle.com> Message-ID: <2119214783.1686414.1457449753940.JavaMail.zimbra@u-pem.fr> Thinking a little bit more about backward compatibility, there is a way to make the string concatenation meta-protocol "more binary compatible". Use Objects in the descriptor of invokedynamic and send a string as bootstrap constant encoding the declared types provided by javac. At runtime, in the bootstrap method, when decoding the declared types, if a type from the declared types is not accessible, substitute it by Object. Then in the generated bytecode, insert a cast to the declared type (or use asType() for strategies based on the method handles). Because we do the casts upfront before calling toString(), we don't loose the precise types (obviously, we hope here that the code that does the concatenation will be inlined with the code the rest of the code otherwise, we will pay the price of a cast). That's said, i'm not sure it's a good idea to implement that, Java is not expected to use erased type at runtime to help backward compatibility. regards, R?mi ----- Mail original ----- > De: "Aleksey Shipilev" > ?: "Maurizio Cimadamore" , compiler-dev at openjdk.java.net > Envoy?: Mardi 8 Mars 2016 14:59:22 > Objet: Re: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class > > On 03/08/2016 02:14 PM, Maurizio Cimadamore wrote: > > On 08/03/16 11:12, Maurizio Cimadamore wrote: > >> > >> > >> On 07/03/16 22:45, Aleksey Shipilev wrote: > >>> I don't think it was redundant: Object.class passes isAccessible check, > >>> and we exit the loop either with a concrete subclass of Object, or > >>> Object itself. But I agree we may want to check for Tag.NONE too. > >> Sorry - you are obviously right here - Object is always accessible, so > >> your code was indeed correct. Feel free to change it back if you like! > >> > >> Maurizio > > Also one last nit: > > > > Type componentType = ((Type.ArrayType) originalType).getComponentType(); > > > > > > getComponentType is an API method - there's an internal routine: > > Types.elemtype which works on any type, and thus you could avoid the cast. > > Thanks Maurizio, fixed both nits: > http://cr.openjdk.java.net/~shade/8151223/webrev.03/ > > -Aleksey > > > From aleksey.shipilev at oracle.com Tue Mar 8 16:44:59 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 8 Mar 2016 19:44:59 +0300 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <2119214783.1686414.1457449753940.JavaMail.zimbra@u-pem.fr> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> <56DE049E.7030601@oracle.com> <56DEB3A5.3060003@oracle.com> <56DEB423.8060503@oracle.com> <56DEDABA.8090703@oracle.com> <2119214783.1686414.1457449753940.JavaMail.zimbra@u-pem.fr> Message-ID: <56DF018B.7040504@oracle.com> Alas, I think this defeats the purpose of having the MethodType spelled out in BSM. It is probably better to fix the corner case with inaccessible classes, and then use the existing BSM machinery as intended. -Aleksey On 03/08/2016 06:09 PM, Remi Forax wrote: > Thinking a little bit more about backward compatibility, > there is a way to make the string concatenation meta-protocol "more binary compatible". > > Use Objects in the descriptor of invokedynamic and send a string as bootstrap constant encoding the declared types provided by javac. > At runtime, in the bootstrap method, when decoding the declared types, if a type from the declared types is not accessible, substitute it by Object. > Then in the generated bytecode, insert a cast to the declared type (or use asType() for strategies based on the method handles). > > Because we do the casts upfront before calling toString(), we don't loose the precise types > (obviously, we hope here that the code that does the concatenation will be inlined with the code the rest of the code > otherwise, we will pay the price of a cast). > > That's said, i'm not sure it's a good idea to implement that, > Java is not expected to use erased type at runtime to help backward compatibility. > > regards, > R?mi > > ----- Mail original ----- >> De: "Aleksey Shipilev" >> ?: "Maurizio Cimadamore" , compiler-dev at openjdk.java.net >> Envoy?: Mardi 8 Mars 2016 14:59:22 >> Objet: Re: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class >> >> On 03/08/2016 02:14 PM, Maurizio Cimadamore wrote: >>> On 08/03/16 11:12, Maurizio Cimadamore wrote: >>>> >>>> >>>> On 07/03/16 22:45, Aleksey Shipilev wrote: >>>>> I don't think it was redundant: Object.class passes isAccessible check, >>>>> and we exit the loop either with a concrete subclass of Object, or >>>>> Object itself. But I agree we may want to check for Tag.NONE too. >>>> Sorry - you are obviously right here - Object is always accessible, so >>>> your code was indeed correct. Feel free to change it back if you like! >>>> >>>> Maurizio >>> Also one last nit: >>> >>> Type componentType = ((Type.ArrayType) originalType).getComponentType(); >>> >>> >>> getComponentType is an API method - there's an internal routine: >>> Types.elemtype which works on any type, and thus you could avoid the cast. >> >> Thanks Maurizio, fixed both nits: >> http://cr.openjdk.java.net/~shade/8151223/webrev.03/ >> >> -Aleksey >> >> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From forax at univ-mlv.fr Tue Mar 8 16:53:10 2016 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Tue, 8 Mar 2016 17:53:10 +0100 (CET) Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DF018B.7040504@oracle.com> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> <56DE049E.7030601@oracle.com> <56DEB3A5.3060003@oracle.com> <56DEB423.8060503@oracle.com> <56DEDABA.8090703@oracle.com> <2119214783.1686414.1457449753940.JavaMail.zimbra@u-pem.fr> <56DF018B.7040504@oracle.com> Message-ID: <482832142.1886797.1457455990847.JavaMail.zimbra@u-pem.fr> yes, R?mi ----- Mail original ----- > De: "Aleksey Shipilev" > ?: "Remi Forax" > Cc: "Maurizio Cimadamore" , compiler-dev at openjdk.java.net, "John Rose" > > Envoy?: Mardi 8 Mars 2016 17:44:59 > Objet: Re: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class > > Alas, I think this defeats the purpose of having the MethodType spelled > out in BSM. It is probably better to fix the corner case with > inaccessible classes, and then use the existing BSM machinery as intended. > > -Aleksey > > On 03/08/2016 06:09 PM, Remi Forax wrote: > > Thinking a little bit more about backward compatibility, > > there is a way to make the string concatenation meta-protocol "more binary > > compatible". > > > > Use Objects in the descriptor of invokedynamic and send a string as > > bootstrap constant encoding the declared types provided by javac. > > At runtime, in the bootstrap method, when decoding the declared types, if a > > type from the declared types is not accessible, substitute it by Object. > > Then in the generated bytecode, insert a cast to the declared type (or use > > asType() for strategies based on the method handles). > > > > Because we do the casts upfront before calling toString(), we don't loose > > the precise types > > (obviously, we hope here that the code that does the concatenation will be > > inlined with the code the rest of the code > > otherwise, we will pay the price of a cast). > > > > That's said, i'm not sure it's a good idea to implement that, > > Java is not expected to use erased type at runtime to help backward > > compatibility. > > > > regards, > > R?mi > > > > ----- Mail original ----- > >> De: "Aleksey Shipilev" > >> ?: "Maurizio Cimadamore" , > >> compiler-dev at openjdk.java.net > >> Envoy?: Mardi 8 Mars 2016 14:59:22 > >> Objet: Re: RFR (S) 8151223: String concatenation fails with implicit > >> toString() on package-private class > >> > >> On 03/08/2016 02:14 PM, Maurizio Cimadamore wrote: > >>> On 08/03/16 11:12, Maurizio Cimadamore wrote: > >>>> > >>>> > >>>> On 07/03/16 22:45, Aleksey Shipilev wrote: > >>>>> I don't think it was redundant: Object.class passes isAccessible check, > >>>>> and we exit the loop either with a concrete subclass of Object, or > >>>>> Object itself. But I agree we may want to check for Tag.NONE too. > >>>> Sorry - you are obviously right here - Object is always accessible, so > >>>> your code was indeed correct. Feel free to change it back if you like! > >>>> > >>>> Maurizio > >>> Also one last nit: > >>> > >>> Type componentType = ((Type.ArrayType) originalType).getComponentType(); > >>> > >>> > >>> getComponentType is an API method - there's an internal routine: > >>> Types.elemtype which works on any type, and thus you could avoid the > >>> cast. > >> > >> Thanks Maurizio, fixed both nits: > >> http://cr.openjdk.java.net/~shade/8151223/webrev.03/ > >> > >> -Aleksey > >> > >> > >> > > > From forax at univ-mlv.fr Tue Mar 8 16:55:45 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 8 Mar 2016 17:55:45 +0100 (CET) Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DEDABA.8090703@oracle.com> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> <56DE049E.7030601@oracle.com> <56DEB3A5.3060003@oracle.com> <56DEB423.8060503@oracle.com> <56DEDABA.8090703@oracle.com> Message-ID: <1571391149.1887417.1457456145833.JavaMail.zimbra@u-pem.fr> Thumb up for me, just a minor nit: i think that sharpestAccessible should be package private and not protected. regards, R?mi ----- Mail original ----- > De: "Aleksey Shipilev" > ?: "Maurizio Cimadamore" , compiler-dev at openjdk.java.net > Envoy?: Mardi 8 Mars 2016 14:59:22 > Objet: Re: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class > > On 03/08/2016 02:14 PM, Maurizio Cimadamore wrote: > > On 08/03/16 11:12, Maurizio Cimadamore wrote: > >> > >> > >> On 07/03/16 22:45, Aleksey Shipilev wrote: > >>> I don't think it was redundant: Object.class passes isAccessible check, > >>> and we exit the loop either with a concrete subclass of Object, or > >>> Object itself. But I agree we may want to check for Tag.NONE too. > >> Sorry - you are obviously right here - Object is always accessible, so > >> your code was indeed correct. Feel free to change it back if you like! > >> > >> Maurizio > > Also one last nit: > > > > Type componentType = ((Type.ArrayType) originalType).getComponentType(); > > > > > > getComponentType is an API method - there's an internal routine: > > Types.elemtype which works on any type, and thus you could avoid the cast. > > Thanks Maurizio, fixed both nits: > http://cr.openjdk.java.net/~shade/8151223/webrev.03/ > > -Aleksey > > > From aleksey.shipilev at oracle.com Tue Mar 8 17:27:00 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 8 Mar 2016 20:27:00 +0300 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <1571391149.1887417.1457456145833.JavaMail.zimbra@u-pem.fr> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> <56DE049E.7030601@oracle.com> <56DEB3A5.3060003@oracle.com> <56DEB423.8060503@oracle.com> <56DEDABA.8090703@oracle.com> <1571391149.1887417.1457456145833.JavaMail.zimbra@u-pem.fr> Message-ID: <56DF0B64.6040502@oracle.com> On 03/08/2016 07:55 PM, Remi Forax wrote: > Thumb up for me, > just a minor nit: i think that sharpestAccessible should be package private and not protected. Good, here: http://cr.openjdk.java.net/~shade/8151223/webrev.04/ Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From maurizio.cimadamore at oracle.com Tue Mar 8 18:26:14 2016 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 8 Mar 2016 18:26:14 +0000 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DF0B64.6040502@oracle.com> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> <56DE049E.7030601@oracle.com> <56DEB3A5.3060003@oracle.com> <56DEB423.8060503@oracle.com> <56DEDABA.8090703@oracle.com> <1571391149.1887417.1457456145833.JavaMail.zimbra@u-pem.fr> <56DF0B64.6040502@oracle.com> Message-ID: <56DF1946.5040102@oracle.com> Thumbs up for me too. Maurizio On 08/03/16 17:27, Aleksey Shipilev wrote: > On 03/08/2016 07:55 PM, Remi Forax wrote: >> Thumb up for me, >> just a minor nit: i think that sharpestAccessible should be package private and not protected. > Good, here: > http://cr.openjdk.java.net/~shade/8151223/webrev.04/ > > Thanks, > -Aleksey > > From volker.simonis at gmail.com Wed Mar 9 09:26:34 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 9 Mar 2016 10:26:34 +0100 Subject: RFR(XS): 8150632: jdk.jshell.TaskFactory should use jdk.Version to check for java.specification.version In-Reply-To: References: Message-ID: PING! Can I please have a review for this trivial change? Thank you and best regards, Volker On Mon, Feb 29, 2016 at 10:20 AM, Volker Simonis wrote: > Also forwarding to kulla-dev which may be more appropriate than compiler-dev. > > > ---------- Forwarded message ---------- > From: Volker Simonis > Date: Thu, Feb 25, 2016 at 1:11 PM > Subject: RFR(XS): 8150632: jdk.jshell.TaskFactory should use > jdk.Version to check for java.specification.version > To: compiler-dev at openjdk.java.net > > > Hi, > > can I please have a review for the following small patch which fixes a > problem with the jshell java.specification.version check: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/8150632/ > https://bugs.openjdk.java.net/browse/JDK-8150632 > > Currently, jdk.jshell.TaskFactory does a hard check against > java.specification.version being "9". Until JDK-8149519 [1] will be > resolved, this check will fail for any newer Java release (i.e. > 9.0.0.1). > > But even if JDK-8149519 will be resolved, I think the compare should > actually check that java.specification.version is greater or equal to > "9". This can be easily done with the new jdk.Version API. > > Thank you and best regards, > Volker > > [1] https://bugs.openjdk.java.net/browse/JDK-8149519 From aleksey.shipilev at oracle.com Wed Mar 9 09:52:46 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 9 Mar 2016 12:52:46 +0300 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DF1946.5040102@oracle.com> References: <56DDCC83.8010601@oracle.com> <56DDD42F.2060707@oracle.com> <56DE049E.7030601@oracle.com> <56DEB3A5.3060003@oracle.com> <56DEB423.8060503@oracle.com> <56DEDABA.8090703@oracle.com> <1571391149.1887417.1457456145833.JavaMail.zimbra@u-pem.fr> <56DF0B64.6040502@oracle.com> <56DF1946.5040102@oracle.com> Message-ID: <56DFF26E.3010400@oracle.com> Thanks Maurizio and Remi. Pushed. -Aleksey On 03/08/2016 09:26 PM, Maurizio Cimadamore wrote: > Thumbs up for me too. > > Maurizio > > On 08/03/16 17:27, Aleksey Shipilev wrote: >> On 03/08/2016 07:55 PM, Remi Forax wrote: >>> Thumb up for me, >>> just a minor nit: i think that sharpestAccessible should be package >>> private and not protected. >> Good, here: >> http://cr.openjdk.java.net/~shade/8151223/webrev.04/ >> >> Thanks, >> -Aleksey >> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From john.r.rose at oracle.com Wed Mar 9 09:53:16 2016 From: john.r.rose at oracle.com (John Rose) Date: Wed, 9 Mar 2016 01:53:16 -0800 (PST) Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class Message-ID: <2876a241-a423-44c3-b7bb-d64d9d3dfee8@default> Accessibility of the chosen class is only part of the headache you are signing up for. Java binary compatibility rules also allow classes to change their super type hierarchy over time. So when you search up the super chain you are traversing links that may not be present when the code is run. You are getting static answers to questions about type relations which may have changed when the code runs. This is one of JLS Chapter 13's favorite tricks. You do not want it as your enemy. Basically, any static query done in javac has to be looked at with skepticism: It can change, if the answer to the query does not come from either the current compilation unit, or else from the Java language runtime selected by the "-target" option of javac. (Remi, this is why java.lang is special. The Groovy compiler is welcome to make static assumptions about groovy.lang or whatever, but javac binds only to java.lang.) My specific recommendation is not to ask questions with non-local answers in the static analysis of string operands. Fail up to Object, or (if you can and you wish) to a java.lang type. HTH ? John ----- Original Message ----- From: aleksey.shipilev at oracle.com To: john.r.rose at oracle.com, forax at univ-mlv.fr Cc: compiler-dev at openjdk.java.net Sent: Tuesday, March 8, 2016 12:53:09 AM GMT -08:00 US/Canada Pacific Subject: Re: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class On 03/08/2016 05:34 AM, John Rose wrote: > The more cleverly we use static information when generating > bytecodes, the more murky will be our binary compatibility story. I > suggest you rely mainly on locally available metadata (caller class > and its nest) and on stable packages (java.lang). Any uncertainty > should send the argument type all the way to Object. Dynamic type > profiling will pick up some of the dropped bits and we won't have > binary compat. puzzles to solve down the road. Like the present bug. Not really sure what you are suggesting to do right now, John. I think sharpening to the most specific accessible class is okay, because JLS Chapter 13 "Binary Compatibility" clearly points out that changing the visibility of classes/members to less access may be binary incompatible. IOW, once you compiled against an accessible class, it is a maintainers' headache to preserve compatibility. From aleksey.shipilev at oracle.com Wed Mar 9 10:18:27 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 9 Mar 2016 13:18:27 +0300 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <2876a241-a423-44c3-b7bb-d64d9d3dfee8@default> References: <2876a241-a423-44c3-b7bb-d64d9d3dfee8@default> Message-ID: <56DFF873.5000409@oracle.com> On 03/09/2016 12:53 PM, John Rose wrote: > Accessibility of the chosen class is only part of the headache you > are signing up for. > > Java binary compatibility rules also allow classes to change their > super type hierarchy over time. > > So when you search up the super chain you are traversing links that > may not be present when the code is run. You are getting static > answers to questions about type relations which may have changed when > the code runs. This is one of JLS Chapter 13's favorite tricks. You > do not want it as your enemy. > > Basically, any static query done in javac has to be looked at with > skepticism: It can change, if the answer to the query does not come > from either the current compilation unit, or else from the Java > language runtime selected by the "-target" option of javac. > > (Remi, this is why java.lang is special. The Groovy compiler is > welcome to make static assumptions about groovy.lang or whatever, but > javac binds only to java.lang.) > > My specific recommendation is not to ask questions with non-local > answers in the static analysis of string operands. Fail up to > Object, or (if you can and you wish) to a java.lang type. Oops, too late for this particular bug, but we can file and work the followup. So, is this what you deem the safest route without sacrificing much of the type information? 1) Emit precise class for accessible classes; 2) Emit Object for inaccessible classes; 3) Do (1),(2) for array component types. Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From aleksey.shipilev at oracle.com Wed Mar 9 10:41:00 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 9 Mar 2016 13:41:00 +0300 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DFF873.5000409@oracle.com> References: <2876a241-a423-44c3-b7bb-d64d9d3dfee8@default> <56DFF873.5000409@oracle.com> Message-ID: <56DFFDBC.6040506@oracle.com> On 03/09/2016 01:18 PM, Aleksey Shipilev wrote: > On 03/09/2016 12:53 PM, John Rose wrote: >> Java binary compatibility rules also allow classes to change their >> super type hierarchy over time. >> >> So when you search up the super chain you are traversing links that >> may not be present when the code is run. You are getting static >> answers to questions about type relations which may have changed when >> the code runs. This is one of JLS Chapter 13's favorite tricks. You >> do not want it as your enemy. As Maurizio points out to me privately, Ch. 13 actually spells out what hierarchy changes are acceptable. Changing the hierarchy is such a way that a leaf inaccessible class is not a subclass of a sharpened accessible class anymore is *not* a binary compatible change, as per: "Changing the direct superclass or the set of direct superinterfaces of a class type will not break compatibility with pre-existing binaries, provided that the total set of superclasses or superinterfaces, respectively, of the class type loses no members." I can get the reluctance of doing this for inaccessible classes, because some handwaving may offer that "private" classes are "okay" to be changed in incompatible manner. Therefore, I'd say we surrender the discussion about the sharpest type for a very narrow corner case, and go semi-pessimistic with: > Oops, too late for this particular bug, but we can file and work the > followup. So, is this what you deem the safest route without sacrificing > much of the type information? > > 1) Emit precise class for accessible classes; > 2) Emit Object for inaccessible classes; > 3) Do (1),(2) for array component types. I am totally fine with current "sharpen to most specific accessible superclass" too. Cheers, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From maurizio.cimadamore at oracle.com Wed Mar 9 10:42:04 2016 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 9 Mar 2016 10:42:04 +0000 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <2876a241-a423-44c3-b7bb-d64d9d3dfee8@default> References: <2876a241-a423-44c3-b7bb-d64d9d3dfee8@default> Message-ID: <56DFFDFC.8010703@oracle.com> I'm not 100% sure about the missing links - here's what Chapter 13 says: "Changing the direct superclass or the set of direct superinterfaces of a class type will not break compatibility with pre-existing binaries, provided that the total set of superclasses or superinterfaces, respectively, of the class type loses no members." Which seems to imply that it is ok to change the supertypes if subtyping relationships are preserved - that is, if S is a supertype of C, S must continue to remain so even in the updated hierarchy. Under those assumptions what the current implementation does doesn't look all that terrible, and I think (as pointed out yesterday) the worst thing can happen in a binary compatible scenario is that you end up picking a type that is not the sharpest possible - oh well. If you do have missing links, then you have a binary incompatible change (i.e. the set of supertypes is not preserved), and if you have a binary incompatible change, what exactly are we trying to save here? There are already millions of other things that can fail as a result of a missing link - as explicitly pointed out in the spec: "If a change to the direct superclass or the set of direct superinterfaces results in any class or interface no longer being a superclass or superinterface, respectively, then linkage errors may result if pre-existing binaries are loaded with the binary of the modified class. Such changes are not recommended for widely distributed classes." So, why should this case be treated specially? Maurizio On 09/03/16 09:53, John Rose wrote: > Accessibility of the chosen class is only part of the headache you are signing up for. > > Java binary compatibility rules also allow classes to change their super type hierarchy over time. > > So when you search up the super chain you are traversing links that may not be present when the code is run. You are getting static answers to questions about type relations which may have changed when the code runs. This is one of JLS Chapter 13's favorite tricks. You do not want it as your enemy. > > Basically, any static query done in javac has to be looked at with skepticism: It can change, if the answer to the query does not come from either the current compilation unit, or else from the Java language runtime selected by the "-target" option of javac. > > (Remi, this is why java.lang is special. The Groovy compiler is welcome to make static assumptions about groovy.lang or whatever, but javac binds only to java.lang.) > > My specific recommendation is not to ask questions with non-local answers in the static analysis of string operands. Fail up to Object, or (if you can and you wish) to a java.lang type. > > > HTH > ? John > > ----- Original Message ----- > From: aleksey.shipilev at oracle.com > To: john.r.rose at oracle.com, forax at univ-mlv.fr > Cc: compiler-dev at openjdk.java.net > Sent: Tuesday, March 8, 2016 12:53:09 AM GMT -08:00 US/Canada Pacific > Subject: Re: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class > > On 03/08/2016 05:34 AM, John Rose wrote: >> The more cleverly we use static information when generating >> bytecodes, the more murky will be our binary compatibility story. I >> suggest you rely mainly on locally available metadata (caller class >> and its nest) and on stable packages (java.lang). Any uncertainty >> should send the argument type all the way to Object. Dynamic type >> profiling will pick up some of the dropped bits and we won't have >> binary compat. puzzles to solve down the road. Like the present bug. > Not really sure what you are suggesting to do right now, John. > > I think sharpening to the most specific accessible class is okay, > because JLS Chapter 13 "Binary Compatibility" clearly points out that > changing the visibility of classes/members to less access may be binary > incompatible. IOW, once you compiled against an accessible class, it is > a maintainers' headache to preserve compatibility. From aleksey.shipilev at oracle.com Wed Mar 9 13:07:27 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 9 Mar 2016 16:07:27 +0300 Subject: RFR (S) 8151516: test/tools/javac/TestIndyStringConcat depends on runtime JDK details Message-ID: <56E0200F.4040801@oracle.com> Hi, I would like to clean up a basic TestIndyStringConcat test that validates ISC is enabled with different sources/targets and hidden compiler options. Alas, it makes that with prodding the stacktraces from JDK concat stub, which is implementation-specific: https://bugs.openjdk.java.net/browse/JDK-8151516 The solution is to look up the actual invokedynamic instructions: http://cr.openjdk.java.net/~shade/8151516/webrev.01/ (I have also added @clean tags in the related test) Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From maurizio.cimadamore at oracle.com Wed Mar 9 14:00:14 2016 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 9 Mar 2016 14:00:14 +0000 Subject: RFR (S) 8151516: test/tools/javac/TestIndyStringConcat depends on runtime JDK details In-Reply-To: <56E0200F.4040801@oracle.com> References: <56E0200F.4040801@oracle.com> Message-ID: <56E02C6E.7010403@oracle.com> Looks good. I've just noted that your test(s) are missing the @bug tag in the header. Please add one, and mention both this JBS issue and the one you fixed with your original patch. I.e. add * @bug 8151516 8151223 Maurizio On 09/03/16 13:07, Aleksey Shipilev wrote: > Hi, > > I would like to clean up a basic TestIndyStringConcat test that > validates ISC is enabled with different sources/targets and hidden > compiler options. Alas, it makes that with prodding the stacktraces from > JDK concat stub, which is implementation-specific: > https://bugs.openjdk.java.net/browse/JDK-8151516 > > The solution is to look up the actual invokedynamic instructions: > http://cr.openjdk.java.net/~shade/8151516/webrev.01/ > > (I have also added @clean tags in the related test) > > Thanks, > -Aleksey > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleksey.shipilev at oracle.com Wed Mar 9 14:06:33 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 9 Mar 2016 17:06:33 +0300 Subject: RFR (S) 8151516: test/tools/javac/TestIndyStringConcat depends on runtime JDK details In-Reply-To: <56E02C6E.7010403@oracle.com> References: <56E0200F.4040801@oracle.com> <56E02C6E.7010403@oracle.com> Message-ID: <56E02DE9.2010002@oracle.com> On 03/09/2016 05:00 PM, Maurizio Cimadamore wrote: > I've just noted that your test(s) are missing the @bug tag in the > header. Please add one, and mention both this JBS issue and the one you > fixed with your original patch. I.e. add > > * @bug 8151516 8151223 Yes, thanks; also added the original JEP integration bug. Here: http://cr.openjdk.java.net/~shade/8151516/webrev.02/ -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From forax at univ-mlv.fr Wed Mar 9 14:17:02 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 9 Mar 2016 15:17:02 +0100 (CET) Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <56DFFDFC.8010703@oracle.com> References: <2876a241-a423-44c3-b7bb-d64d9d3dfee8@default> <56DFFDFC.8010703@oracle.com> Message-ID: <531729471.2654969.1457533022406.JavaMail.zimbra@u-pem.fr> I agree with Maurizio (and Aleksey). R?mi ----- Mail original ----- > De: "Maurizio Cimadamore" > ?: "John Rose" , "aleksey shipilev" > Cc: compiler-dev at openjdk.java.net > Envoy?: Mercredi 9 Mars 2016 11:42:04 > Objet: Re: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class > > I'm not 100% sure about the missing links - here's what Chapter 13 says: > > "Changing the direct superclass or the set of direct superinterfaces of > a class type will not break compatibility with pre-existing binaries, > provided that the total set of superclasses or superinterfaces, > respectively, of the class type loses no members." > > Which seems to imply that it is ok to change the supertypes if subtyping > relationships are preserved - that is, if S is a supertype of C, S must > continue to remain so even in the updated hierarchy. > > Under those assumptions what the current implementation does doesn't > look all that terrible, and I think (as pointed out yesterday) the worst > thing can happen in a binary compatible scenario is that you end up > picking a type that is not the sharpest possible - oh well. > > If you do have missing links, then you have a binary incompatible change > (i.e. the set of supertypes is not preserved), and if you have a binary > incompatible change, what exactly are we trying to save here? There are > already millions of other things that can fail as a result of a missing > link - as explicitly pointed out in the spec: > > "If a change to the direct superclass or the set of direct > superinterfaces results in any class or interface no longer being a > superclass or superinterface, respectively, then linkage errors may > result if pre-existing binaries are loaded with the binary of the > modified class. Such changes are not recommended for widely distributed > classes." > > So, why should this case be treated specially? > > Maurizio > > On 09/03/16 09:53, John Rose wrote: > > Accessibility of the chosen class is only part of the headache you are > > signing up for. > > > > Java binary compatibility rules also allow classes to change their super > > type hierarchy over time. > > > > So when you search up the super chain you are traversing links that may not > > be present when the code is run. You are getting static answers to > > questions about type relations which may have changed when the code runs. > > This is one of JLS Chapter 13's favorite tricks. You do not want it as > > your enemy. > > > > Basically, any static query done in javac has to be looked at with > > skepticism: It can change, if the answer to the query does not come from > > either the current compilation unit, or else from the Java language > > runtime selected by the "-target" option of javac. > > > > (Remi, this is why java.lang is special. The Groovy compiler is welcome to > > make static assumptions about groovy.lang or whatever, but javac binds > > only to java.lang.) > > > > My specific recommendation is not to ask questions with non-local answers > > in the static analysis of string operands. Fail up to Object, or (if you > > can and you wish) to a java.lang type. > > > > > > HTH > > ? John > > > > ----- Original Message ----- > > From: aleksey.shipilev at oracle.com > > To: john.r.rose at oracle.com, forax at univ-mlv.fr > > Cc: compiler-dev at openjdk.java.net > > Sent: Tuesday, March 8, 2016 12:53:09 AM GMT -08:00 US/Canada Pacific > > Subject: Re: RFR (S) 8151223: String concatenation fails with implicit > > toString() on package-private class > > > > On 03/08/2016 05:34 AM, John Rose wrote: > >> The more cleverly we use static information when generating > >> bytecodes, the more murky will be our binary compatibility story. I > >> suggest you rely mainly on locally available metadata (caller class > >> and its nest) and on stable packages (java.lang). Any uncertainty > >> should send the argument type all the way to Object. Dynamic type > >> profiling will pick up some of the dropped bits and we won't have > >> binary compat. puzzles to solve down the road. Like the present bug. > > Not really sure what you are suggesting to do right now, John. > > > > I think sharpening to the most specific accessible class is okay, > > because JLS Chapter 13 "Binary Compatibility" clearly points out that > > changing the visibility of classes/members to less access may be binary > > incompatible. IOW, once you compiled against an accessible class, it is > > a maintainers' headache to preserve compatibility. > > From maurizio.cimadamore at oracle.com Wed Mar 9 14:29:49 2016 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 9 Mar 2016 14:29:49 +0000 Subject: RFR (S) 8151516: test/tools/javac/TestIndyStringConcat depends on runtime JDK details In-Reply-To: <56E02DE9.2010002@oracle.com> References: <56E0200F.4040801@oracle.com> <56E02C6E.7010403@oracle.com> <56E02DE9.2010002@oracle.com> Message-ID: <56E0335D.10002@oracle.com> Looks good - thanks Maurizio On 09/03/16 14:06, Aleksey Shipilev wrote: > On 03/09/2016 05:00 PM, Maurizio Cimadamore wrote: >> I've just noted that your test(s) are missing the @bug tag in the >> header. Please add one, and mention both this JBS issue and the one you >> fixed with your original patch. I.e. add >> >> * @bug 8151516 8151223 > Yes, thanks; also added the original JEP integration bug. > > Here: > http://cr.openjdk.java.net/~shade/8151516/webrev.02/ > > -Aleksey > From aleksey.shipilev at oracle.com Wed Mar 9 15:32:41 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 9 Mar 2016 18:32:41 +0300 Subject: RFR (S) 8151516: test/tools/javac/TestIndyStringConcat depends on runtime JDK details In-Reply-To: <56E0335D.10002@oracle.com> References: <56E0200F.4040801@oracle.com> <56E02C6E.7010403@oracle.com> <56E02DE9.2010002@oracle.com> <56E0335D.10002@oracle.com> Message-ID: <56E04219.20807@oracle.com> Excellent! Pushed. -Aleksey On 03/09/2016 05:29 PM, Maurizio Cimadamore wrote: > Looks good - thanks > > Maurizio > > On 09/03/16 14:06, Aleksey Shipilev wrote: >> On 03/09/2016 05:00 PM, Maurizio Cimadamore wrote: >>> I've just noted that your test(s) are missing the @bug tag in the >>> header. Please add one, and mention both this JBS issue and the one you >>> fixed with your original patch. I.e. add >>> >>> * @bug 8151516 8151223 >> Yes, thanks; also added the original JEP integration bug. >> >> Here: >> http://cr.openjdk.java.net/~shade/8151516/webrev.02/ >> >> -Aleksey >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From robert.field at oracle.com Wed Mar 9 16:59:29 2016 From: robert.field at oracle.com (Robert Field) Date: Wed, 09 Mar 2016 08:59:29 -0800 Subject: RFR(XS): 8150632: jdk.jshell.TaskFactory should use jdk.Version to check for java.specification.version In-Reply-To: References: Message-ID: <1535c51a968.2765.4011f3a8741ca2aabce58b8b81f42d24@oracle.com> Thumbs up! Thanks, Robert On March 9, 2016 1:26:56 AM Volker Simonis wrote: > PING! > > Can I please have a review for this trivial change? > > Thank you and best regards, > Volker > > > On Mon, Feb 29, 2016 at 10:20 AM, Volker Simonis > wrote: >> Also forwarding to kulla-dev which may be more appropriate than compiler-dev. >> >> >> ---------- Forwarded message ---------- >> From: Volker Simonis >> Date: Thu, Feb 25, 2016 at 1:11 PM >> Subject: RFR(XS): 8150632: jdk.jshell.TaskFactory should use >> jdk.Version to check for java.specification.version >> To: compiler-dev at openjdk.java.net >> >> >> Hi, >> >> can I please have a review for the following small patch which fixes a >> problem with the jshell java.specification.version check: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2016/8150632/ >> https://bugs.openjdk.java.net/browse/JDK-8150632 >> >> Currently, jdk.jshell.TaskFactory does a hard check against >> java.specification.version being "9". Until JDK-8149519 [1] will be >> resolved, this check will fail for any newer Java release (i.e. >> 9.0.0.1). >> >> But even if JDK-8149519 will be resolved, I think the compare should >> actually check that java.specification.version is greater or equal to >> "9". This can be easily done with the new jdk.Version API. >> >> Thank you and best regards, >> Volker >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8149519 From john.r.rose at oracle.com Thu Mar 10 23:42:21 2016 From: john.r.rose at oracle.com (John Rose) Date: Thu, 10 Mar 2016 15:42:21 -0800 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <531729471.2654969.1457533022406.JavaMail.zimbra@u-pem.fr> References: <2876a241-a423-44c3-b7bb-d64d9d3dfee8@default> <56DFFDFC.8010703@oracle.com> <531729471.2654969.1457533022406.JavaMail.zimbra@u-pem.fr> Message-ID: <84A8938C-91CD-4DCE-B13A-50E58305A1F4@oracle.com> Ok, now I mostly agree with R?mi and Maurizio. If a static search finds a super, we can assume that super will be present at runtime; so says Ch.13. The assumption of accessibility (of that chosen super) requires additional assurances which are (probably) in the spirit of 13 but not the letter. Here's my preference: Unless there is language in 13 guaranteeing accessibility of supers to clients (unlikely, but Maurizio can find it if it is there) then we should treat all supers as possibly-inaccessible (at runtime) except 1) those in the nest (compilation unit) of the class containing the indy call and 2) those in java.lang, which includes good old Object as a fallback. And 3) arrays thereof. Supers in the same package but not in the same nest, or in other package, if not java.lang, cannot be trusted to be accessible at runtime. Example: someone refactors a package-local super to be public and later refactors it non-public to reduce exports. Happens all the time. ? John On Mar 9, 2016, at 6:17 AM, Remi Forax wrote: >> provided that the total set of superclasses or superinterfaces, >> respectively, of the class type loses no members. From maurizio.cimadamore at oracle.com Mon Mar 14 10:29:02 2016 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 14 Mar 2016 10:29:02 +0000 Subject: RFR (S) 8151223: String concatenation fails with implicit toString() on package-private class In-Reply-To: <84A8938C-91CD-4DCE-B13A-50E58305A1F4@oracle.com> References: <2876a241-a423-44c3-b7bb-d64d9d3dfee8@default> <56DFFDFC.8010703@oracle.com> <531729471.2654969.1457533022406.JavaMail.zimbra@u-pem.fr> <84A8938C-91CD-4DCE-B13A-50E58305A1F4@oracle.com> Message-ID: <56E6926E.7010100@oracle.com> I can find two pieces of text that would seem to rule out the accessibility change that you describe: JLS, 14.4.3: public classes: "Changing a class that is not declared public to be declared public does not break compatibility with pre-existing binaries. If a class that was declared public is changed to not be declared public, then an IllegalAccessError is thrown if a pre-existing binary is linked that needs but no longer has access to the class type; such a change is not recommended for widely distributed classes." And, JLS, 15.12.4.3. Check Accessibility of Type and Method: "let T be the qualifying type of the method invocation (?13.1 ) and let m be the name of the method as determined at compile time (?15.12.3). [...] If either T or |m| is not accessible, then an |IllegalAccessError| occurs (?12.3 )." So, the first rule tells us that restricting accessibility of classes breaks BC - and the second rule explicitly mandates a VM implementation to throw IAE when an inaccessible qualifier is found on a method call. If you have code like this: class A { void m() { } } class B extends A { void test() { ((A)this).m(); } } javac will emit a descriptor for the invokevirtual of the kind A::m. Any access change to A which makes A inaccessible from B would then cause IAE at runtime, right? Maurizio On 10/03/16 23:42, John Rose wrote: > Ok, now I mostly agree with R?mi and Maurizio. If a static search finds a super, we can assume that super will be present at runtime; so says Ch.13. The assumption of accessibility (of that chosen super) requires additional assurances which are (probably) in the spirit of 13 but not the letter. > > Here's my preference: Unless there is language in 13 guaranteeing accessibility of supers to clients (unlikely, but Maurizio can find it if it is there) then we should treat all supers as possibly-inaccessible (at runtime) except 1) those in the nest (compilation unit) of the class containing the indy call and 2) those in java.lang, which includes good old Object as a fallback. And 3) arrays thereof. > > Supers in the same package but not in the same nest, or in other package, if not java.lang, cannot be trusted to be accessible at runtime. Example: someone refactors a package-local super to be public and later refactors it non-public to reduce exports. Happens all the time. > > ? John > > On Mar 9, 2016, at 6:17 AM, Remi Forax wrote: > >>> provided that the total set of superclasses or superinterfaces, >>> respectively, of the class type loses no members. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at oracle.com Thu Mar 17 00:19:50 2016 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 16 Mar 2016 17:19:50 -0700 Subject: RFR: change langtools tests to use ProblemList instead of @ignore Message-ID: <56E9F826.5080901@oracle.com> In the langtools test suite, the convention has been to use the jtreg @ignore mechanism to identify tests which should not be executed for some reason. But other repos in OpenJDK use the more recent and more flexible "exclude list" mechanism, as exemplified by files such as /test/ProblemList.txt. This is to set up the langtools/test/ProblemList.txt file with tests that currently contain @ignore in their test description. The @ignore tags are not being removed at this point, to allow for a changeover period until everyone is excluding tests using the ProblemList file, with an option such as the following: -exclude:langtools/test/ProblemList.txt It is regrettable that the number of ignored tests has been growing. As a separate exercise, we need to work on fixing the tests on this list. JBS: https://bugs.openjdk.java.net/browse/JDK-8152048 Webrev: http://cr.openjdk.java.net/~jjg/8152048/webrev.00/ -- Jon From joe.darcy at oracle.com Thu Mar 17 00:43:51 2016 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Wed, 16 Mar 2016 17:43:51 -0700 Subject: RFR: change langtools tests to use ProblemList instead of @ignore In-Reply-To: <56E9F826.5080901@oracle.com> References: <56E9F826.5080901@oracle.com> Message-ID: <56E9FDC7.9060705@oracle.com> Looks fine Jon; thanks, -Joe On 3/16/2016 5:19 PM, Jonathan Gibbons wrote: > In the langtools test suite, the convention has been to use the jtreg > @ignore mechanism to identify tests which should not be executed for > some reason. But other repos in OpenJDK use the more recent and > more flexible "exclude list" mechanism, as exemplified by files such > as /test/ProblemList.txt. > > This is to set up the langtools/test/ProblemList.txt file with tests > that currently contain @ignore in their test description. The @ignore > tags are not being removed at this point, to allow for a changeover > period until everyone is excluding tests using the ProblemList file, > with an option such as the following: > -exclude:langtools/test/ProblemList.txt > > It is regrettable that the number of ignored tests has been growing. > As a separate exercise, we need to work on fixing the tests on this list. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8152048 > Webrev: http://cr.openjdk.java.net/~jjg/8152048/webrev.00/ > > -- Jon From bsrbnd at gmail.com Thu Mar 17 11:26:34 2016 From: bsrbnd at gmail.com (bsrbnd) Date: Thu, 17 Mar 2016 12:26:34 +0100 Subject: Symbolic annotation processor In-Reply-To: References: Message-ID: Hi, After a very fast and superficial reading of the JLS, I wrote some jtreg tests for points that might be impacted by the symbolic-processor. Source code of the tests, grouped in two files for more readability, can be found at: https://github.com/bsrbnd/draft/tree/master/src/tests Some of them show a bit more precisely the benefit of incorporating in some way the symbolic programming paradigm (the ability to manipulate the symbols of a language) to enhance the reflection (and perhaps even more...). As generics are earased during runtime, it could be hard to get the reflective elements of members using generics heavily (and even harder to maintain). Consider the following example inspired by JLS8, chapter 4.4: class MyTypeVariable { public static class C {} public static interface I {} @Symbolic private void myMethod(T t, T[] u) {} } Without the symbolic view of members, you have to write something like the following to get the reflective object: Method m = MyTypeVariable.class.getDeclaredMethod("myMethod", MyTypeVariable.C.class, MyTypeVariable.C[].class)); But, with the symbolic-processor, you delegate the generics and erasures handling to the compiler: Method m = MyTypeVariable.$myMethod.reflect(); This is, of course, easier to write and to maintain. I've also tried to go further with this paradigm by introducing dynamically constructed symbolic expressions (a bit like LISP does) and evaluate them on a particular instance, for example: class DynamicExpression { @Symbolic private Integer a = 3; @Symbolic private Integer b = 2; @Symbolic private Integer c = 1; @Symbolic public Integer add(Integer i, Integer j) {return i + j;} @Symbolic public Integer sub(Integer i, Integer j) {return i - j;} } MethodSymbol $symbolicExpression = $add.apply($a, $sub.apply($b, $c)); Integer d = (Integer)$symbolicExpression.evaluate(new DynamicExpression()); This expression evaluates naturally to 4 for this instance. The full running example can be found at: https://github.com/bsrbnd/draft/blob/master/src/examples/DynamicExpression.java Symbolic expressions could be useful for evolutionary algorithms used in genetic programming, where dynamic expressions are iteratively crossed-over and mutated to find the best approximation of a solution. I hope that I didn't miss anything in the JLS and that something like this hasn't already been incorporated in any existing project. Regards, Bernard 2015-12-10 14:50 GMT+01:00 bsrbnd : > Hi, > > Following our discussion (October 2015), I wrote an > annotation-processor (only a draft for now) that gives a symbolic view > of class members, for example: > @Symbolic private Object myField; > @Symbolic("_") private void myMethod(Object param) {} > > We can either access the name or the reflective object of members the > following way: > String name = $myField.toString(); // $ is the default prefix > Method m = _myMethod.reflect(); // _ is a custom prefix, useful for > overloaded identifiers > > You can find a simple example here: > https://github.com/bsrbnd/draft/blob/master/src/examples/Example.java > > Full source code of the annotation-processor prototype is given there: > https://github.com/bsrbnd/draft > > Comments are welcome, but I know this is probably not the right > mailing-list for that. > > Regards, > > Bernard From mandy.chung at oracle.com Wed Mar 23 01:46:53 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 22 Mar 2016 18:46:53 -0700 Subject: Review request: 8152503: tools/javac/completionDeps/DepsAndAnno.java fails after jigsaw m3 Message-ID: Also problem list a new test tools/jdeps/moidules/GenModuleInfo.java diff --git a/test/tools/javac/completionDeps/DepsAndAnno.java b/test/tools/javac/completionDeps/DepsAndAnno.java --- a/test/tools/javac/completionDeps/DepsAndAnno.java +++ b/test/tools/javac/completionDeps/DepsAndAnno.java @@ -26,6 +26,10 @@ * @bug 8078600 * @summary Make sure -XDcompletionDeps does not cause an infinite loop. * @library /tools/lib + * @modules jdk.compiler/com.sun.tools.javac.api + * jdk.compiler/com.sun.tools.javac.file + * jdk.compiler/com.sun.tools.javac.main + * jdk.jdeps/com.sun.tools.javap * @build ToolBox * @run main/othervm/timeout=10 DepsAndAnno */ diff --git a/test/ProblemList.txt b/test/ProblemList.txt --- a/test/ProblemList.txt +++ b/test/ProblemList.txt @@ -91,3 +91,9 @@ tools/sjavac/IncCompileFullyQualifiedRef.java 8152055 generic-all Requires dependency code to deal with in-method dependencies. tools/sjavac/IncCompileWithChanges.java 8152055 generic-all Requires dependency code to deal with in-method dependencies. +########################################################################### +# +# jdeps + +tools/jdeps/moidules/GenModuleInfo.java 8152502 windows-all fails to clean up files + From joe.darcy at oracle.com Wed Mar 23 02:33:33 2016 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 22 Mar 2016 19:33:33 -0700 Subject: Review request: 8152503: tools/javac/completionDeps/DepsAndAnno.java fails after jigsaw m3 In-Reply-To: References: Message-ID: <56F2007D.6060302@oracle.com> Looks fine Mandy; thanks, -Joe On 3/22/2016 6:46 PM, Mandy Chung wrote: > Also problem list a new test tools/jdeps/moidules/GenModuleInfo.java > > diff --git a/test/tools/javac/completionDeps/DepsAndAnno.java b/test/tools/javac/completionDeps/DepsAndAnno.java > --- a/test/tools/javac/completionDeps/DepsAndAnno.java > +++ b/test/tools/javac/completionDeps/DepsAndAnno.java > @@ -26,6 +26,10 @@ > * @bug 8078600 > * @summary Make sure -XDcompletionDeps does not cause an infinite loop. > * @library /tools/lib > + * @modules jdk.compiler/com.sun.tools.javac.api > + * jdk.compiler/com.sun.tools.javac.file > + * jdk.compiler/com.sun.tools.javac.main > + * jdk.jdeps/com.sun.tools.javap > * @build ToolBox > * @run main/othervm/timeout=10 DepsAndAnno > */ > > > > diff --git a/test/ProblemList.txt b/test/ProblemList.txt > --- a/test/ProblemList.txt > +++ b/test/ProblemList.txt > @@ -91,3 +91,9 @@ > tools/sjavac/IncCompileFullyQualifiedRef.java 8152055 generic-all Requires dependency code to deal with in-method dependencies. > tools/sjavac/IncCompileWithChanges.java 8152055 generic-all Requires dependency code to deal with in-method dependencies. > > +########################################################################### > +# > +# jdeps > + > +tools/jdeps/moidules/GenModuleInfo.java 8152502 windows-all fails to clean up files > + From jonathan.gibbons at oracle.com Mon Mar 28 20:34:43 2016 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 28 Mar 2016 13:34:43 -0700 Subject: RFR: 8152897: refactor ToolBox to allow reduced documented dependencies Message-ID: <56F99563.4090603@oracle.com> JBS: https://bugs.openjdk.java.net/browse/JDK-8152897 Review: http://cr.openjdk.java.net/~jjg/8152897/webrev.00/index.html This is a test-only change, which updates the ToolBox test library from being a single top-level class in the unnamed package to a small set of classes in a new named package. Although this does increase the amount of overhead in the test description header, it also allows us to be more precise in the test dependencies, such that we can reduce otherwise unnecessary compilation dependencies. This will also make it more practical to extend the set of supported ToolBox tasks, such as for jmod, jlink etc, so that only tests that would want to use such tasks would incur any need to build the tasks. The update is primarily for javac tests, but also includes changes for tests for jshell and sjavac which also use the ToolBox test library. There is no significant change to the functionality of any tests. The build dependencies have been verified by running each test separately within jtreg, using a clean JTwork/classes directory for each test. -- Jon From kumar.x.srinivasan at oracle.com Wed Mar 30 18:16:24 2016 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Wed, 30 Mar 2016 11:16:24 -0700 Subject: RFR: 8152897: refactor ToolBox to allow reduced documented dependencies In-Reply-To: <56F99563.4090603@oracle.com> References: <56F99563.4090603@oracle.com> Message-ID: <56FC17F8.2070707@oracle.com> Jon, What about javadoc ? With the new module options more so for the tool than the doclet. Can we have a javadocTask ? Thanks Kumar > JBS: https://bugs.openjdk.java.net/browse/JDK-8152897 > Review: http://cr.openjdk.java.net/~jjg/8152897/webrev.00/index.html > > This is a test-only change, which updates the ToolBox test library > from being a single top-level class in the unnamed package to a small > set of classes in a new named package. Although this does increase the > amount of overhead in the test description header, it also allows us > to be more precise in the test dependencies, such that we can reduce > otherwise unnecessary compilation dependencies. This will also make it > more practical to extend the set of supported ToolBox tasks, such as > for jmod, jlink etc, so that only tests that would want to use such > tasks would incur any need to build the tasks. > > The update is primarily for javac tests, but also includes changes > for tests for jshell and sjavac which also use the ToolBox test library. > > There is no significant change to the functionality of any tests. The > build dependencies have been verified by running each test separately > within jtreg, using a clean JTwork/classes directory for each test. > > -- Jon From vicente.romero at oracle.com Wed Mar 30 21:08:30 2016 From: vicente.romero at oracle.com (Vicente-Arturo Romero-Zaldivar) Date: Wed, 30 Mar 2016 17:08:30 -0400 Subject: RFR: 8152897: refactor ToolBox to allow reduced documented dependencies In-Reply-To: <56F99563.4090603@oracle.com> References: <56F99563.4090603@oracle.com> Message-ID: <56FC404E.5050407@oracle.com> Hi Jon, Great job, thanks a lot for doing this work. Some minor comments: - class Result at Task.java, I know that Result is the original name for the class but thinking again about it, I think that probably we need a more specific name like TaskResult for example. We can probably say the same about enum Mode in the same class. I'm not strong about this name changes and in any case they can be done in a separate patch. - there is dead code at JavacTask.java, method: private int runAPI(PrintWriter pw) throws IOException - there is a new line missing at the end of the new ToolBox.java file As the comments are really minor I accept the patch, there is no need for another iteration, Thanks, Vicente On 03/28/2016 04:34 PM, Jonathan Gibbons wrote: > JBS: https://bugs.openjdk.java.net/browse/JDK-8152897 > Review: http://cr.openjdk.java.net/~jjg/8152897/webrev.00/index.html > > This is a test-only change, which updates the ToolBox test library > from being a single top-level class in the unnamed package to a small > set of classes in a new named package. Although this does increase the > amount of overhead in the test description header, it also allows us > to be more precise in the test dependencies, such that we can reduce > otherwise unnecessary compilation dependencies. This will also make it > more practical to extend the set of supported ToolBox tasks, such as > for jmod, jlink etc, so that only tests that would want to use such > tasks would incur any need to build the tasks. > > The update is primarily for javac tests, but also includes changes > for tests for jshell and sjavac which also use the ToolBox test library. > > There is no significant change to the functionality of any tests. The > build dependencies have been verified by running each test separately > within jtreg, using a clean JTwork/classes directory for each test. > > -- Jon