From stuart.marks at oracle.com Wed Mar 5 22:11:05 2025 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 5 Mar 2025 14:11:05 -0800 Subject: anomaly with apidiff output Message-ID: <5cfaa166-b6c7-4847-a48b-368436cdc2b3@oracle.com> Hi Apidiff Devs, I'd like to report an anomaly with apidiff. I made a recent change to the JDK specification, which can be seen here: https://github.com/openjdk/jdk/compare/master...stuart-marks:jdk:JDK-8138614-relax-new-string-requirement This is a change to the doc comments of four AbstractStringBuilder methods: subSequence, substring(i), substring(i, i), and toString(). Note that ASB is a non-public class which has two public subclasses, StringBuffer and StringBuilder. Thus there should be a total of eight changes: four methods in each of the two public classes. I did two javadoc runs: one on the tip of this branch ("new") and one for its right parent which is the commit from master that was merged into it ("old"). The diffs are minimal. The javadoc output is as expected: I can see changes in the specs of four methods in each of the StringBuilder and StringBuffer classes. I build apidiff from source a couple weeks ago. The command line I ran was something like: apidiff --api old --api-directory javadoc-old \ --api new --api-directory javadoc-new \ --include 'java.base/java.lang.*' --output-directory sb.apidiff (Where javadoc-old and javadoc-new are the respective javadoc output directories.) It seemed mostly successful in that it inspected all the java.lang classes, and it found differences in the right files. The output is visible here: https://cr.openjdk.org/~smarks/misc/sb.apidiff/ However, the actual differences it found were incorrect. The diffs for StringBuffer seem correct. However, for StringBuilder, only one difference is shown, for the toString() method. The diffs for the other methods are missing. Also, perhaps significant, is that a couple other methods are missing from StringBuilder, setCharAt() and setLength(), though there aren't any differences in the specs of these methods. Daniel Fuchs suggested that https://bugs.openjdk.org/browse/CODETOOLS-7903843 might be related. Indeed it might be, but I can't tell. Anyone have any ideas? I can file a bug with this info if it's helpful. s'marks From jjg3 at pobox.com Thu Mar 6 16:31:58 2025 From: jjg3 at pobox.com (Jonathan Gibbons) Date: Thu, 6 Mar 2025 08:31:58 -0800 Subject: anomaly with apidiff output In-Reply-To: <5cfaa166-b6c7-4847-a48b-368436cdc2b3@oracle.com> References: <5cfaa166-b6c7-4847-a48b-368436cdc2b3@oracle.com> Message-ID: Stuart, Without deep-diving too much too quickly, your command line seems to be incomplete. It is not enough to *just* provide the api directory; you need to provide the source and/or class files as well. When comparing JDK docs, this is best done by using the `--jdk-build` option, which internally expands to the appropriate javac-level options. If you do not provide the necessary options, the tool will fall back to using the platform classes. Maybe there is a way to enhance the command-line analysis to warn when the set of given options look at lest superficially insufficient. -- Jon On 3/5/25 2:11 PM, Stuart Marks wrote: > Hi Apidiff Devs, > > I'd like to report an anomaly with apidiff. I made a recent change to > the JDK specification, which can be seen here: > > > https://github.com/openjdk/jdk/compare/master...stuart-marks:jdk:JDK-8138614-relax-new-string-requirement > > > This is a change to the doc comments of four AbstractStringBuilder > methods: subSequence, substring(i), substring(i, i), and toString(). > Note that ASB is a non-public class which has two public subclasses, > StringBuffer and StringBuilder. Thus there should be a total of eight > changes: four methods in each of the two public classes. > > I did two javadoc runs: one on the tip of this branch ("new") and one > for its right parent which is the commit from master that was merged > into it ("old"). The diffs are minimal. The javadoc output is as > expected: I can see changes in the specs of four methods in each of > the StringBuilder and StringBuffer classes. > > I build apidiff from source a couple weeks ago. The command line I ran > was something like: > > ??? apidiff --api old --api-directory javadoc-old \ > ??????? --api new --api-directory javadoc-new \ > ??????? --include 'java.base/java.lang.*' --output-directory sb.apidiff > > (Where javadoc-old and javadoc-new are the respective javadoc output > directories.) It seemed mostly successful in that it inspected all the > java.lang classes, and it found differences in the right files. > > The output is visible here: > > ??? https://cr.openjdk.org/~smarks/misc/sb.apidiff/ > > However, the actual differences it found were incorrect. The diffs for > StringBuffer seem correct. However, for StringBuilder, only one > difference is shown, for the toString() method. The diffs for the > other methods are missing. Also, perhaps significant, is that a couple > other methods are missing from StringBuilder, setCharAt() and > setLength(), though there aren't any differences in the specs of these > methods. > > Daniel Fuchs suggested that > https://bugs.openjdk.org/browse/CODETOOLS-7903843 might be related. > Indeed it might be, but I can't tell. > > Anyone have any ideas? I can file a bug with this info if it's helpful. > > s'marks > From stuart.marks at oracle.com Sat Mar 8 19:04:37 2025 From: stuart.marks at oracle.com (Stuart Marks) Date: Sat, 8 Mar 2025 11:04:37 -0800 Subject: [External] : Re: anomaly with apidiff output In-Reply-To: References: <5cfaa166-b6c7-4847-a48b-368436cdc2b3@oracle.com> Message-ID: Hi Jon, Good to hear from you! I guess I've been operating under the assumption that apidiff works (or can work) by gathering all the information from two versions of the HTML javadoc output and generating the differences between them. (That's the way specdiff works and how I've been using it for quite some time.) If apidiff doesn't work that way, then ok, I'll figure out a different approach. I guess what's surprising to me is that it *almost* works. It successfully found the two classes that were impacted and found and displayed the correct diffs for one of the classes but not the second. And the second class is *mostly* there but some bits are missing. So it seems to me like there's just some bug there that's preventing it from working completely, and not a fundamental problem with the model of comparing HTML javadoc output. I haven't looked at the code, though, so it's entirely possible that I'm way off base on this. I'll try out some alternative approaches for generating the diffs that I'm looking for. Thanks for your suggestions. s'marks On 3/6/25 8:31 AM, Jonathan Gibbons wrote: > Stuart, > > Without deep-diving too much too quickly, your command line seems to be incomplete. > > It is not enough to *just* provide the api directory; you need to provide the source > and/or class files as well. When comparing JDK docs, this is best done by using the > `--jdk-build` option, which internally expands to the appropriate javac-level options. > > If you do not provide the necessary options, the tool will fall back to using the > platform classes. Maybe there is a way to enhance the command-line analysis to warn > when the set of given options look at lest superficially insufficient. > > -- Jon > > On 3/5/25 2:11 PM, Stuart Marks wrote: >> Hi Apidiff Devs, >> >> I'd like to report an anomaly with apidiff. I made a recent change to the JDK >> specification, which can be seen here: >> >> >> https://urldefense.com/v3/__https://github.com/openjdk/jdk/compare/ >> master...stuart-marks:jdk:JDK-8138614-relax-new-string-requirement__;!! >> ACWV5N9M2RV99hQ!Nttbo- >> eq1OfDZAq3juoWytRGuIZooffq5gUBLSrnn8NoGzFg2OV7PVk7szakD5cgg1xlxKC7RcLUjg$ >> >> This is a change to the doc comments of four AbstractStringBuilder methods: >> subSequence, substring(i), substring(i, i), and toString(). Note that ASB is a >> non-public class which has two public subclasses, StringBuffer and StringBuilder. >> Thus there should be a total of eight changes: four methods in each of the two >> public classes. >> >> I did two javadoc runs: one on the tip of this branch ("new") and one for its >> right parent which is the commit from master that was merged into it ("old"). The >> diffs are minimal. The javadoc output is as expected: I can see changes in the >> specs of four methods in each of the StringBuilder and StringBuffer classes. >> >> I build apidiff from source a couple weeks ago. The command line I ran was >> something like: >> >> ??? apidiff --api old --api-directory javadoc-old \ >> ??????? --api new --api-directory javadoc-new \ >> ??????? --include 'java.base/java.lang.*' --output-directory sb.apidiff >> >> (Where javadoc-old and javadoc-new are the respective javadoc output directories.) >> It seemed mostly successful in that it inspected all the java.lang classes, and it >> found differences in the right files. >> >> The output is visible here: >> >> ??? https://cr.openjdk.org/~smarks/misc/sb.apidiff/ >> >> However, the actual differences it found were incorrect. The diffs for >> StringBuffer seem correct. However, for StringBuilder, only one difference is >> shown, for the toString() method. The diffs for the other methods are missing. >> Also, perhaps significant, is that a couple other methods are missing from >> StringBuilder, setCharAt() and setLength(), though there aren't any differences in >> the specs of these methods. >> >> Daniel Fuchs suggested that https://bugs.openjdk.org/browse/CODETOOLS-7903843 >> might be related. Indeed it might be, but I can't tell. >> >> Anyone have any ideas? I can file a bug with this info if it's helpful. >> >> s'marks >> From jjg3 at pobox.com Mon Mar 10 00:28:43 2025 From: jjg3 at pobox.com (Jonathan Gibbons) Date: Sun, 9 Mar 2025 17:28:43 -0700 Subject: [External] : Re: anomaly with apidiff output In-Reply-To: References: <5cfaa166-b6c7-4847-a48b-368436cdc2b3@oracle.com> Message-ID: Stuart, With respect, you're way off base here. apidiff uses the declarations in the source and class files to perform a structural comparison using the Language Model API (JSR269) available to `javac` plugins and extension classes. Having determined the declarations to be compared, and having compared the underlying elements, it will then optionally in addition try and locate the generated API documentation to run a comparison of the HTML description generated by javadoc. There is not sufficient information in the HTML files to fully reverse engineer the Language Model elements that can otherwise be obtained using the `javac` front end. Also note that while there has been much work to standardize how to locate the basic HTML description for an element (in terms of HTML elements, and class and id attributes) there is no such internal/informal standard for the presentation of the signature of the element, which is the closest approximation in the HTML to the Language Model element. Yes, there are other API comparison tools that just try and compare the generated HTML documentation. Assuming you are typically comparing the API in different JDK builds, I highly recommend? using the `--jdk-build` API option. If that doesn't address your needs, let me know, and I'll see if I can come up with additional suggestions for you. You might find it instructive to review the presentation I gave last summer; I'm assuming you can find the saved PDF. If not, contact me privately. -- Jon On 3/8/25 11:04 AM, Stuart Marks wrote: > Hi Jon, > > Good to hear from you! > > I guess I've been operating under the assumption that apidiff works > (or can work) by gathering all the information from two versions of > the HTML javadoc output and generating the differences between them. > (That's the way specdiff works and how I've been using it for quite > some time.) > > If apidiff doesn't work that way, then ok, I'll figure out a different > approach. > > I guess what's surprising to me is that it *almost* works. It > successfully found the two classes that were impacted and found and > displayed the correct diffs for one of the classes but not the second. > And the second class is *mostly* there but some bits are missing. So > it seems to me like there's just some bug there that's preventing it > from working completely, and not a fundamental problem with the model > of comparing HTML javadoc output. > > I haven't looked at the code, though, so it's entirely possible that > I'm way off base on this. > > I'll try out some alternative approaches for generating the diffs that > I'm looking for. Thanks for your suggestions. > > s'marks > > > > On 3/6/25 8:31 AM, Jonathan Gibbons wrote: >> Stuart, >> >> Without deep-diving too much too quickly, your command line seems to >> be incomplete. >> >> It is not enough to *just* provide the api directory; you need to >> provide the source and/or class files as well. When comparing JDK >> docs, this is best done by using the `--jdk-build` option, which >> internally expands to the appropriate javac-level options. >> >> If you do not provide the necessary options, the tool will fall back >> to using the platform classes. Maybe there is a way to enhance the >> command-line analysis to warn when the set of given options look at >> lest superficially insufficient. >> >> -- Jon >> >> On 3/5/25 2:11 PM, Stuart Marks wrote: >>> Hi Apidiff Devs, >>> >>> I'd like to report an anomaly with apidiff. I made a recent change >>> to the JDK specification, which can be seen here: >>> >>> >>> https://urldefense.com/v3/__https://github.com/openjdk/jdk/compare/ >>> master...stuart-marks:jdk:JDK-8138614-relax-new-string-requirement__;!! >>> ACWV5N9M2RV99hQ!Nttbo- >>> eq1OfDZAq3juoWytRGuIZooffq5gUBLSrnn8NoGzFg2OV7PVk7szakD5cgg1xlxKC7RcLUjg$ >>> >>> This is a change to the doc comments of four AbstractStringBuilder >>> methods: subSequence, substring(i), substring(i, i), and toString(). >>> Note that ASB is a non-public class which has two public subclasses, >>> StringBuffer and StringBuilder. Thus there should be a total of >>> eight changes: four methods in each of the two public classes. >>> >>> I did two javadoc runs: one on the tip of this branch ("new") and >>> one for its right parent which is the commit from master that was >>> merged into it ("old"). The diffs are minimal. The javadoc output is >>> as expected: I can see changes in the specs of four methods in each >>> of the StringBuilder and StringBuffer classes. >>> >>> I build apidiff from source a couple weeks ago. The command line I >>> ran was something like: >>> >>> ??? apidiff --api old --api-directory javadoc-old \ >>> ??????? --api new --api-directory javadoc-new \ >>> ??????? --include 'java.base/java.lang.*' --output-directory sb.apidiff >>> >>> (Where javadoc-old and javadoc-new are the respective javadoc output >>> directories.) It seemed mostly successful in that it inspected all >>> the java.lang classes, and it found differences in the right files. >>> >>> The output is visible here: >>> >>> ??? https://cr.openjdk.org/~smarks/misc/sb.apidiff/ >>> >>> However, the actual differences it found were incorrect. The diffs >>> for StringBuffer seem correct. However, for StringBuilder, only one >>> difference is shown, for the toString() method. The diffs for the >>> other methods are missing. Also, perhaps significant, is that a >>> couple other methods are missing from StringBuilder, setCharAt() and >>> setLength(), though there aren't any differences in the specs of >>> these methods. >>> >>> Daniel Fuchs suggested that >>> https://bugs.openjdk.org/browse/CODETOOLS-7903843 might be related. >>> Indeed it might be, but I can't tell. >>> >>> Anyone have any ideas? I can file a bug with this info if it's helpful. >>> >>> s'marks >>> > From stuart.marks at oracle.com Tue Mar 11 21:20:03 2025 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 11 Mar 2025 14:20:03 -0700 Subject: [External] : Re: anomaly with apidiff output In-Reply-To: References: <5cfaa166-b6c7-4847-a48b-368436cdc2b3@oracle.com> Message-ID: On 3/9/25 5:28 PM, Jonathan Gibbons wrote: > With respect, you're way off base here. > > apidiff uses the declarations in the source and class files to perform a structural > comparison using the Language Model API (JSR269) available to `javac` plugins and > extension classes. Having determined the declarations to be compared, and having > compared the underlying elements, it will then optionally in addition try and locate > the generated API documentation to run a comparison of the HTML description > generated by javadoc. > > There is not sufficient information in the HTML files to fully reverse engineer the > Language Model elements that can otherwise be obtained using the `javac` front end. OK, fair, I remember you saying this, and the docs imply that the sources and class files are required. That said, since apidiff seemed to accept the javadoc HTML output, I figured I'd give it a try, and it looked like it allllmost worked.... But I understand that making it work in this mode (using only HTML as input) is probably untenable in the general case, and that's fine. Previously, you had said: > If you do not provide the necessary options, the tool will fall back to using the platform classes. Maybe there is a way to enhance the command-line analysis to warn when the set of given options look at lest superficially insufficient. Yes, it seems like that would be good. *** Anyway, that aside, I took your advice and instead ran apidiff, providing it with sources and class files. I did two full JDK builds that differed only in the spec changes to AbstractStringBuilder, which would produce changes only in the javadoc output of StringBuilder and StringBuffer. The command line was something like this: $ apidiff --api old --jdk-build jdk-tmp-a/build/macosx-aarch64 \ --module-source-path jdk-tmp-a/open/src/java.base/share/classes/ \ --api new --jdk-build jdk-tmp-b/build/macosx-aarch64 \ --module-source-path jdk-tmp-b/open/src/java.base/share/classes/ \ --include 'java.base/java.lang.*' --output-directory apidiff.out And the output was pretty much the same as previously. That is, I got a nice report showing differences in StringBuffer and StringBuilder. StringBuffer had the correct differences in four methods, and StringBuilder was missing some method declarations (both changed and unchanged) and showed diffs only in the toString() method. For reference, here's the previous output: https://cr.openjdk.org/~smarks/misc/sb.apidiff/ I could upload the new output but it's essentially the same to my eye. Before going further, is this the right way to invoke apidiff? s'marks From jjg3 at pobox.com Wed Mar 12 16:13:24 2025 From: jjg3 at pobox.com (Jonathan Gibbons) Date: Wed, 12 Mar 2025 09:13:24 -0700 Subject: [External] : Re: anomaly with apidiff output In-Reply-To: References: <5cfaa166-b6c7-4847-a48b-368436cdc2b3@oracle.com> Message-ID: <7abc58ba-012d-4bf4-a6f9-8b914ace1bf9@pobox.com> If you use `--jdk-build` you typically don't need to set any other javac-like options, including `--module-source-path`. `--jdk-build` will automatically use the source files in the `src.zip` in the given build. The only reason that may not be enough would be if you are wanting to compare any classes whose sourcs is not in src.zip -- yes, some files might be omitted in certain situations, but that's a different, more complicated story. -- Jon On 3/11/25 2:20 PM, Stuart Marks wrote: > > > On 3/9/25 5:28 PM, Jonathan Gibbons wrote: >> With respect, you're way off base here. >> >> apidiff uses the declarations in the source and class files to >> perform a structural comparison using the Language Model API (JSR269) >> available to `javac` plugins and extension classes. Having determined >> the declarations to be compared, and having compared the underlying >> elements, it will then optionally in addition try and locate the >> generated API documentation to run a comparison of the HTML >> description generated by javadoc. >> >> There is not sufficient information in the HTML files to fully >> reverse engineer the Language Model elements that can otherwise be >> obtained using the `javac` front end. > > OK, fair, I remember you saying this, and the docs imply that the > sources and class files are required. That said, since apidiff seemed > to accept the javadoc HTML output, I figured I'd give it a try, and it > looked like it allllmost worked.... > > But I understand that making it work in this mode (using only HTML as > input) is probably untenable in the general case, and that's fine. > Previously, you had said: > >> If you do not provide the necessary options, the tool will fall back >> to using the platform classes. Maybe there is a way to enhance the >> command-line analysis to warn when the set of given options look at >> lest superficially insufficient. > > Yes, it seems like that would be good. > > *** > > Anyway, that aside, I took your advice and instead ran apidiff, > providing it with sources and class files. I did two full JDK builds > that differed only in the spec changes to AbstractStringBuilder, which > would produce changes only in the javadoc output of StringBuilder and > StringBuffer. The command line was something like this: > > $ apidiff --api old --jdk-build jdk-tmp-a/build/macosx-aarch64 \ > ??? --module-source-path jdk-tmp-a/open/src/java.base/share/classes/ \ > ??? --api new --jdk-build jdk-tmp-b/build/macosx-aarch64 \ > ??? --module-source-path jdk-tmp-b/open/src/java.base/share/classes/ \ > ??? --include 'java.base/java.lang.*' --output-directory apidiff.out > > And the output was pretty much the same as previously. That is, I got > a nice report showing differences in StringBuffer and StringBuilder. > StringBuffer had the correct differences in four methods, and > StringBuilder was missing some method declarations (both changed and > unchanged) and showed diffs only in the toString() method. > > For reference, here's the previous output: > > ??? https://cr.openjdk.org/~smarks/misc/sb.apidiff/ > > I could upload the new output but it's essentially the same to my eye. > > Before going further, is this the right way to invoke apidiff? > > s'marks >