RFR: 8352249: Remove incidental whitespace in traditional doc comments [v2]

[Hannes Wallnöfer]

On Wed, 19 Mar 2025 08:22:25 GMT, Hannes Wallnöfer <hannesw at openjdk.org> wrote:

>> Please review a patch to remove incidental indentation from traditional doc comments, comparable to doing a `String.stripIndent()` on the comment contents but without special treatment of a last blank line. This adds a `stripIndent()` method to the `Tokens.Comment` interface and a new `JavadocTokenizer.StrippedComment` nested class that implements indentation stripping while maintaining a map of position offsets to the original comment. 
>> 
>> While the patch changes `javadoc` output by removing leading whitespace, the change is generally not visible in the browser except in `<pre>` elements, which is the point of this enhancement. 
>> 
>> The change affects most tree positions in the AST checker tests in javac/doctree, but mostly does not affect the structure of the parsed trees, with the exception of `BreakIterator` tests in `FirstSentenceTest.java`. 
>> 
>> `BreakIterator` does not recognize `.\n` immediately followed by a lower case letter as sentence break, while it recognizes the break if the letter is an upper-case letter *or* if there is a space between the line break and the letter. I find this rule a bit peculiar but AFAICT we can't influence the behavior of `BreakIterator`, so I have added tests that cover both lower and upper-case behavior.
>> 
>> The source position lookup in the stripped comment is implemented by creating a new `OffsetMap` that translates from the stripped to the original comment, then using the original comment's `OffsetMap` to translate the position to the source file. `OffsetMap` is relatively lightweight (usually 2 `int[]` elements per comment paragraph), so the added overhead is not too bad. 
>> 
>> Inspired by [JDK-8305688](https://bugs.openjdk.org/browse/JDK-8305688) I did various test builds with restricted jobs and memory settings, but didn't notice any change in processing or memory overhead to API docs builds.
>
> Hannes Wallnöfer has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Whitespace normalization in PrettyCheck becomes simpler

Thanks!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24032#issuecomment-2739289037