RFR: JDK-8305100 [REDO] Clean up JavadocTokenizer

[Jonathan Gibbons]

On Fri, 31 Mar 2023 11:10:42 GMT, Jim Laskey <jlaskey at openjdk.org> wrote:

> Previous work had cleaned up the JavacTokenizer, but left JavadocTokenizer alone, mostly due to its perplexing state. This is follow up work to simplify the JavadocTokenizer.
> 
> Redo changes:
> 
> 1. Needed to check for end of data after @deprecated
> 2. Position of newline in offset map was incorrect (out by one).
> 3. Skip whitespace after first line (put offsets out by one).

Looks good to me

-------------

Marked as reviewed by jjg (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/13265#pullrequestreview-1367587275