RFR: JDK-8303912 - Clean up JavadocTokenizer [v4]

[Jim Laskey]

> Previous work had cleaned up the JavacTokenizer, but left JavadocTokenizer alone, mostly due to its perplexing state. This is follow up work to simplify the JavadocTokenizer.

Jim Laskey has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:

 - this-escape suppression
 - Merge branch 'master' into 8303912
 - Recommended changes
 - Corrections from review
 - Merge branch 'master' into 8303912
 - Clean up JavadocTokenizer

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/12953/files
  - new: https://git.openjdk.org/jdk/pull/12953/files/6e3be1eb..82f97e13

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=12953&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12953&range=02-03

  Stats: 23179 lines in 744 files changed: 11220 ins; 8251 del; 3708 mod
  Patch: https://git.openjdk.org/jdk/pull/12953.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/12953/head:pull/12953

PR: https://git.openjdk.org/jdk/pull/12953