RFR: JDK-8303912 - Clean up JavadocTokenizer [v2]

[Jim Laskey]

> Previous work had cleaned up the JavacTokenizer, but left JavadocTokenizer alone, mostly due to its perplexing state. This is follow up work to simplify the JavadocTokenizer.

Jim Laskey has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:

 - Corrections from review
 - Merge branch 'master' into 8303912
 - Clean up JavadocTokenizer

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/12953/files
  - new: https://git.openjdk.org/jdk/pull/12953/files/652fcd8e..e3e31d09

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=12953&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12953&range=00-01

  Stats: 80523 lines in 781 files changed: 45055 ins; 21180 del; 14288 mod
  Patch: https://git.openjdk.org/jdk/pull/12953.diff
  Fetch: git fetch https://git.openjdk.org/jdk pull/12953/head:pull/12953

PR: https://git.openjdk.org/jdk/pull/12953