git: openjdk/jdk: 8253904: Revert Tokenizer improvements JDK-8224225

[Jim Laskey]

Changeset: 8fda5b82
Author:    Jim Laskey <jlaskey at openjdk.org>
Date:      2020-10-01 17:42:29 +0000
URL:       https://git.openjdk.java.net/jdk/commit/8fda5b82

8253904: Revert Tokenizer improvements JDK-8224225

Reviewed-by: mcimadamore

! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavaTokenizer.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavadocTokenizer.java
- src/jdk.compiler/share/classes/com/sun/tools/javac/parser/TextBlockSupport.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/UnicodeReader.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties
! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_ja.properties
! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_zh_CN.properties
! src/jdk.compiler/share/classes/com/sun/tools/javac/util/DiagnosticSource.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/util/LayoutCharacters.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/util/Position.java
! test/langtools/tools/javac/Digits.out
! test/langtools/tools/javac/diags/examples.not-yet.txt
! test/langtools/tools/javac/unicode/NonasciiDigit.out
! test/langtools/tools/javac/unicode/SubChar.java
! test/langtools/tools/javac/unicode/SupplementaryJavaID2.out
! test/langtools/tools/javac/unicode/SupplementaryJavaID3.out
! test/langtools/tools/javac/unicode/SupplementaryJavaID4.out
! test/langtools/tools/javac/unicode/SupplementaryJavaID5.out