git: openjdk/jdk: 8224225: Tokenizer improvements

[Jim Laskey]

Changeset: 90c131f2
Author:    Jim Laskey <jlaskey at openjdk.org>
Date:      2020-10-01 15:39:01 +0000
URL:       https://git.openjdk.java.net/jdk/commit/90c131f2

8224225: Tokenizer improvements

Reviewed-by: mcimadamore

! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavaTokenizer.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavadocTokenizer.java
+ src/jdk.compiler/share/classes/com/sun/tools/javac/parser/TextBlockSupport.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/UnicodeReader.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties
! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_ja.properties
! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_zh_CN.properties
! src/jdk.compiler/share/classes/com/sun/tools/javac/util/DiagnosticSource.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/util/LayoutCharacters.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/util/Position.java
! test/langtools/tools/javac/Digits.out
! test/langtools/tools/javac/diags/examples.not-yet.txt
! test/langtools/tools/javac/unicode/NonasciiDigit.out
! test/langtools/tools/javac/unicode/SubChar.java
! test/langtools/tools/javac/unicode/SupplementaryJavaID2.out
! test/langtools/tools/javac/unicode/SupplementaryJavaID3.out
! test/langtools/tools/javac/unicode/SupplementaryJavaID4.out
! test/langtools/tools/javac/unicode/SupplementaryJavaID5.out