RFR: 8254073: Tokenizer improvements (revised) [v2]

[Jim Laskey]

On Tue, 6 Oct 2020 20:48:34 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Jim Laskey has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   8254073: Tokenizer improvements (revised)
>
> test/langtools/tools/javac/lexer/JavaLexerTest2.java line 50:
> 
>> 48: public class JavaLexerTest2 {
>> 49:     static final TestTuple[] TESTS = {
>> 50:             new TestTuple("0bL",         LONGLITERAL, true),
> 
> Stylistic (optional) comment - we could have a common TestTuple superclass and two subclasses called Success and
> Failure, so that, by looking at the TESTS array it will be apparent what the behavior should be

Done

> test/langtools/tools/javac/lexer/JavaLexerTest2.java line 110:
> 
>> 108:         boolean willFail;
>> 109:
>> 110:         TestTuple(String input, TokenKind kind, String expected, boolean willFail) {
> 
> Is anything using this?

It does now.

> test/langtools/tools/javac/lexer/JavaLexerTest2.java line 48:
> 
>> 46: import static com.sun.tools.javac.parser.Tokens.TokenKind.*;
>> 47:
>> 48: public class JavaLexerTest2 {
> 
> Should we merge JavaLexerTest and this one? After all you have all the required infra in here?

Done.

-------------

PR: https://git.openjdk.java.net/jdk/pull/525