RFR JDK-8139414: java.util.Scanner hasNext() returns true, next() throws NoSuchElementException

[Stuart Marks]

Hi Sherman,

The fix looks good.

It would be helpful if the test for 8072582 generated the string instead of 
using a literal that's more than 1K long. The exact length is significant 
because Scanner's default buffer size is 1024, so the delimiter has to straddle 
the buffer boundary.

The 8139414 test generates its string, which is nicer. In this case the test is 
taken from the bug report, but in my opinion the addition of the "boundary" 
variable (which is the string ";") makes things more obscure. I'd suggest 
inlining it.

For both test cases it might be helpful to have a little utility that appends n 
copies of a char to a StringBuilder.

Thanks,

s'marks

On 6/8/16 1:57 PM, Xueming Shen wrote:
> Hi,
>
> Please help review the change for
>
> JDK-8139414: java.util.Scanner hasNext() returns true, next() throws
> NoSuchElementException
> JDK-8072582: Scanner delimits incorrectly when delimiter spans a buffer boundary
>
> issue: https://bugs.openjdk.java.net/browse/JDK-8139414
>        https://bugs.openjdk.java.net/browse/JDK-8072582
> webrev: http://cr.openjdk.java.net/~sherman/8072582_8139414/webrev
>
> In both cases the delimiter pattern is a kinda of "alternation" regex construct
> which can "match" the existing characters at the end of the internal buffer as
> delimiters, AND can extend to match more delimiters if more input is available.
>
> In issue JDK-8139414, the hasNext() uses hasTokenInBuffer() to find the delimiters
> "-;". It does not go beyond the boundary to check if there is more character, such
> as "-" that can also be part of the delimiters). So hasNext() returns true with the
> assumption that there is a token because there is/are more character after "-;".
> But method getCompleteTokenInBuffer() (used by next() implementation), which
> has the logic to check beyond the boundary even the delimiter pattern already
> has a match. It matches "-;-" as the delimiters and then find no "next" (null)
> after
> that.
>
> Similar for issue 8072582. This time the getCompleteTokenInBuffer does not
> use the "lookingAt() and beyond" logic for the second delimiters, which triggers
> problem when the delimiter pattern has different match result (beginning position)
> for cases within boundary and beyond boundary.
>
> The proposed fix here is to always check if there is more input when match
> delimiters at the internal buffer boundary.
>
> Thanks,
> Sherman