RFR 8192974: overhaul descriptor string parsing

[Henry Jen]

Looks good.

Cheers,
Henry



> On Dec 7, 2017, at 3:28 AM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
> 
> Here's a new webrev:
> 
> http://cr.openjdk.java.net/~mcimadamore/8192974-v2/
> 
> This adds a very simple lookahead(1) support in the lexer, so that the parser can easily disambiguate - this is used in two cases:
> 
> 1) when parsing a function, '*' could be either a varargs or the (unknown) size of an array parameter layout
> 
> 2) as you noted, when parsing a bitfied, a number could be either part of the bitfield, or the start of the subsequent array
> 
> (funny to note how all ambiguities are caused by the fact that an array can start with tokens that can also appear in other places - a potential simplifying move would be to just pick a different syntax for arrays, so that an unambiguous prefix is added).
> 
> Maurizio
> 
> 
> On 07/12/17 10:57, Maurizio Cimadamore wrote:
>> Heh - good catch - no way around it - I need lookahead(1) ;-)
>> 
>> Maurizio
>> 
>> 
>> On 06/12/17 19:55, Henry Jen wrote:
>>> I like the rewrite, and it looks good.
>>> 
>>> One thing I haven’t tested, but based on my reading, is the troubling case of bitfields where an array is after,
>>> 
>>> i:2b1b3b20i
>>> 
>>> This is an integer breaks into 3 bitfields, and an array of 20 integers. this can cause an error being thrown at DescriptorParser:227?
>>> 
>>> Cheers,
>>> Henry
>>> 
>>> 
>>> 
>>> 
>>>> On Dec 5, 2017, at 5:25 AM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
>>>> 
>>>> Whoops - forgot webrev:
>>>> 
>>>> http://cr.openjdk.java.net/~mcimadamore/8192974/
>>>> 
>>>> Maurizio
>>>> 
>>>> 
>>>> On 05/12/17 13:23, Maurizio Cimadamore wrote:
>>>>> Hi,
>>>>> this patch is a rewriting on the layout descriptor parser code - the previous code was working correctly, but was a bit too convoluted and lacked documentation. The new code is simpler and adds the grammar productions in the javadoc of the various parsing methods. This work should make it easier to add new features to the layout descriptor DSL (such as symbolic references).
>>>>> 
>>>>> The new parser has a single entry point - namely 'parseLayout', which gives back a Stream<Type>.
>>>>> 
>>>>> I've written a big test case that generates the set of sentences that can be expressed in the descriptor grammar - of course since this set is infinite, we need to do some tricks in order to prevent infinite expansions of productions - and we also need to do some tricks not to generate too many combinations.
>>>>> 
>>>>> Currently, the test checks approx 7 millions different combinations - and it does so in 10 seconds and with 1G of heap, which I think it's acceptable.
>>>>> 
>>>>> I also had to tweak the existing DescriptorTest as that was relying on the structure of the old parser.
>>>>> 
>>>>> Cheers
>>>>> Maurizio
>>>>> 
>> 
>