Combined grammar changes for binary literals and underscores in literals

[Joe Darcy]

Welcome back Bruce!

Bruce Chapman wrote:
> Joe, Derek et al,
>
> I am just writing up the combined integer literal omnibus proposal.
>
> A couple of questions about this grammar
>
>
> 1)
>
> considering
>
> DecimalNumeral
>    0
>    NonZeroDigit Digits_opt
>    NonZeroDigit Underscores Digits // New   (1)
>
> Underscores
>    _
>    Underscores _
>
> Digits:
>    Digit
>    Digit DigitsAndUnderscores_opt Digit (2)
>
>
> I think we can drop rule (1) above, if rule (2) is changed to
>
> DigitsAndUnderscores_opt Digit
>
> since there is already a leading NonZeroDigit, so we can start a 
> "Digits" with an underscore, so we don't need the special rule to allow 
> for an underscore immediately after the NonZeroDigit. 
>
> Agreed?
>
> If so we can also make corresponding change for Octal rules then drop 
> the Underscores production as these are the only places it is used.
>
>
> 2)
>
> I am wondering whether it would enhance grokkability of we renamed the 
> productions
>
> HexDigitsAndUnderscores (and other radix friends, and corresponding singular forms)
>
>
> to be (eg)
> HexMidDigits
>
> because it can't be used at either end of the HexDigits, but can be in 
> the middle.
>
> - if we did this the hex rules would change from
>
> HexDigits:
>    HexDigit
>    HexDigit HexDigitsAndUnderscores_opt HexDigit
>
> HexDigitsAndUnderscores:
>    HexDigitOrUnderscore
>    HexDigitsAndUnderscores HexDigitOrUnderscore
>
> HexDigitOrUnderscore
>    HexDigit
>    _
>
>
>
> to
>
> HexDigits:
>    HexDigit
>    HexDigit HexMidDigits_opt HexDigit
>
> HexMidDigits:
>    HexMidDigit
>    HexMidDigits HexMidDigit  (but actually vice versa as previously 
> discussed off-list with Alex Buckley)
>
> HexMidDigit
>    HexDigit
>    _
>
>
>
>
> Comments?
>
>   

One advantage of the way I wrote up the grammar is that the other 
appearances of Digits and HexDigits, such as in floating-point literals 
in section 3.10.2, automatically inherit the underscore policy without 
having to having leading digit guard productions in all places.  The 
current implementation does allow underscores in floating-point literals 
too, which is tested for in the regression tests.

-Joe


> Bruce
>
>
>
>
>
>
> Joe Darcy wrote:
>   
>> Hello.
>>
>> It has been pointed out to me off-list that the grammar as written below 
>> would disallow two-digit numbers, which is not desired.  This problem is 
>> corrected by changing the productions like
>>
>>     Digit DigitsAndUnderscores Digit
>>
>> to be
>>
>>     Digit DigitsAndUnderscores_opt Digit
>>
>> -Joe
>>
>> On 07/15/09 02:22 PM, Joseph D. Darcy wrote:
>>   
>>     
>>> Hello.
>>>
>>> Getting around to some grammatical hacking, below is a grammar I think 
>>> properly captures the desired changes for combining the binary 
>>> literals and underscores in literals proposals.  This grammar does 
>>> *not* include support for unsigned literals or autosizing literals.
>>>
>>> The basic approach is to redefine the FooDigits productions to be a 
>>> non-empty sequence of Foo digits and underscores starting and ending 
>>> with a Foo digit rather than just a non-empty sequence Foo digits.  
>>> Each FooDigits nonterminal has two rules, one for a single digit and 
>>> one for two or more digits.  The rules for two or more digits force a 
>>> true digit, rather than an underscore, as the first and last character.
>>>
>>> Changing the FooDigits definitions allows underscores in both integer 
>>> and floating-point literals.  In more detail:
>>>
>>> * Replace the productions Digits, HexDigits, OctalDigits, and  in 
>>> JLSv3 3.10.1 as shown below
>>> * Add productions for BinaryDigits
>>> * Add rules for FooDigitsAndUnderscores and FooDigitOrUnderscore
>>> * Add productions to DecimalNumeral and OctalNumeral:
>>> * Add a rule for Underscores
>>>
>>> IntegerLiteral:
>>>        DecimalIntegerLiteral
>>>        HexIntegerLiteral
>>>        OctalIntegerLiteral
>>>        BinaryIntegerLiteral         // New
>>>
>>> BinaryIntegerLiteral:
>>>        BinaryNumeral IntegerTypeSuffix_opt
>>>
>>> BinaryNumeral:
>>>        0 b BinaryDigits
>>>        0 B BinaryDigits
>>>
>>>
>>> DecimalNumeral
>>>   0
>>>   NonZeroDigit Digits_opt
>>>   NonZeroDigit Underscores Digits // New
>>>
>>> Underscores
>>>    _
>>>    Underscores _
>>>
>>> Digits:
>>>    Digit
>>>    Digit DigitsAndUnderscores Digit
>>>
>>> DigitsAndUnderscores:
>>>    DigitOrUnderscore
>>>    DigitsAndUnderscores DigitOrUnderscore
>>>
>>> DigitOrUnderscore
>>>    Digit
>>>    _
>>>
>>>
>>> HexDigits:
>>>    HexDigit
>>>    HexDigit HexDigitsAndUnderscores HexDigit
>>>
>>> HexDigitsAndUnderscores:
>>>    HexDigitOrUnderscore
>>>    HexDigitsAndUnderscores HexDigitOrUnderscore
>>>
>>> HexDigitOrUnderscore
>>>    HexDigit
>>>    _
>>>
>>> OctalNumeral:
>>>    0 OctalDigits
>>>    0 Underscores OctalDigits // New
>>>
>>> OctalDigits:
>>>    OctalDigit
>>>    OctalDigit OctalDigitsAndUnderscores OctalDigit
>>>
>>> OctalDigitsAndUnderscores:
>>>    OctalDigitOrUnderscore
>>>    OctalDigitsAndUnderscores OctalDigitOrUnderscore
>>>
>>> OctalDigitOrUnderscore
>>>    OctalDigit
>>>    _
>>>
>>> BinaryDigits:
>>>    BinaryDigit
>>>    BinaryDigit BinaryDigitsAndUnderscores BinaryDigit
>>>
>>> BinaryDigitsAndUnderscores:
>>>    BinaryDigitOrUnderscore
>>>    BinaryDigitsAndUnderscores BinaryDigitOrUnderscore
>>>
>>> BinaryDigitOrUnderscore
>>>    BinaryDigit
>>>    _
>>>
>>> BinaryDigit: one of
>>>        0 1
>>>
>>> -Joe
>>>     
>>>       
>>
>>   
>>     
>
>
>