From gavin.bierman at oracle.com  Wed Mar  1 00:42:31 2023
From: gavin.bierman at oracle.com (Gavin Bierman)
Date: Wed, 1 Mar 2023 00:42:31 +0000
Subject: Draft JEPs for Patterns in switch and Record Patterns
In-Reply-To: <em8dd9e7c9-1fb0-48b8-b735-562264471a7a@888eef6f.com>
References: <322686D2-BD86-4F05-BA33-C4017C772DB5@oracle.com>
 <em8dd9e7c9-1fb0-48b8-b735-562264471a7a@888eef6f.com>
Message-ID: <73047CA6-1D68-43C9-8D86-070346732586@oracle.com>

Fixed. Thanks.

> On 28 Feb 2023, at 21:11, John Hendrikx <hjohn at xs4all.nl> wrote:
> 
> In https://openjdk.org/jeps/8300542, the example in "Improved support for enum constant case labels" seems to be incorrect.
> 
> I think the `goodEnumSwitch2`:
> 
>    static void goodEnumSwitch2(Currency c) {
> 
> Should use `Coin` as parameter:
> 
>    static void goodEnumSwitch2(Coin c) {
> 
> Also, the enum is called `Coin`, but `Coins` is used a qualifier in several places.
> 
> --John
> 
> ------ Original Message ------
> From "Gavin Bierman" <gavin.bierman at oracle.com>
> To "amber-dev at openjdk.org" <amber-dev at openjdk.org>
> Cc "amber-spec-experts" <amber-spec-experts at openjdk.org>
> Date 28/02/2023 17:21:42
> Subject Draft JEPs for Patterns in switch and Record Patterns
> 
>> Hello,
>> 
>> We are planning to finalize the two pattern matching JEPs in JDK 21. Drafts of
>> these final JEPs are available here:
>> 
>> Pattern matching for switch: https://openjdk.org/jeps/8300542
>> Record patterns:             https://openjdk.org/jeps/8300541
>> 
>> We're proposing some small changes from the preview versions about to appear in
>> JDK 20. These include:
>> 
>> - We're dropping parenthesized patterns. They were leftover from a previous
>> version of patterns, and they weren't used very much. They complicate the spec
>> for not a lot of gain.
>> 
>> - We're going to support case labels that are the qualified name of enum
>> constants, and allow switches over non-enum types to have enum case labels
>> provided they use the qualified names of the enum constants and these labels
>> are assignment compatible with the switch type.
>> 
>> - We're dropping the support for record patterns in the header of enhanced for
>> statements. These will re-appear in a separate forthcoming JEP.
>> 
>> Please take a look at these new JEPs and give us your feedback (either on this
>> list or directly to me).
>> 
>> Thanks,
>> Gavin
>> 
> 


From gavin.bierman at oracle.com  Wed Mar  1 00:42:40 2023
From: gavin.bierman at oracle.com (Gavin Bierman)
Date: Wed, 1 Mar 2023 00:42:40 +0000
Subject: Draft JEPs for Patterns in switch and Record Patterns
In-Reply-To: <1668423924.590526.1677619805010.JavaMail.zimbra@u-pem.fr>
References: <322686D2-BD86-4F05-BA33-C4017C772DB5@oracle.com>
 <em8dd9e7c9-1fb0-48b8-b735-562264471a7a@888eef6f.com>
 <1668423924.590526.1677619805010.JavaMail.zimbra@u-pem.fr>
Message-ID: <C5B315B6-453D-4278-95C2-79D01A456455@oracle.com>

Fixed. Thanks.

> On 28 Feb 2023, at 21:30, Remi Forax <forax at univ-mlv.fr> wrote:
> 
> ----- Original Message -----
>> From: "John Hendrikx" <hjohn at xs4all.nl>
>> To: "Gavin Bierman" <gavin.bierman at oracle.com>, "amber-dev" <amber-dev at openjdk.org>
>> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.org>
>> Sent: Tuesday, February 28, 2023 10:11:39 PM
>> Subject: Re: Draft JEPs for Patterns in switch and Record Patterns
> 
>> In https://openjdk.org/jeps/8300542, the example in "Improved support
>> for enum constant case labels" seems to be incorrect.
>> 
>> I think the `goodEnumSwitch2`:
>> 
>>    static void goodEnumSwitch2(Currency c) {
>> 
>> Should use `Coin` as parameter:
>> 
>>    static void goodEnumSwitch2(Coin c) {
>> 
>> Also, the enum is called `Coin`, but `Coins` is used a qualifier in
>> several places.
>> 
>> --John
> 
> Also s.equalsIgnoreCase("YES") is better than s.toUpperCase().equals("YES")
> 
> R?mi
> 
>> 
>> ------ Original Message ------
>> From "Gavin Bierman" <gavin.bierman at oracle.com>
>> To "amber-dev at openjdk.org" <amber-dev at openjdk.org>
>> Cc "amber-spec-experts" <amber-spec-experts at openjdk.org>
>> Date 28/02/2023 17:21:42
>> Subject Draft JEPs for Patterns in switch and Record Patterns
>> 
>>> Hello,
>>> 
>>> We are planning to finalize the two pattern matching JEPs in JDK 21. Drafts of
>>> these final JEPs are available here:
>>> 
>>> Pattern matching for switch: https://openjdk.org/jeps/8300542
>>> Record patterns:             https://openjdk.org/jeps/8300541
>>> 
>>> We're proposing some small changes from the preview versions about to appear in
>>> JDK 20. These include:
>>> 
>>> - We're dropping parenthesized patterns. They were leftover from a previous
>>> version of patterns, and they weren't used very much. They complicate the spec
>>> for not a lot of gain.
>>> 
>>> - We're going to support case labels that are the qualified name of enum
>>> constants, and allow switches over non-enum types to have enum case labels
>>> provided they use the qualified names of the enum constants and these labels
>>> are assignment compatible with the switch type.
>>> 
>>> - We're dropping the support for record patterns in the header of enhanced for
>>> statements. These will re-appear in a separate forthcoming JEP.
>>> 
>>> Please take a look at these new JEPs and give us your feedback (either on this
>>> list or directly to me).
>>> 
>>> Thanks,
>>> Gavin


From gavin.bierman at oracle.com  Wed Mar  1 14:17:30 2023
From: gavin.bierman at oracle.com (Gavin Bierman)
Date: Wed, 1 Mar 2023 14:17:30 +0000
Subject: Third draft spec for String Templates (JEP 430) now available
In-Reply-To: <5b74e6f9-508a-548e-a674-35135a680243@oracle.com>
References: <00A12ECB-155D-4EFE-B257-D05C998387CD@oracle.com>
 <12D00B25-7539-49AE-BB6E-C00DA3E38DEE@oracle.com>
 <5b74e6f9-508a-548e-a674-35135a680243@oracle.com>
Message-ID: <F3F903C3-8424-4E5E-A85E-C7329A97CD53@oracle.com>

Thanks Alex (and apologies for missing your earlier email - don?t know what happened there).

Updated spec available at:

https://cr.openjdk.org/~gbierman/jep430/latest/

Thanks,
Gavin

On 28 Feb 2023, at 18:37, Alex Buckley <alex.buckley at oracle.com<mailto:alex.buckley at oracle.com>> wrote:

1. Escape sequences are legal in fragments, so 3.10.7 should open with something like:

"In character literals, string literals, and text blocks (?3.10.4, ?3.10.5, ?3.10.6), +and in fragments of a template (3.13),+ the escape sequences allow for ..."


2. 3.10.7 goes on to say that "An escape sequence in the content of a character literal, string literal, or text block is interpreted by ...". This statement constitutes one half of a two-part exposition of interpretation, the other half being found in 3.10.4/5/6 where `String.translateEscapes` is mentioned. Two-way exposition is hard to write, expensive to maintain, but helpful to read. 3.13 is upfront that StringTemplateBegin, StringTemplateMid, and StringTemplateEnd tokens enjoy interpretation, so let's get them named in 3.10.7. 3.13 is less upfront about TextBlockTemplateBegin, TextBlockTemplateMid, and TextBlockTemplateEnd tokens, let me set them aside for now.


3. 3.10.7 should end with a note that the character sequence \{ is not an escape sequence but rather has meaning in a template (3.13).


4. In 3.13, consider "A fragment ~consists of~ +represents+ a non-expression part of a template."  (Ch.3 is big on "representing", or perhaps "denoting", things.)  This sentence would work well in the opening paragraph of the section.


5. 15.8.6 needs more polishing, per the mail I sent about it last time. I'll discuss offline.

Alex

On 2/28/2023 8:15 AM, Gavin Bierman wrote:
Following the feedback, a small update of the String Templates Spec is now available:
https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html <https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html>
Any further comments welcomed!
Thanks,
Gavin
On 9 Feb 2023, at 15:28, Gavin Bierman <gavin.bierman at oracle.com<mailto:gavin.bierman at oracle.com> <mailto:gavin.bierman at oracle.com>> wrote:

Dear experts:

A new, updated spec covering JEP 430 (String Templates) is now available at:

https://cr.openjdk.java.net/~gbierman/jep430/latest <https://cr.openjdk.java.net/~gbierman/jep430/latest>

This is a substantial rewrite: it now covers more fully how templates are tokenized, how to deal with ambiguities, and how text block templates are dealt with.

Comments welcomed!
Gavin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230301/948dd278/attachment-0001.htm>

From alex.buckley at oracle.com  Wed Mar  1 18:19:27 2023
From: alex.buckley at oracle.com (Alex Buckley)
Date: Wed, 1 Mar 2023 10:19:27 -0800
Subject: Third draft spec for String Templates (JEP 430) now available
In-Reply-To: <F3F903C3-8424-4E5E-A85E-C7329A97CD53@oracle.com>
References: <00A12ECB-155D-4EFE-B257-D05C998387CD@oracle.com>
 <12D00B25-7539-49AE-BB6E-C00DA3E38DEE@oracle.com>
 <5b74e6f9-508a-548e-a674-35135a680243@oracle.com>
 <F3F903C3-8424-4E5E-A85E-C7329A97CD53@oracle.com>
Message-ID: <a55a4349-9b4b-ad00-2dd1-0645f7aced9a@oracle.com>

Ch.3 looks good.

Some points about 15.8.6:

1. This pair of sentences is very good and clear:

   A string template with n embedded expressions (n>0) consists of the 
alternate interleaving of n+1 fragments with the n embedded expressions. 
The first fragment is a StringTemplateBegin token (3.13); the next n-1 
fragments are StringTemplateMid tokens; the last fragment is a 
StringTemplateEnd token.

2. I think this sentence is a leftover and should be removed:

   A string template with n (n > 0) embedded expressions, consists of 
the alternate interleaving of n+1 fragments (one StringTemplateBegin 
token, n-1 StringTemplateMid tokens, and one StringTemplateEnd token ) 
with the n embedded expressions.

3. I think this sentence is correct but should be phrased as a pair of 
sentences, like in #1 above.

   A text block template with n (n > 0) embedded expressions, consists 
of the alternate interleaving of n+1 fragments (one 
TextBlockTemplateBegin token, n-1 TextBlockTemplateMid tokens, and one 
TextBlockTemplateEnd token) with the n embedded expressions.

4. Consider this sentence:

   A string template with n embedded expressions and n+1 fragments has 
n+1 fragment strings, where each fragment string is the content of the 
corresponding fragment (3.13).

(In the previous draft, we said the same thing, worded differently: From 
the fragments in a string template, a sequence of corresponding fragment 
strings is derived by taking the content of each fragment token (3.13).)

However, the _content_ of the corresponding fragment in 3.13 does not 
have its escape sequences interpreted. I'm pretty sure you want its 
escape sequences to have been interpreted. For that, you need 15.8.6's 
fragment strings to lean on this artifact from 3.13:

   The string represented by a StringTemplateBegin, StringTemplateMid, 
or StringTemplateEnd token is given by its content with every escape 
sequence interpreted, as if by execution of String.translateEscapes on 
the content.

5. I'm not a fan of the factoring whereby escape sequence interpretation 
happens for a string template's tokens in 3.13 but a text block 
template's tokens in 15.8.6. I realize why text block templates are more 
complicated, but I would like to avoid the reader flipping between the 
sections and saying "Didn't we do this already?" Perhaps we just defer 
string templates' escape sequence interpretation to 15.8.6. That said, 
refactoring this topic is not urgent, and can wait until after preview.

Alex

On 3/1/2023 6:17 AM, Gavin Bierman wrote:
> Thanks Alex (and apologies for missing your earlier email - don?t know 
> what happened there).
> 
> Updated spec available at:
> 
> https://cr.openjdk.org/~gbierman/jep430/latest/ 
> <https://cr.openjdk.org/~gbierman/jep430/latest/>
> 
> Thanks,
> Gavin
> 
>> On 28 Feb 2023, at 18:37, Alex Buckley <alex.buckley at oracle.com 
>> <mailto:alex.buckley at oracle.com>> wrote:
>>
>> 1. Escape sequences are legal in fragments, so 3.10.7 should open with 
>> something like:
>>
>> "In character literals, string literals, and text blocks (?3.10.4, 
>> ?3.10.5, ?3.10.6), +and in fragments of a template (3.13),+ the escape 
>> sequences allow for ..."
>>
>>
>> 2. 3.10.7 goes on to say that "An escape sequence in the content of a 
>> character literal, string literal, or text block is interpreted by 
>> ...". This statement constitutes one half of a two-part exposition of 
>> interpretation, the other half being found in 3.10.4/5/6 where 
>> `String.translateEscapes` is mentioned. Two-way exposition is hard to 
>> write, expensive to maintain, but helpful to read. 3.13 is upfront 
>> that StringTemplateBegin, StringTemplateMid, and StringTemplateEnd 
>> tokens enjoy interpretation, so let's get them named in 3.10.7. 3.13 
>> is less upfront about TextBlockTemplateBegin, TextBlockTemplateMid, 
>> and TextBlockTemplateEnd tokens, let me set them aside for now.
>>
>>
>> 3. 3.10.7 should end with a note that the character sequence \{ is not 
>> an escape sequence but rather has meaning in a template (3.13).
>>
>>
>> 4. In 3.13, consider "A fragment ~consists of~ +represents+ a 
>> non-expression part of a template." ?(Ch.3 is big on "representing", 
>> or perhaps "denoting", things.) ?This sentence would work well in the 
>> opening paragraph of the section.
>>
>>
>> 5. 15.8.6 needs more polishing, per the mail I sent about it last 
>> time. I'll discuss offline.
>>
>> Alex
>>
>> On 2/28/2023 8:15 AM, Gavin Bierman wrote:
>>> Following the feedback, a small update of the String Templates Spec 
>>> is now available:
>>> https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html <https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html> <https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html <https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html>>
>>> Any further comments welcomed!
>>> Thanks,
>>> Gavin
>>>> On 9 Feb 2023, at 15:28, Gavin Bierman <gavin.bierman at oracle.com 
>>>> <mailto:gavin.bierman at oracle.com> <mailto:gavin.bierman at oracle.com 
>>>> <mailto:gavin.bierman at oracle.com>>> wrote:
>>>>
>>>> Dear experts:
>>>>
>>>> A new, updated spec covering JEP 430 (String Templates) is now 
>>>> available at:
>>>>
>>>> https://cr.openjdk.java.net/~gbierman/jep430/latest 
>>>> <https://cr.openjdk.java.net/~gbierman/jep430/latest> 
>>>> <https://cr.openjdk.java.net/~gbierman/jep430/latest 
>>>> <https://cr.openjdk.java.net/~gbierman/jep430/latest>>
>>>>
>>>> This is a substantial rewrite: it now covers more fully how 
>>>> templates are tokenized, how to deal with ambiguities, and how text 
>>>> block templates are dealt with.
>>>>
>>>> Comments welcomed!
>>>> Gavin
> 

From gavin.bierman at oracle.com  Thu Mar  2 10:37:54 2023
From: gavin.bierman at oracle.com (Gavin Bierman)
Date: Thu, 2 Mar 2023 10:37:54 +0000
Subject: Third draft spec for String Templates (JEP 430) now available
In-Reply-To: <a55a4349-9b4b-ad00-2dd1-0645f7aced9a@oracle.com>
References: <00A12ECB-155D-4EFE-B257-D05C998387CD@oracle.com>
 <12D00B25-7539-49AE-BB6E-C00DA3E38DEE@oracle.com>
 <5b74e6f9-508a-548e-a674-35135a680243@oracle.com>
 <F3F903C3-8424-4E5E-A85E-C7329A97CD53@oracle.com>
 <a55a4349-9b4b-ad00-2dd1-0645f7aced9a@oracle.com>
Message-ID: <FB44BEE7-B50A-4551-98B8-F1314F2323CA@oracle.com>

Thanks Alex. Updated.

> On 1 Mar 2023, at 18:19, Alex Buckley <alex.buckley at oracle.com> wrote:
> 
> Ch.3 looks good.
> 
> Some points about 15.8.6:
> 
> 1. This pair of sentences is very good and clear:
> 
>  A string template with n embedded expressions (n>0) consists of the alternate interleaving of n+1 fragments with the n embedded expressions. The first fragment is a StringTemplateBegin token (3.13); the next n-1 fragments are StringTemplateMid tokens; the last fragment is a StringTemplateEnd token.
> 
> 2. I think this sentence is a leftover and should be removed:
> 
>  A string template with n (n > 0) embedded expressions, consists of the alternate interleaving of n+1 fragments (one StringTemplateBegin token, n-1 StringTemplateMid tokens, and one StringTemplateEnd token ) with the n embedded expressions.
> 
> 3. I think this sentence is correct but should be phrased as a pair of sentences, like in #1 above.
> 
>  A text block template with n (n > 0) embedded expressions, consists of the alternate interleaving of n+1 fragments (one TextBlockTemplateBegin token, n-1 TextBlockTemplateMid tokens, and one TextBlockTemplateEnd token) with the n embedded expressions.
> 
> 4. Consider this sentence:
> 
>  A string template with n embedded expressions and n+1 fragments has n+1 fragment strings, where each fragment string is the content of the corresponding fragment (3.13).
> 
> (In the previous draft, we said the same thing, worded differently: From the fragments in a string template, a sequence of corresponding fragment strings is derived by taking the content of each fragment token (3.13).)
> 
> However, the _content_ of the corresponding fragment in 3.13 does not have its escape sequences interpreted. I'm pretty sure you want its escape sequences to have been interpreted. For that, you need 15.8.6's fragment strings to lean on this artifact from 3.13:
> 
>  The string represented by a StringTemplateBegin, StringTemplateMid, or StringTemplateEnd token is given by its content with every escape sequence interpreted, as if by execution of String.translateEscapes on the content.
> 
> 5. I'm not a fan of the factoring whereby escape sequence interpretation happens for a string template's tokens in 3.13 but a text block template's tokens in 15.8.6. I realize why text block templates are more complicated, but I would like to avoid the reader flipping between the sections and saying "Didn't we do this already?" Perhaps we just defer string templates' escape sequence interpretation to 15.8.6. That said, refactoring this topic is not urgent, and can wait until after preview.
> 
> Alex
> 
> On 3/1/2023 6:17 AM, Gavin Bierman wrote:
>> Thanks Alex (and apologies for missing your earlier email - don?t know what happened there).
>> Updated spec available at:
>> https://cr.openjdk.org/~gbierman/jep430/latest/ <https://cr.openjdk.org/~gbierman/jep430/latest/>
>> Thanks,
>> Gavin
>>> On 28 Feb 2023, at 18:37, Alex Buckley <alex.buckley at oracle.com <mailto:alex.buckley at oracle.com>> wrote:
>>> 
>>> 1. Escape sequences are legal in fragments, so 3.10.7 should open with something like:
>>> 
>>> "In character literals, string literals, and text blocks (?3.10.4, ?3.10.5, ?3.10.6), +and in fragments of a template (3.13),+ the escape sequences allow for ..."
>>> 
>>> 
>>> 2. 3.10.7 goes on to say that "An escape sequence in the content of a character literal, string literal, or text block is interpreted by ...". This statement constitutes one half of a two-part exposition of interpretation, the other half being found in 3.10.4/5/6 where `String.translateEscapes` is mentioned. Two-way exposition is hard to write, expensive to maintain, but helpful to read. 3.13 is upfront that StringTemplateBegin, StringTemplateMid, and StringTemplateEnd tokens enjoy interpretation, so let's get them named in 3.10.7. 3.13 is less upfront about TextBlockTemplateBegin, TextBlockTemplateMid, and TextBlockTemplateEnd tokens, let me set them aside for now.
>>> 
>>> 
>>> 3. 3.10.7 should end with a note that the character sequence \{ is not an escape sequence but rather has meaning in a template (3.13).
>>> 
>>> 
>>> 4. In 3.13, consider "A fragment ~consists of~ +represents+ a non-expression part of a template."  (Ch.3 is big on "representing", or perhaps "denoting", things.)  This sentence would work well in the opening paragraph of the section.
>>> 
>>> 
>>> 5. 15.8.6 needs more polishing, per the mail I sent about it last time. I'll discuss offline.
>>> 
>>> Alex
>>> 
>>> On 2/28/2023 8:15 AM, Gavin Bierman wrote:
>>>> Following the feedback, a small update of the String Templates Spec is now available:
>>>> https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html <https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html> <https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html <https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html>>
>>>> Any further comments welcomed!
>>>> Thanks,
>>>> Gavin
>>>>> On 9 Feb 2023, at 15:28, Gavin Bierman <gavin.bierman at oracle.com <mailto:gavin.bierman at oracle.com> <mailto:gavin.bierman at oracle.com <mailto:gavin.bierman at oracle.com>>> wrote:
>>>>> 
>>>>> Dear experts:
>>>>> 
>>>>> A new, updated spec covering JEP 430 (String Templates) is now available at:
>>>>> 
>>>>> https://cr.openjdk.java.net/~gbierman/jep430/latest <https://cr.openjdk.java.net/~gbierman/jep430/latest> <https://cr.openjdk.java.net/~gbierman/jep430/latest <https://cr.openjdk.java.net/~gbierman/jep430/latest>>
>>>>> 
>>>>> This is a substantial rewrite: it now covers more fully how templates are tokenized, how to deal with ambiguities, and how text block templates are dealt with.
>>>>> 
>>>>> Comments welcomed!
>>>>> Gavin


From alex.buckley at oracle.com  Thu Mar  2 16:33:24 2023
From: alex.buckley at oracle.com (Alex Buckley)
Date: Thu, 2 Mar 2023 08:33:24 -0800
Subject: Third draft spec for String Templates (JEP 430) now available
In-Reply-To: <FB44BEE7-B50A-4551-98B8-F1314F2323CA@oracle.com>
References: <00A12ECB-155D-4EFE-B257-D05C998387CD@oracle.com>
 <12D00B25-7539-49AE-BB6E-C00DA3E38DEE@oracle.com>
 <5b74e6f9-508a-548e-a674-35135a680243@oracle.com>
 <F3F903C3-8424-4E5E-A85E-C7329A97CD53@oracle.com>
 <a55a4349-9b4b-ad00-2dd1-0645f7aced9a@oracle.com>
 <FB44BEE7-B50A-4551-98B8-F1314F2323CA@oracle.com>
Message-ID: <33369ccf-8dc1-19ba-d9f2-97b2473e8854@oracle.com>

All looks good.

Alex

On 3/2/2023 2:37 AM, Gavin Bierman wrote:
> Thanks Alex. Updated.
> 
>> On 1 Mar 2023, at 18:19, Alex Buckley <alex.buckley at oracle.com> wrote:
>>
>> Ch.3 looks good.
>>
>> Some points about 15.8.6:
>>
>> 1. This pair of sentences is very good and clear:
>>
>>   A string template with n embedded expressions (n>0) consists of the alternate interleaving of n+1 fragments with the n embedded expressions. The first fragment is a StringTemplateBegin token (3.13); the next n-1 fragments are StringTemplateMid tokens; the last fragment is a StringTemplateEnd token.
>>
>> 2. I think this sentence is a leftover and should be removed:
>>
>>   A string template with n (n > 0) embedded expressions, consists of the alternate interleaving of n+1 fragments (one StringTemplateBegin token, n-1 StringTemplateMid tokens, and one StringTemplateEnd token ) with the n embedded expressions.
>>
>> 3. I think this sentence is correct but should be phrased as a pair of sentences, like in #1 above.
>>
>>   A text block template with n (n > 0) embedded expressions, consists of the alternate interleaving of n+1 fragments (one TextBlockTemplateBegin token, n-1 TextBlockTemplateMid tokens, and one TextBlockTemplateEnd token) with the n embedded expressions.
>>
>> 4. Consider this sentence:
>>
>>   A string template with n embedded expressions and n+1 fragments has n+1 fragment strings, where each fragment string is the content of the corresponding fragment (3.13).
>>
>> (In the previous draft, we said the same thing, worded differently: From the fragments in a string template, a sequence of corresponding fragment strings is derived by taking the content of each fragment token (3.13).)
>>
>> However, the _content_ of the corresponding fragment in 3.13 does not have its escape sequences interpreted. I'm pretty sure you want its escape sequences to have been interpreted. For that, you need 15.8.6's fragment strings to lean on this artifact from 3.13:
>>
>>   The string represented by a StringTemplateBegin, StringTemplateMid, or StringTemplateEnd token is given by its content with every escape sequence interpreted, as if by execution of String.translateEscapes on the content.
>>
>> 5. I'm not a fan of the factoring whereby escape sequence interpretation happens for a string template's tokens in 3.13 but a text block template's tokens in 15.8.6. I realize why text block templates are more complicated, but I would like to avoid the reader flipping between the sections and saying "Didn't we do this already?" Perhaps we just defer string templates' escape sequence interpretation to 15.8.6. That said, refactoring this topic is not urgent, and can wait until after preview.
>>
>> Alex
>>
>> On 3/1/2023 6:17 AM, Gavin Bierman wrote:
>>> Thanks Alex (and apologies for missing your earlier email - don?t know what happened there).
>>> Updated spec available at:
>>> https://cr.openjdk.org/~gbierman/jep430/latest/ <https://cr.openjdk.org/~gbierman/jep430/latest/>
>>> Thanks,
>>> Gavin
>>>> On 28 Feb 2023, at 18:37, Alex Buckley <alex.buckley at oracle.com <mailto:alex.buckley at oracle.com>> wrote:
>>>>
>>>> 1. Escape sequences are legal in fragments, so 3.10.7 should open with something like:
>>>>
>>>> "In character literals, string literals, and text blocks (?3.10.4, ?3.10.5, ?3.10.6), +and in fragments of a template (3.13),+ the escape sequences allow for ..."
>>>>
>>>>
>>>> 2. 3.10.7 goes on to say that "An escape sequence in the content of a character literal, string literal, or text block is interpreted by ...". This statement constitutes one half of a two-part exposition of interpretation, the other half being found in 3.10.4/5/6 where `String.translateEscapes` is mentioned. Two-way exposition is hard to write, expensive to maintain, but helpful to read. 3.13 is upfront that StringTemplateBegin, StringTemplateMid, and StringTemplateEnd tokens enjoy interpretation, so let's get them named in 3.10.7. 3.13 is less upfront about TextBlockTemplateBegin, TextBlockTemplateMid, and TextBlockTemplateEnd tokens, let me set them aside for now.
>>>>
>>>>
>>>> 3. 3.10.7 should end with a note that the character sequence \{ is not an escape sequence but rather has meaning in a template (3.13).
>>>>
>>>>
>>>> 4. In 3.13, consider "A fragment ~consists of~ +represents+ a non-expression part of a template."  (Ch.3 is big on "representing", or perhaps "denoting", things.)  This sentence would work well in the opening paragraph of the section.
>>>>
>>>>
>>>> 5. 15.8.6 needs more polishing, per the mail I sent about it last time. I'll discuss offline.
>>>>
>>>> Alex
>>>>
>>>> On 2/28/2023 8:15 AM, Gavin Bierman wrote:
>>>>> Following the feedback, a small update of the String Templates Spec is now available:
>>>>> https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html <https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html> <https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html <https://cr.openjdk.org/~gbierman/jep430/jep430-20230222/specs/string-templates-jls.html>>
>>>>> Any further comments welcomed!
>>>>> Thanks,
>>>>> Gavin
>>>>>> On 9 Feb 2023, at 15:28, Gavin Bierman <gavin.bierman at oracle.com <mailto:gavin.bierman at oracle.com> <mailto:gavin.bierman at oracle.com <mailto:gavin.bierman at oracle.com>>> wrote:
>>>>>>
>>>>>> Dear experts:
>>>>>>
>>>>>> A new, updated spec covering JEP 430 (String Templates) is now available at:
>>>>>>
>>>>>> https://cr.openjdk.java.net/~gbierman/jep430/latest <https://cr.openjdk.java.net/~gbierman/jep430/latest> <https://cr.openjdk.java.net/~gbierman/jep430/latest <https://cr.openjdk.java.net/~gbierman/jep430/latest>>
>>>>>>
>>>>>> This is a substantial rewrite: it now covers more fully how templates are tokenized, how to deal with ambiguities, and how text block templates are dealt with.
>>>>>>
>>>>>> Comments welcomed!
>>>>>> Gavin
> 

From brian.goetz at oracle.com  Mon Mar  6 18:24:54 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Mon, 6 Mar 2023 13:24:54 -0500
Subject: Deconstruction patterns
Message-ID: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>

Time to look ahead to the next installment of pattern matching: 
deconstruction patterns, which generalize record patterns.? This 
document does an end-to-end walkthrough (at a sketchy level of detail) 
through declaration, overloading, use, translation, and reflection of 
deconstruction patterns.

I would like to *not* discuss syntax at this time.? There's a lengthy 
discussion to be had about syntax, and we'll have that, but let's nail 
down model, semantics, and translation first.

As usual, I would prefer that people either (a) post a single reply 
addressing the totality of this sketch or (b) start _new threads_ if you 
want to discuss a specific aspect.? A quick "I'll just reply to this 
minor detail" seems to often derail the conversation in such a way that 
it never comes back.? If this all looks fine to you, a quick "no 
surprises here" will keep us from suspensefully waiting for feedback.


# Deconstruction patterns -- translation, use, and reflection

As we are wrapping up record patterns, it's time to look ahead to the 
next major
part of the pattern matching story -- extending the capabilities of record
patterns to all classes that want to support destructuring. Record 
patterns are
simply a special case of _deconstruction patterns_ or _deconstructors_, 
where we
derive the deconstructor API, implementation, and use from the state 
description
of the record.? For an arbitrary class, a deconstruction patterns will 
require
an explicit member declaration, with a header identifying the names and 
types of
the bindings and a body that extracts the bindings from the representation.

## Deconstructors

Just as constructors are special cases of methods, deconstruction 
patterns are
special cases of a more general notion of declared pattern, which also 
includes
static matchers (the dual of static methods) and instance matchers (the 
dual of
instance methods.)? Specifically, unlike the more general notion of 
matcher, a
deconstructor must be _total_; it must always match.? This document will 
focus
exclusively on deconstructors, and we'll come back to static and instance
matchers in due time.? (But note that some of the design choices in the 
simple
case of deconstructors may be constrained by the more general case.)

There are a number of choices for how we might syntactically represent a
deconstructor (or more generally, a declared pattern.)? For purposes of
illustration, this document picks one possible syntactic expression of
deconstructors, but it is premature to devolve into a syntax discussion 
at this
time.

```
class Point {
 ??? final double x, y;

 ??? public Point(double x, double y) {
 ??????? this.x = x;
 ??????? this.y = y;
 ??? }

 ??? public matcher Point(double x, double y) {
 ??????? x = this.x;
 ??????? y = this.y;
 ??? }
}
```

This example illustrates two aspects of the duality between constructors and
their corresponding deconstructors.? Their APIs are duals: a constructor 
takes N
parameters containing the desired description of the object state and 
produces a
constructed object; a deconstructor starts from the constructed object 
and has N
bindings (outputs) that receive the desired state components. Similarly, 
their
implementations are duals: the body of the constructor initializes the 
object
representation from the description, and the body of the deconstructor 
extracts
the description from the representation.? A deconstructor is best 
understood as
a _co-constructor_.

The `Point` example above is special in two ways.? First, the internal
representation of a `Point`, and the API of the constructor and 
deconstructor,
are the same: `(double x, double y)`.? We can call the API implied by the
constructor and deconstructor the _external representation_, and for 
`Point`,
both the internal and external representations are the same. (This is one of
the requirements for being a candidate to be a record.)? And second, the
constructor is _total_; it does not reject any combinations of arguments.

Here's another version of `Point` which does not have these special 
aspects; it
uses the same internal representation as before, but chooses a pair of 
strings
as the external representation:

```
class Point2 {
 ??? final double x, y;

 ??? public Point2(String x, String y) {
 ??????? this.x = Double.parseDouble(x);
 ??????? this.y = Double.parseDouble(y);
 ??? }

 ??? public matcher Point2(String x, String y) {
 ??????? x = Double.toString(this.x);
 ??????? y = Double.toSTring(this.y);
 ??? }
}
```

The method `Double::parseDouble` will throw `NumberFormatException` if its
argument does not describe a suitable value, so unlike the `Point` 
constructor,
the `Point2` constructor is partial: it will reject `new Double("foo", 
"bar")`.
And the internal representation is no longer the same as the external
representation.? Less obviously, there are valid string values that we can
provide to the constructor, but which cannot be represented exactly as 
`double`,
and which will be approximated; the string value
`"3.22222222222222222222222222222222222222"` will be approximated with the
double value `3.2222222222222223`.

This example highlights more clearly how the constructor and 
deconstructor form
an _embedding-projection pair_ between the internal and external
representations.? While some external representations might be invalid, 
and some
might result in approximation, deconstruct-then-construct is always an 
identity
transformation.? Indeed, the specification of `java.lang.Record` 
requires that
if we deconstruct a record with its accessors, and pass the resulting values
back to the constructor, we should get a new record that is `equals` to the
original.

The fact that constructor and deconstructor (and eventually, factory and 
static
matcher) form an embedding-projection pair is why we are able to derive
higher-level language features, such as [safer
serialization](https://openjdk.org/projects/amber/design-notes/towards-better-serialization)
and [functional transformation of immutable
objects](https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md),
from a matched set of constructor and deconstructor.

Of course, users are free to implement constructors without 
deconstructors, or
constructors and deconstructors whose external representations don't 
match up,
or even matching constructors and deconstructors that are not 
behaviorally dual.
But providing a matched set (or several) of constructors and deconstructors
enables reliably reversible aggregation, and allows us to mechanically 
derive
useful higher-level features such as withers.

#### Overloading

Just as constructors can be overloaded, deconstructors can be overloaded 
for the
same reason: multiple constructors can expose multiple external 
representations
for aggregation, and corresponding deconstructors can recover those multiple
external representations.? Any matching pair of 
constructor-deconstructor (and
eventually, factory-deconstructor) is a candidate for use in higher-level
features based on the embedding-projection nature of the
constructor-deconstructor pair.

Just as deconstruction is dual to construction, overloading of 
deconstructors is
dual to that of constructors: rather than restricting which sets of 
parameters
can be overloaded against each other, we do so with the bindings 
instead.? For
constructors of a given arity, we require that their signatures not be
override-equivalent; for deconstructors of a given arity, we require the 
same of
their bindings.

For a deconstructor (and declared patterns in general), we derive a _binding
signature_ (and an erased _binding descriptor_) which treats the binding 
list as
a parameter list.? The overload rule outlined above requires that binding
signatures for two deconstructors of the same arity not be 
override-equivalent.
(We will find it useful later to derive a `MethodType` for the binding
descriptor; this is a `MethodType` whose return type is `V` and whose 
parameter
types are the erased types of the bindings.)

#### Digression: embedding-projection pairs

Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B 
-> A`,
forms an _embedding-projection pair_ if `p . e` (embed then project) is an
identity function, and `e . p` (project then embed) _approximates_ the input
according to a domain-specific approximation metric (which is a complete 
partial
ordering on `B`.)

When applied to constructor-deconstructor pairs, this says that 
deconstructing
an object and then reconstructing it with the resulting bindings should 
result
in an equivalent object, and constructing an object from an external
representation and then deconstructing it back into that external 
representation
should result in an approximation of the original external 
representation.? (A
complete partial ordering models constructor failure as the non-terminating
bottom value, which is considered an infinitely bad approximation to
everything.)

Embedding-projection pairs have a number of desirable properties, such 
as the
composition of two e-p pairs is an e-p pair; this property is at the 
heart of
using constructor-deconstructor pairs for improved serialization and 
functional
transformation.

## Invoking deconstructors

We've already seen how to "invoke" deconstructors: through pattern matching.
What we've been calling "record patterns" are merely deconstruction patterns
derived mechanically from the state description, just as we do with 
constructors
and accessors; there is little difference between record patterns and
deconstruction patterns other than the ability to declare them explicitly.
(There is an accidental difference in the translation, in that we currently
implement record patterns by appealing to individual accessors rather than a
single deconstructor, but this may eventually converge as well.)

The use-site syntax of deconstruction bears a deliberate similarity to 
that of
construction; `new Point(x, y)` is deconstructed by `case Point(var x, 
var y)`.

#### Overload selection

In the presence of overloaded deconstructors, we need to figure out which
deconstructor a deconstruction pattern `C(P*)` is referring to. The 
details are
similar to overload selection for methods, except that we operate on the
bindings rather than the parameters.? We first search for _applicable 
matchers_,
using increasingly loose criteria (first excluding boxing, unboxing, and
varargs; then including boxing and unboxing but not varargs; and 
finally, all
candidates) and then selecting the most applicable.

It is tempting to try and bypass the three-phase selection process and use a
simpler notion of applicability (perhaps noting that we got this process for
compatibility with existing overload selection decisions when autoboxing and
varargs were added, and that there are few deconstructor invocations to be
compatible with yet.)? But because existing overloaded constructors use this
mechanism, and there is significant value in pairing constructors and
deconstructors, attempting to invent a simpler-but-different overload 
selection
mechanism for deconstructors would inevitably undermine the duality between
matching constructor-deconstructor pairs. So compatibility (this time, with
existing overloaded constructors) once again forces our hand.

The specification for overload selection is complicated significantly by 
poly
expressions (e.g., lambdas); fortunately, there are no "poly patterns", 
and so,
while the structure of JLS 15.12.2 is retained for overload selection of
deconstruction patterns, much of the detail is left behind.

## Translation

We translate patterns into synthetic methods with a `Matcher` attribute; 
this
method implements the matcher behavior.? The translation scheme derives 
from a
number of requirements, only some of which are in play for deconstructors.

The matcher method for a deconstructor is a final instance method that 
takes no
parameters and returns `Object`, perhaps with a special name (just as
constructors are called `<init>`.)

#### Carriers

Because the matcher methods implements the matcher behavior, but a 
matcher may
"return" multiple bindings (or failure), we must encode the bindings in some
way.? For this, we use a _carrier object_.? The choice of carrier is 
largely a
footprint/specificity tradeoff.? One could imagine a carrier class per 
matcher,
or a carrier class per matcher descriptor, or using `Object[]` as a 
carrier for
everything, or caching some number of common shapes (e.g, three ints and two
refs).? This sort of tuning should be separate from the protocol encoded 
in the
bytecode of the pattern method and its clients.

We use a small _carrier runtime_ to decouple pattern translation from 
carrier
selection.? (This same carrier runtime is used by string templates as well.)
This allows tradeoffs in runtime characteristics (e.g., carrier per 
matcher vs
sharing carriers across matchers, dropping carrier identity with value types
later, etc) without affecting the translation. The carrier API consists 
of condy
bootstraps like:

```
static MethodHandle carrierFactory(MethodType matcherDescriptor) { ... }
static MethodHandle carrierAccessor(MethodType matcherDescriptor, int 
bindingNo) { ... }
```

The `matcherDescriptor` is a `MethodType` describing the binding types.? The
`carrierFactory` method returns a method handle which takes the bindings and
produces a carrier object; the `carrierAccessor` method returns method 
handles
that take the carrier object and return the corresponding binding.? To 
indicate
success, the matcher method invokes the carrier factory method handle and
returns the result; to indicate failure (deconstructors cannot fail, but 
other
matchers can) the matcher method returns null.

We would translate the XY deconstructor from `Point` as follows 
(pseudo-code):

```
#100: MethodType[(II)V]
#101: Condy[bsm=Carriers::carrierFactory, args=[#100]]

final synthetic Object Point$MANGLE() {
 ??? aload_0
 ??? getfield Point::x
 ??? aload_0
 ??? getfield Point::y
 ??? LDC #101
 ??? invokevirtual MethodHandle::invoke(II)V
 ??? areturn
}
```

Constant `#100` contains a `MethodType` holding the binding descriptor; 
constant
`#101` holds a method handle whose parameters are the parameter types of the
binding descriptor and returns `Object`.

At the use site, matching a deconstruction pattern is performed by 
invoking the
matcher method on the appropriate target object, and then extracting the
components with the carrier accessor method handles if the match is 
successful.
(Deconstructors are total, so are always successful, but for other patterns,
null is returned from the matcher method on failure to match.)

#### Method names

The name of the matcher method is mangled to support overloading. The JVM
permits overloading on parameter types, but not return types (and overloaded
matchers are effectively overloaded on return types.)? We take the 
approach of
encoding the erasure of the matcher descriptor in the name of the 
pattern.? This
has several desirable properties: it is stable (the name is derived 
solely from
stable aspects of the declaration), for matchers with override-equivalent
signatures (deconstructors can't be overridden, but other patterns can be),
these map to true overrides in the translation, and valid overloads of 
matchers
will always have distinct names.

We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
descriptor as the mangled disambiguator, which is exactly as stable as 
any other
method descriptor derived from source declarations.

#### Attributes

Because patterns are methods, we can take advantage of all the 
affordances of
methods.? We can use access bits to control accessibility; we can use the
attributes that carry annotations, method parameter metadata, and generics
signatures to carry information about the pattern declaration (and its 
(input)
parameters, when we get to those kinds of matchers).? What's missing is 
the fact
that this is a pattern implementation and not an ordinary method, and a 
place to
put metadata for bindings.? To address the first, we can add the following
attribute on matcher methods:

 ??? Matcher {
 ??????? u2 name;??????????????????????????? // "Matcher"
 ??????? u4 length;
 ??????? u2 patternFlags;
 ??????? u2 patternName;???????????????????? // UTF8
 ??????? u2 patternDescr;??????????????????? // MethodType
 ??????? u2 attributes_count;
 ??????? attribute_info attributes[attributes_count];
 ??? }

This says that "this method is a pattern".? The source name of the pattern
declaration is reified as `patternName`, and the matcher descriptor, which
encodes the types of the bindings, is reified as a `MethodType` in
`patternDescr`.? The `flags` word can carry matcher-specific information 
such as
"this matcher is a deconstructor" or "this matcher is total".

A matcher method may have the usual variety of method attributes, such as
`RuntimeInvisibleAnnotations` for annotations on the matcher declaration 
itself.

If we wish to encode information about the matcher _bindings_, we do so with
attributes inside the `Matcher` annotation itself.? Attributes such as
`Signature`, `ParameterNames`, `RuntimeVisibleParameterAnnotations`, 
etc, can
appear in a `Matcher` and are interpreted relative to the matcher 
signature or
descriptor.? So if we had a matcher:

```
matcher Foo(@Bar List<String> list) { ... }
```

then the `Matcher` would contain the signature attribute corresponding to
`(List<String>)` and a `RuntimeXxxParameterAnnotations` attribute 
describing the
`@Bar` annotation on the first "parameter".

#### Reflection

Since matchers are a new kind of class member, they will need a new kind of
reflective object, and a method that is analogous to 
`Class::getConstructors`.
The reflective object should extend `Executable`, as all of the existing 
methods
on `Executable` make sense for patterns (using `Object` as the return 
type.)? If
the pattern is reflectively invoked, it returns `null` for no match, or an
`Object[]` which is the boxing of the values in the carrier.

We will then need some additional methods to describe the bindings, so the
subtype of `Executable` has methods like `getBindings`, 
`getAnnotatedBindings`,
`getGenericBindings`, `isDeconstructor`, `isPartial`, etc.? These 
methods will
decode the `Matcher` attribute and its embedded attributes.

## Summary

This design borrows from previous rounds, but makes a number of 
simplifications.

 ?- The bindings of a pattern are captured in a `MethodType`, called the 
_matcher
 ?? descriptor_.? The parameters of the matcher descriptor are the types 
of the
 ?? bindings; the return type is either `V` or the minimal type that 
will match
 ?? (but is not as important as the bindings.)
 ?- Matchers are translated as methods whose names are derived 
deterministically
 ?? from the name of the matcher and the erasure of the pattern 
descriptor. These
 ?? are called _matcher methods_.? Matcher methods take as parameters 
the input
 ?? parameters of the pattern (if any), and return `Object`.
 ?- The returned object is an opaque carrier.? Null means the pattern didn't
 ?? match.? A non-null value is the carrier type (from the carrier 
runtime) which
 ?? is derived from the pattern descriptor.
 ?- Matcher methods are not directly invocable from the source language; 
they are
 ?? invoked indirectly through pattern matching or reflection.
 ?- Generated code invokes the matcher method and interprets the 
returned value
 ?? according to the protocol, using MHs from the carrier runtime to 
access the
 ?? bindings.
 ?- Matcher methods have a `Matcher` attribute, which captures 
information about
 ?? the matcher as a whole (is a total/partial, a deconstructor, etc) and
 ?? parameter-related attributes which describe the bindings.
 ?- Matchers are reflected through a new subtype of `Executable`, which 
exposes
 ?? new methods to reflect over bindings.
 ?- When invoking a matcher reflectively, the carrier is boxed to an 
Object[].




From forax at univ-mlv.fr  Tue Mar  7 08:55:47 2023
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 7 Mar 2023 09:55:47 +0100 (CET)
Subject: Overloading of matcher method Was: Deconstruction patterns
In-Reply-To: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
Message-ID: <969212125.5143862.1678179347294.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Monday, March 6, 2023 7:24:54 PM
> Subject: Deconstruction patterns

> Time to look ahead to the next installment of pattern matching:
> deconstruction patterns, which generalize record patterns.? This
> document does an end-to-end walkthrough (at a sketchy level of detail)
> through declaration, overloading, use, translation, and reflection of
> deconstruction patterns.
> 
> I would like to *not* discuss syntax at this time.? There's a lengthy
> discussion to be had about syntax, and we'll have that, but let's nail
> down model, semantics, and translation first.
> 
> As usual, I would prefer that people either (a) post a single reply
> addressing the totality of this sketch or (b) start _new threads_ if you
> want to discuss a specific aspect.? A quick "I'll just reply to this
> minor detail" seems to often derail the conversation in such a way that
> it never comes back.? If this all looks fine to you, a quick "no
> surprises here" will keep us from suspensefully waiting for feedback.
> 
> 
> # Deconstruction patterns -- translation, use, and reflection
> 
> As we are wrapping up record patterns, it's time to look ahead to the
> next major
> part of the pattern matching story -- extending the capabilities of record
> patterns to all classes that want to support destructuring. Record
> patterns are
> simply a special case of _deconstruction patterns_ or _deconstructors_,
> where we
> derive the deconstructor API, implementation, and use from the state
> description
> of the record.? For an arbitrary class, a deconstruction patterns will
> require
> an explicit member declaration, with a header identifying the names and
> types of
> the bindings and a body that extracts the bindings from the representation.
> 
> ## Deconstructors
> 
> Just as constructors are special cases of methods, deconstruction
> patterns are
> special cases of a more general notion of declared pattern, which also
> includes
> static matchers (the dual of static methods) and instance matchers (the
> dual of
> instance methods.)? Specifically, unlike the more general notion of
> matcher, a
> deconstructor must be _total_; it must always match.? This document will
> focus
> exclusively on deconstructors, and we'll come back to static and instance
> matchers in due time.? (But note that some of the design choices in the
> simple
> case of deconstructors may be constrained by the more general case.)
> 
> There are a number of choices for how we might syntactically represent a
> deconstructor (or more generally, a declared pattern.)? For purposes of
> illustration, this document picks one possible syntactic expression of
> deconstructors, but it is premature to devolve into a syntax discussion
> at this
> time.
> 
> ```
> class Point {
> ??? final double x, y;
> 
> ??? public Point(double x, double y) {
> ??????? this.x = x;
> ??????? this.y = y;
> ??? }
> 
> ??? public matcher Point(double x, double y) {
> ??????? x = this.x;
> ??????? y = this.y;
> ??? }
> }
> ```
> 
> This example illustrates two aspects of the duality between constructors and
> their corresponding deconstructors.? Their APIs are duals: a constructor
> takes N
> parameters containing the desired description of the object state and
> produces a
> constructed object; a deconstructor starts from the constructed object
> and has N
> bindings (outputs) that receive the desired state components. Similarly,
> their
> implementations are duals: the body of the constructor initializes the
> object
> representation from the description, and the body of the deconstructor
> extracts
> the description from the representation.? A deconstructor is best
> understood as
> a _co-constructor_.
> 
> The `Point` example above is special in two ways.? First, the internal
> representation of a `Point`, and the API of the constructor and
> deconstructor,
> are the same: `(double x, double y)`.? We can call the API implied by the
> constructor and deconstructor the _external representation_, and for
> `Point`,
> both the internal and external representations are the same. (This is one of
> the requirements for being a candidate to be a record.)? And second, the
> constructor is _total_; it does not reject any combinations of arguments.
> 
> Here's another version of `Point` which does not have these special
> aspects; it
> uses the same internal representation as before, but chooses a pair of
> strings
> as the external representation:
> 
> ```
> class Point2 {
> ??? final double x, y;
> 
> ??? public Point2(String x, String y) {
> ??????? this.x = Double.parseDouble(x);
> ??????? this.y = Double.parseDouble(y);
> ??? }
> 
> ??? public matcher Point2(String x, String y) {
> ??????? x = Double.toString(this.x);
> ??????? y = Double.toSTring(this.y);
> ??? }
> }
> ```
> 
> The method `Double::parseDouble` will throw `NumberFormatException` if its
> argument does not describe a suitable value, so unlike the `Point`
> constructor,
> the `Point2` constructor is partial: it will reject `new Double("foo",
> "bar")`.
> And the internal representation is no longer the same as the external
> representation.? Less obviously, there are valid string values that we can
> provide to the constructor, but which cannot be represented exactly as
> `double`,
> and which will be approximated; the string value
> `"3.22222222222222222222222222222222222222"` will be approximated with the
> double value `3.2222222222222223`.
> 
> This example highlights more clearly how the constructor and
> deconstructor form
> an _embedding-projection pair_ between the internal and external
> representations.? While some external representations might be invalid,
> and some
> might result in approximation, deconstruct-then-construct is always an
> identity
> transformation.? Indeed, the specification of `java.lang.Record`
> requires that
> if we deconstruct a record with its accessors, and pass the resulting values
> back to the constructor, we should get a new record that is `equals` to the
> original.
> 
> The fact that constructor and deconstructor (and eventually, factory and
> static
> matcher) form an embedding-projection pair is why we are able to derive
> higher-level language features, such as [safer
> serialization](https://openjdk.org/projects/amber/design-notes/towards-better-serialization)
> and [functional transformation of immutable
> objects](https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md),
> from a matched set of constructor and deconstructor.
> 
> Of course, users are free to implement constructors without
> deconstructors, or
> constructors and deconstructors whose external representations don't
> match up,
> or even matching constructors and deconstructors that are not
> behaviorally dual.
> But providing a matched set (or several) of constructors and deconstructors
> enables reliably reversible aggregation, and allows us to mechanically
> derive
> useful higher-level features such as withers.
> 
> #### Overloading
> 
> Just as constructors can be overloaded, deconstructors can be overloaded
> for the
> same reason: multiple constructors can expose multiple external
> representations
> for aggregation, and corresponding deconstructors can recover those multiple
> external representations.? Any matching pair of
> constructor-deconstructor (and
> eventually, factory-deconstructor) is a candidate for use in higher-level
> features based on the embedding-projection nature of the
> constructor-deconstructor pair.
> 
> Just as deconstruction is dual to construction, overloading of
> deconstructors is
> dual to that of constructors: rather than restricting which sets of
> parameters
> can be overloaded against each other, we do so with the bindings
> instead.? For
> constructors of a given arity, we require that their signatures not be
> override-equivalent; for deconstructors of a given arity, we require the
> same of
> their bindings.
> 
> For a deconstructor (and declared patterns in general), we derive a _binding
> signature_ (and an erased _binding descriptor_) which treats the binding
> list as
> a parameter list.? The overload rule outlined above requires that binding
> signatures for two deconstructors of the same arity not be
> override-equivalent.
> (We will find it useful later to derive a `MethodType` for the binding
> descriptor; this is a `MethodType` whose return type is `V` and whose
> parameter
> types are the erased types of the bindings.)


I am still not able to compute how it is supposed to work.
Given that inside a record pattern you can have type patterns, the type of bindings can not be used to select matcher methods, because the type of the bindings may be a subtypes of the type of the matcher method.

class Foo {
  public Foo(String s) { ... }
  public Foo(CharSequence seq) { ... } 

  public matcher Foo(String s) { ... }
  public matcher Foo(CharSequence seq) { ... }
}

If i want to call Foo(CharSequence) with a String, i can use a cast, new Foo((CharSequence) "foo") and the compiler selects the right overload.

How i can do the same to select the right matcher method inside a deconstructor pattern ?

regards,
R?mi

From forax at univ-mlv.fr  Tue Mar  7 09:06:41 2023
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 7 Mar 2023 10:06:41 +0100 (CET)
Subject: Deconstruction patterns
In-Reply-To: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
Message-ID: <1819623456.5154827.1678180001685.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Monday, March 6, 2023 7:24:54 PM
> Subject: Deconstruction patterns

> Time to look ahead to the next installment of pattern matching:
> deconstruction patterns, which generalize record patterns.? This
> document does an end-to-end walkthrough (at a sketchy level of detail)
> through declaration, overloading, use, translation, and reflection of
> deconstruction patterns.
> 
> I would like to *not* discuss syntax at this time.? There's a lengthy
> discussion to be had about syntax, and we'll have that, but let's nail
> down model, semantics, and translation first.
> 
> As usual, I would prefer that people either (a) post a single reply
> addressing the totality of this sketch or (b) start _new threads_ if you
> want to discuss a specific aspect.? A quick "I'll just reply to this
> minor detail" seems to often derail the conversation in such a way that
> it never comes back.? If this all looks fine to you, a quick "no
> surprises here" will keep us from suspensefully waiting for feedback.
> 
> 
> # Deconstruction patterns -- translation, use, and reflection
> 
> As we are wrapping up record patterns, it's time to look ahead to the
> next major
> part of the pattern matching story -- extending the capabilities of record
> patterns to all classes that want to support destructuring. Record
> patterns are
> simply a special case of _deconstruction patterns_ or _deconstructors_,
> where we
> derive the deconstructor API, implementation, and use from the state
> description
> of the record.? For an arbitrary class, a deconstruction patterns will
> require
> an explicit member declaration, with a header identifying the names and
> types of
> the bindings and a body that extracts the bindings from the representation.
> 
> ## Deconstructors
> 
> Just as constructors are special cases of methods, deconstruction
> patterns are
> special cases of a more general notion of declared pattern, which also
> includes
> static matchers (the dual of static methods) and instance matchers (the
> dual of
> instance methods.)? Specifically, unlike the more general notion of
> matcher, a
> deconstructor must be _total_; it must always match.? This document will
> focus
> exclusively on deconstructors, and we'll come back to static and instance
> matchers in due time.? (But note that some of the design choices in the
> simple
> case of deconstructors may be constrained by the more general case.)
> 
> There are a number of choices for how we might syntactically represent a
> deconstructor (or more generally, a declared pattern.)? For purposes of
> illustration, this document picks one possible syntactic expression of
> deconstructors, but it is premature to devolve into a syntax discussion
> at this
> time.
> 
> ```
> class Point {
> ??? final double x, y;
> 
> ??? public Point(double x, double y) {
> ??????? this.x = x;
> ??????? this.y = y;
> ??? }
> 
> ??? public matcher Point(double x, double y) {
> ??????? x = this.x;
> ??????? y = this.y;
> ??? }
> }
> ```

Can we discuss about the semantics of this syntax, given that it has an effect on the translation ?

By example if the matcher method is written that way (x and y are swapped)

  public matcher Point(double x, double y) {
    y = this.y;
    x = this.x;
  }

does it means that the translation should be more something like

final synthetic Object Point$MANGLE() {
  aload_0
  getfield Point::y
  dstore 1
  aload_0
  getfield Point::x
  dstore 3
  LDC #101
  dload3
  dload1
  invokevirtual MethodHandle::invoke(II)V
  areturn
}

so if there are side effects in the expressions that are initializing the local x and the local y, there are performed in the right order ?

regards,
R?mi

From forax at univ-mlv.fr  Tue Mar  7 09:09:25 2023
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 7 Mar 2023 10:09:25 +0100 (CET)
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
Message-ID: <667851490.5158494.1678180165559.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Monday, March 6, 2023 7:24:54 PM
> Subject: Deconstruction patterns

> Time to look ahead to the next installment of pattern matching:
> deconstruction patterns, which generalize record patterns.? This
> document does an end-to-end walkthrough (at a sketchy level of detail)
> through declaration, overloading, use, translation, and reflection of
> deconstruction patterns.
> 
> I would like to *not* discuss syntax at this time.? There's a lengthy
> discussion to be had about syntax, and we'll have that, but let's nail
> down model, semantics, and translation first.
> 
> As usual, I would prefer that people either (a) post a single reply
> addressing the totality of this sketch or (b) start _new threads_ if you
> want to discuss a specific aspect.? A quick "I'll just reply to this
> minor detail" seems to often derail the conversation in such a way that
> it never comes back.? If this all looks fine to you, a quick "no
> surprises here" will keep us from suspensefully waiting for feedback.
> 
> 
> # Deconstruction patterns -- translation, use, and reflection
> 
> As we are wrapping up record patterns, it's time to look ahead to the
> next major
> part of the pattern matching story -- extending the capabilities of record
> patterns to all classes that want to support destructuring. Record
> patterns are
> simply a special case of _deconstruction patterns_ or _deconstructors_,
> where we
> derive the deconstructor API, implementation, and use from the state
> description
> of the record.? For an arbitrary class, a deconstruction patterns will
> require
> an explicit member declaration, with a header identifying the names and
> types of
> the bindings and a body that extracts the bindings from the representation.
> 
> ## Deconstructors
> 
> Just as constructors are special cases of methods, deconstruction
> patterns are
> special cases of a more general notion of declared pattern, which also
> includes
> static matchers (the dual of static methods) and instance matchers (the
> dual of
> instance methods.)? Specifically, unlike the more general notion of
> matcher, a
> deconstructor must be _total_; it must always match.? This document will
> focus
> exclusively on deconstructors, and we'll come back to static and instance
> matchers in due time.? (But note that some of the design choices in the
> simple
> case of deconstructors may be constrained by the more general case.)
> 
> There are a number of choices for how we might syntactically represent a
> deconstructor (or more generally, a declared pattern.)? For purposes of
> illustration, this document picks one possible syntactic expression of
> deconstructors, but it is premature to devolve into a syntax discussion
> at this
> time.
> 
> ```
> class Point {
> ??? final double x, y;
> 
> ??? public Point(double x, double y) {
> ??????? this.x = x;
> ??????? this.y = y;
> ??? }
> 
> ??? public matcher Point(double x, double y) {
> ??????? x = this.x;
> ??????? y = this.y;
> ??? }
> }
> ```
> 
> This example illustrates two aspects of the duality between constructors and
> their corresponding deconstructors.? Their APIs are duals: a constructor
> takes N
> parameters containing the desired description of the object state and
> produces a
> constructed object; a deconstructor starts from the constructed object
> and has N
> bindings (outputs) that receive the desired state components. Similarly,
> their
> implementations are duals: the body of the constructor initializes the
> object
> representation from the description, and the body of the deconstructor
> extracts
> the description from the representation.? A deconstructor is best
> understood as
> a _co-constructor_.
> 
> The `Point` example above is special in two ways.? First, the internal
> representation of a `Point`, and the API of the constructor and
> deconstructor,
> are the same: `(double x, double y)`.? We can call the API implied by the
> constructor and deconstructor the _external representation_, and for
> `Point`,
> both the internal and external representations are the same. (This is one of
> the requirements for being a candidate to be a record.)? And second, the
> constructor is _total_; it does not reject any combinations of arguments.
> 
> Here's another version of `Point` which does not have these special
> aspects; it
> uses the same internal representation as before, but chooses a pair of
> strings
> as the external representation:
> 
> ```
> class Point2 {
> ??? final double x, y;
> 
> ??? public Point2(String x, String y) {
> ??????? this.x = Double.parseDouble(x);
> ??????? this.y = Double.parseDouble(y);
> ??? }
> 
> ??? public matcher Point2(String x, String y) {
> ??????? x = Double.toString(this.x);
> ??????? y = Double.toSTring(this.y);
> ??? }
> }
> ```
> 
> The method `Double::parseDouble` will throw `NumberFormatException` if its
> argument does not describe a suitable value, so unlike the `Point`
> constructor,
> the `Point2` constructor is partial: it will reject `new Double("foo",
> "bar")`.
> And the internal representation is no longer the same as the external
> representation.? Less obviously, there are valid string values that we can
> provide to the constructor, but which cannot be represented exactly as
> `double`,
> and which will be approximated; the string value
> `"3.22222222222222222222222222222222222222"` will be approximated with the
> double value `3.2222222222222223`.
> 
> This example highlights more clearly how the constructor and
> deconstructor form
> an _embedding-projection pair_ between the internal and external
> representations.? While some external representations might be invalid,
> and some
> might result in approximation, deconstruct-then-construct is always an
> identity
> transformation.? Indeed, the specification of `java.lang.Record`
> requires that
> if we deconstruct a record with its accessors, and pass the resulting values
> back to the constructor, we should get a new record that is `equals` to the
> original.
> 
> The fact that constructor and deconstructor (and eventually, factory and
> static
> matcher) form an embedding-projection pair is why we are able to derive
> higher-level language features, such as [safer
> serialization](https://openjdk.org/projects/amber/design-notes/towards-better-serialization)
> and [functional transformation of immutable
> objects](https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md),
> from a matched set of constructor and deconstructor.
> 
> Of course, users are free to implement constructors without
> deconstructors, or
> constructors and deconstructors whose external representations don't
> match up,
> or even matching constructors and deconstructors that are not
> behaviorally dual.
> But providing a matched set (or several) of constructors and deconstructors
> enables reliably reversible aggregation, and allows us to mechanically
> derive
> useful higher-level features such as withers.
> 
> #### Overloading
> 
> Just as constructors can be overloaded, deconstructors can be overloaded
> for the
> same reason: multiple constructors can expose multiple external
> representations
> for aggregation, and corresponding deconstructors can recover those multiple
> external representations.? Any matching pair of
> constructor-deconstructor (and
> eventually, factory-deconstructor) is a candidate for use in higher-level
> features based on the embedding-projection nature of the
> constructor-deconstructor pair.
> 
> Just as deconstruction is dual to construction, overloading of
> deconstructors is
> dual to that of constructors: rather than restricting which sets of
> parameters
> can be overloaded against each other, we do so with the bindings
> instead.? For
> constructors of a given arity, we require that their signatures not be
> override-equivalent; for deconstructors of a given arity, we require the
> same of
> their bindings.
> 
> For a deconstructor (and declared patterns in general), we derive a _binding
> signature_ (and an erased _binding descriptor_) which treats the binding
> list as
> a parameter list.? The overload rule outlined above requires that binding
> signatures for two deconstructors of the same arity not be
> override-equivalent.
> (We will find it useful later to derive a `MethodType` for the binding
> descriptor; this is a `MethodType` whose return type is `V` and whose
> parameter
> types are the erased types of the bindings.)
> 
> #### Digression: embedding-projection pairs
> 
> Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B
> -> A`,
> forms an _embedding-projection pair_ if `p . e` (embed then project) is an
> identity function, and `e . p` (project then embed) _approximates_ the input
> according to a domain-specific approximation metric (which is a complete
> partial
> ordering on `B`.)
> 
> When applied to constructor-deconstructor pairs, this says that
> deconstructing
> an object and then reconstructing it with the resulting bindings should
> result
> in an equivalent object, and constructing an object from an external
> representation and then deconstructing it back into that external
> representation
> should result in an approximation of the original external
> representation.? (A
> complete partial ordering models constructor failure as the non-terminating
> bottom value, which is considered an infinitely bad approximation to
> everything.)
> 
> Embedding-projection pairs have a number of desirable properties, such
> as the
> composition of two e-p pairs is an e-p pair; this property is at the
> heart of
> using constructor-deconstructor pairs for improved serialization and
> functional
> transformation.
> 
> ## Invoking deconstructors
> 
> We've already seen how to "invoke" deconstructors: through pattern matching.
> What we've been calling "record patterns" are merely deconstruction patterns
> derived mechanically from the state description, just as we do with
> constructors
> and accessors; there is little difference between record patterns and
> deconstruction patterns other than the ability to declare them explicitly.
> (There is an accidental difference in the translation, in that we currently
> implement record patterns by appealing to individual accessors rather than a
> single deconstructor, but this may eventually converge as well.)
> 
> The use-site syntax of deconstruction bears a deliberate similarity to
> that of
> construction; `new Point(x, y)` is deconstructed by `case Point(var x,
> var y)`.
> 
> #### Overload selection
> 
> In the presence of overloaded deconstructors, we need to figure out which
> deconstructor a deconstruction pattern `C(P*)` is referring to. The
> details are
> similar to overload selection for methods, except that we operate on the
> bindings rather than the parameters.? We first search for _applicable
> matchers_,
> using increasingly loose criteria (first excluding boxing, unboxing, and
> varargs; then including boxing and unboxing but not varargs; and
> finally, all
> candidates) and then selecting the most applicable.
> 
> It is tempting to try and bypass the three-phase selection process and use a
> simpler notion of applicability (perhaps noting that we got this process for
> compatibility with existing overload selection decisions when autoboxing and
> varargs were added, and that there are few deconstructor invocations to be
> compatible with yet.)? But because existing overloaded constructors use this
> mechanism, and there is significant value in pairing constructors and
> deconstructors, attempting to invent a simpler-but-different overload
> selection
> mechanism for deconstructors would inevitably undermine the duality between
> matching constructor-deconstructor pairs. So compatibility (this time, with
> existing overloaded constructors) once again forces our hand.
> 
> The specification for overload selection is complicated significantly by
> poly
> expressions (e.g., lambdas); fortunately, there are no "poly patterns",
> and so,
> while the structure of JLS 15.12.2 is retained for overload selection of
> deconstruction patterns, much of the detail is left behind.
> 
> ## Translation
> 
> We translate patterns into synthetic methods with a `Matcher` attribute;
> this
> method implements the matcher behavior.? The translation scheme derives
> from a
> number of requirements, only some of which are in play for deconstructors.
> 
> The matcher method for a deconstructor is a final instance method that
> takes no
> parameters and returns `Object`, perhaps with a special name (just as
> constructors are called `<init>`.)
> 
> #### Carriers
> 
> Because the matcher methods implements the matcher behavior, but a
> matcher may
> "return" multiple bindings (or failure), we must encode the bindings in some
> way.? For this, we use a _carrier object_.? The choice of carrier is
> largely a
> footprint/specificity tradeoff.? One could imagine a carrier class per
> matcher,
> or a carrier class per matcher descriptor, or using `Object[]` as a
> carrier for
> everything, or caching some number of common shapes (e.g, three ints and two
> refs).? This sort of tuning should be separate from the protocol encoded
> in the
> bytecode of the pattern method and its clients.
> 
> We use a small _carrier runtime_ to decouple pattern translation from
> carrier
> selection.? (This same carrier runtime is used by string templates as well.)
> This allows tradeoffs in runtime characteristics (e.g., carrier per
> matcher vs
> sharing carriers across matchers, dropping carrier identity with value types
> later, etc) without affecting the translation. The carrier API consists
> of condy
> bootstraps like:
> 
> ```
> static MethodHandle carrierFactory(MethodType matcherDescriptor) { ... }
> static MethodHandle carrierAccessor(MethodType matcherDescriptor, int
> bindingNo) { ... }
> ```
> 
> The `matcherDescriptor` is a `MethodType` describing the binding types.? The
> `carrierFactory` method returns a method handle which takes the bindings and
> produces a carrier object; the `carrierAccessor` method returns method
> handles
> that take the carrier object and return the corresponding binding.? To
> indicate
> success, the matcher method invokes the carrier factory method handle and
> returns the result; to indicate failure (deconstructors cannot fail, but
> other
> matchers can) the matcher method returns null.
> 
> We would translate the XY deconstructor from `Point` as follows
> (pseudo-code):
> 
> ```
> #100: MethodType[(II)V]
> #101: Condy[bsm=Carriers::carrierFactory, args=[#100]]
> 
> final synthetic Object Point$MANGLE() {
> ??? aload_0
> ??? getfield Point::x
> ??? aload_0
> ??? getfield Point::y
> ??? LDC #101
> ??? invokevirtual MethodHandle::invoke(II)V
> ??? areturn
> }
> ```
> 
> Constant `#100` contains a `MethodType` holding the binding descriptor;
> constant
> `#101` holds a method handle whose parameters are the parameter types of the
> binding descriptor and returns `Object`.
> 
> At the use site, matching a deconstruction pattern is performed by
> invoking the
> matcher method on the appropriate target object, and then extracting the
> components with the carrier accessor method handles if the match is
> successful.
> (Deconstructors are total, so are always successful, but for other patterns,
> null is returned from the matcher method on failure to match.)
> 
> #### Method names
> 
> The name of the matcher method is mangled to support overloading. The JVM
> permits overloading on parameter types, but not return types (and overloaded
> matchers are effectively overloaded on return types.)? We take the
> approach of
> encoding the erasure of the matcher descriptor in the name of the
> pattern.? This
> has several desirable properties: it is stable (the name is derived
> solely from
> stable aspects of the declaration), for matchers with override-equivalent
> signatures (deconstructors can't be overridden, but other patterns can be),
> these map to true overrides in the translation, and valid overloads of
> matchers
> will always have distinct names.
> 
> We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
> descriptor as the mangled disambiguator, which is exactly as stable as
> any other
> method descriptor derived from source declarations.

Why deconstructor can not be overriden ?

Does it means that we can not have a deconstructor on an interface ?

regards,
R?mi

From forax at univ-mlv.fr  Tue Mar  7 09:21:15 2023
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 7 Mar 2023 10:21:15 +0100 (CET)
Subject: Matcher method name mangling Was: Deconstruction patterns
In-Reply-To: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
Message-ID: <1219316046.5174614.1678180875926.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Monday, March 6, 2023 7:24:54 PM
> Subject: Deconstruction patterns

> Time to look ahead to the next installment of pattern matching:
> deconstruction patterns, which generalize record patterns.? This
> document does an end-to-end walkthrough (at a sketchy level of detail)
> through declaration, overloading, use, translation, and reflection of
> deconstruction patterns.
> 
> I would like to *not* discuss syntax at this time.? There's a lengthy
> discussion to be had about syntax, and we'll have that, but let's nail
> down model, semantics, and translation first.
> 
> As usual, I would prefer that people either (a) post a single reply
> addressing the totality of this sketch or (b) start _new threads_ if you
> want to discuss a specific aspect.? A quick "I'll just reply to this
> minor detail" seems to often derail the conversation in such a way that
> it never comes back.? If this all looks fine to you, a quick "no
> surprises here" will keep us from suspensefully waiting for feedback.
> 
> 
> # Deconstruction patterns -- translation, use, and reflection
> 
> As we are wrapping up record patterns, it's time to look ahead to the
> next major
> part of the pattern matching story -- extending the capabilities of record
> patterns to all classes that want to support destructuring. Record
> patterns are
> simply a special case of _deconstruction patterns_ or _deconstructors_,
> where we
> derive the deconstructor API, implementation, and use from the state
> description
> of the record.? For an arbitrary class, a deconstruction patterns will
> require
> an explicit member declaration, with a header identifying the names and
> types of
> the bindings and a body that extracts the bindings from the representation.
> 
> ## Deconstructors
> 
> Just as constructors are special cases of methods, deconstruction
> patterns are
> special cases of a more general notion of declared pattern, which also
> includes
> static matchers (the dual of static methods) and instance matchers (the
> dual of
> instance methods.)? Specifically, unlike the more general notion of
> matcher, a
> deconstructor must be _total_; it must always match.? This document will
> focus
> exclusively on deconstructors, and we'll come back to static and instance
> matchers in due time.? (But note that some of the design choices in the
> simple
> case of deconstructors may be constrained by the more general case.)
> 
> There are a number of choices for how we might syntactically represent a
> deconstructor (or more generally, a declared pattern.)? For purposes of
> illustration, this document picks one possible syntactic expression of
> deconstructors, but it is premature to devolve into a syntax discussion
> at this
> time.
> 
> ```
> class Point {
> ??? final double x, y;
> 
> ??? public Point(double x, double y) {
> ??????? this.x = x;
> ??????? this.y = y;
> ??? }
> 
> ??? public matcher Point(double x, double y) {
> ??????? x = this.x;
> ??????? y = this.y;
> ??? }
> }
> ```
> 
> This example illustrates two aspects of the duality between constructors and
> their corresponding deconstructors.? Their APIs are duals: a constructor
> takes N
> parameters containing the desired description of the object state and
> produces a
> constructed object; a deconstructor starts from the constructed object
> and has N
> bindings (outputs) that receive the desired state components. Similarly,
> their
> implementations are duals: the body of the constructor initializes the
> object
> representation from the description, and the body of the deconstructor
> extracts
> the description from the representation.? A deconstructor is best
> understood as
> a _co-constructor_.
> 
> The `Point` example above is special in two ways.? First, the internal
> representation of a `Point`, and the API of the constructor and
> deconstructor,
> are the same: `(double x, double y)`.? We can call the API implied by the
> constructor and deconstructor the _external representation_, and for
> `Point`,
> both the internal and external representations are the same. (This is one of
> the requirements for being a candidate to be a record.)? And second, the
> constructor is _total_; it does not reject any combinations of arguments.
> 
> Here's another version of `Point` which does not have these special
> aspects; it
> uses the same internal representation as before, but chooses a pair of
> strings
> as the external representation:
> 
> ```
> class Point2 {
> ??? final double x, y;
> 
> ??? public Point2(String x, String y) {
> ??????? this.x = Double.parseDouble(x);
> ??????? this.y = Double.parseDouble(y);
> ??? }
> 
> ??? public matcher Point2(String x, String y) {
> ??????? x = Double.toString(this.x);
> ??????? y = Double.toSTring(this.y);
> ??? }
> }
> ```
> 
> The method `Double::parseDouble` will throw `NumberFormatException` if its
> argument does not describe a suitable value, so unlike the `Point`
> constructor,
> the `Point2` constructor is partial: it will reject `new Double("foo",
> "bar")`.
> And the internal representation is no longer the same as the external
> representation.? Less obviously, there are valid string values that we can
> provide to the constructor, but which cannot be represented exactly as
> `double`,
> and which will be approximated; the string value
> `"3.22222222222222222222222222222222222222"` will be approximated with the
> double value `3.2222222222222223`.
> 
> This example highlights more clearly how the constructor and
> deconstructor form
> an _embedding-projection pair_ between the internal and external
> representations.? While some external representations might be invalid,
> and some
> might result in approximation, deconstruct-then-construct is always an
> identity
> transformation.? Indeed, the specification of `java.lang.Record`
> requires that
> if we deconstruct a record with its accessors, and pass the resulting values
> back to the constructor, we should get a new record that is `equals` to the
> original.
> 
> The fact that constructor and deconstructor (and eventually, factory and
> static
> matcher) form an embedding-projection pair is why we are able to derive
> higher-level language features, such as [safer
> serialization](https://openjdk.org/projects/amber/design-notes/towards-better-serialization)
> and [functional transformation of immutable
> objects](https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md),
> from a matched set of constructor and deconstructor.
> 
> Of course, users are free to implement constructors without
> deconstructors, or
> constructors and deconstructors whose external representations don't
> match up,
> or even matching constructors and deconstructors that are not
> behaviorally dual.
> But providing a matched set (or several) of constructors and deconstructors
> enables reliably reversible aggregation, and allows us to mechanically
> derive
> useful higher-level features such as withers.
> 
> #### Overloading
> 
> Just as constructors can be overloaded, deconstructors can be overloaded
> for the
> same reason: multiple constructors can expose multiple external
> representations
> for aggregation, and corresponding deconstructors can recover those multiple
> external representations.? Any matching pair of
> constructor-deconstructor (and
> eventually, factory-deconstructor) is a candidate for use in higher-level
> features based on the embedding-projection nature of the
> constructor-deconstructor pair.
> 
> Just as deconstruction is dual to construction, overloading of
> deconstructors is
> dual to that of constructors: rather than restricting which sets of
> parameters
> can be overloaded against each other, we do so with the bindings
> instead.? For
> constructors of a given arity, we require that their signatures not be
> override-equivalent; for deconstructors of a given arity, we require the
> same of
> their bindings.
> 
> For a deconstructor (and declared patterns in general), we derive a _binding
> signature_ (and an erased _binding descriptor_) which treats the binding
> list as
> a parameter list.? The overload rule outlined above requires that binding
> signatures for two deconstructors of the same arity not be
> override-equivalent.
> (We will find it useful later to derive a `MethodType` for the binding
> descriptor; this is a `MethodType` whose return type is `V` and whose
> parameter
> types are the erased types of the bindings.)
> 
> #### Digression: embedding-projection pairs
> 
> Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B
> -> A`,
> forms an _embedding-projection pair_ if `p . e` (embed then project) is an
> identity function, and `e . p` (project then embed) _approximates_ the input
> according to a domain-specific approximation metric (which is a complete
> partial
> ordering on `B`.)
> 
> When applied to constructor-deconstructor pairs, this says that
> deconstructing
> an object and then reconstructing it with the resulting bindings should
> result
> in an equivalent object, and constructing an object from an external
> representation and then deconstructing it back into that external
> representation
> should result in an approximation of the original external
> representation.? (A
> complete partial ordering models constructor failure as the non-terminating
> bottom value, which is considered an infinitely bad approximation to
> everything.)
> 
> Embedding-projection pairs have a number of desirable properties, such
> as the
> composition of two e-p pairs is an e-p pair; this property is at the
> heart of
> using constructor-deconstructor pairs for improved serialization and
> functional
> transformation.
> 
> ## Invoking deconstructors
> 
> We've already seen how to "invoke" deconstructors: through pattern matching.
> What we've been calling "record patterns" are merely deconstruction patterns
> derived mechanically from the state description, just as we do with
> constructors
> and accessors; there is little difference between record patterns and
> deconstruction patterns other than the ability to declare them explicitly.
> (There is an accidental difference in the translation, in that we currently
> implement record patterns by appealing to individual accessors rather than a
> single deconstructor, but this may eventually converge as well.)
> 
> The use-site syntax of deconstruction bears a deliberate similarity to
> that of
> construction; `new Point(x, y)` is deconstructed by `case Point(var x,
> var y)`.
> 
> #### Overload selection
> 
> In the presence of overloaded deconstructors, we need to figure out which
> deconstructor a deconstruction pattern `C(P*)` is referring to. The
> details are
> similar to overload selection for methods, except that we operate on the
> bindings rather than the parameters.? We first search for _applicable
> matchers_,
> using increasingly loose criteria (first excluding boxing, unboxing, and
> varargs; then including boxing and unboxing but not varargs; and
> finally, all
> candidates) and then selecting the most applicable.
> 
> It is tempting to try and bypass the three-phase selection process and use a
> simpler notion of applicability (perhaps noting that we got this process for
> compatibility with existing overload selection decisions when autoboxing and
> varargs were added, and that there are few deconstructor invocations to be
> compatible with yet.)? But because existing overloaded constructors use this
> mechanism, and there is significant value in pairing constructors and
> deconstructors, attempting to invent a simpler-but-different overload
> selection
> mechanism for deconstructors would inevitably undermine the duality between
> matching constructor-deconstructor pairs. So compatibility (this time, with
> existing overloaded constructors) once again forces our hand.
> 
> The specification for overload selection is complicated significantly by
> poly
> expressions (e.g., lambdas); fortunately, there are no "poly patterns",
> and so,
> while the structure of JLS 15.12.2 is retained for overload selection of
> deconstruction patterns, much of the detail is left behind.
> 
> ## Translation
> 
> We translate patterns into synthetic methods with a `Matcher` attribute;
> this
> method implements the matcher behavior.? The translation scheme derives
> from a
> number of requirements, only some of which are in play for deconstructors.
> 
> The matcher method for a deconstructor is a final instance method that
> takes no
> parameters and returns `Object`, perhaps with a special name (just as
> constructors are called `<init>`.)
> 
> #### Carriers
> 
> Because the matcher methods implements the matcher behavior, but a
> matcher may
> "return" multiple bindings (or failure), we must encode the bindings in some
> way.? For this, we use a _carrier object_.? The choice of carrier is
> largely a
> footprint/specificity tradeoff.? One could imagine a carrier class per
> matcher,
> or a carrier class per matcher descriptor, or using `Object[]` as a
> carrier for
> everything, or caching some number of common shapes (e.g, three ints and two
> refs).? This sort of tuning should be separate from the protocol encoded
> in the
> bytecode of the pattern method and its clients.
> 
> We use a small _carrier runtime_ to decouple pattern translation from
> carrier
> selection.? (This same carrier runtime is used by string templates as well.)
> This allows tradeoffs in runtime characteristics (e.g., carrier per
> matcher vs
> sharing carriers across matchers, dropping carrier identity with value types
> later, etc) without affecting the translation. The carrier API consists
> of condy
> bootstraps like:
> 
> ```
> static MethodHandle carrierFactory(MethodType matcherDescriptor) { ... }
> static MethodHandle carrierAccessor(MethodType matcherDescriptor, int
> bindingNo) { ... }
> ```
> 
> The `matcherDescriptor` is a `MethodType` describing the binding types.? The
> `carrierFactory` method returns a method handle which takes the bindings and
> produces a carrier object; the `carrierAccessor` method returns method
> handles
> that take the carrier object and return the corresponding binding.? To
> indicate
> success, the matcher method invokes the carrier factory method handle and
> returns the result; to indicate failure (deconstructors cannot fail, but
> other
> matchers can) the matcher method returns null.
> 
> We would translate the XY deconstructor from `Point` as follows
> (pseudo-code):
> 
> ```
> #100: MethodType[(II)V]
> #101: Condy[bsm=Carriers::carrierFactory, args=[#100]]
> 
> final synthetic Object Point$MANGLE() {
> ??? aload_0
> ??? getfield Point::x
> ??? aload_0
> ??? getfield Point::y
> ??? LDC #101
> ??? invokevirtual MethodHandle::invoke(II)V
> ??? areturn
> }
> ```
> 
> Constant `#100` contains a `MethodType` holding the binding descriptor;
> constant
> `#101` holds a method handle whose parameters are the parameter types of the
> binding descriptor and returns `Object`.
> 
> At the use site, matching a deconstruction pattern is performed by
> invoking the
> matcher method on the appropriate target object, and then extracting the
> components with the carrier accessor method handles if the match is
> successful.
> (Deconstructors are total, so are always successful, but for other patterns,
> null is returned from the matcher method on failure to match.)
> 
> #### Method names
> 
> The name of the matcher method is mangled to support overloading. The JVM
> permits overloading on parameter types, but not return types (and overloaded
> matchers are effectively overloaded on return types.)? We take the
> approach of
> encoding the erasure of the matcher descriptor in the name of the
> pattern.? This
> has several desirable properties: it is stable (the name is derived
> solely from
> stable aspects of the declaration), for matchers with override-equivalent
> signatures (deconstructors can't be overridden, but other patterns can be),
> these map to true overrides in the translation, and valid overloads of
> matchers
> will always have distinct names.
> 
> We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
> descriptor as the mangled disambiguator, which is exactly as stable as
> any other
> method descriptor derived from source declarations.

I do not think you need mangling, you need different name but matcher$0, matcher$1, etc or something similar with a counter should be enough.

The Matcher attribute contains enough information to do the linking both at compile time and at runtime.

The idea is that in the bytecode, you can derive the MethodType from the invokeVirtual (or invokedynamic) and then ask the VM the method that has a Matcher attribute that matches that MethodType.
So the exact name of the matcher method should not be present in the bytecode of a switch that uses a deconstructor, only the corresponding MethodType should be present.

Mangling method names appear in stack trace, if we can have "light mangling", i.e. a counter likes with lambdas, instead of a C++ like mangling, i think it's a win.

R?mi

From forax at univ-mlv.fr  Tue Mar  7 09:21:12 2023
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 7 Mar 2023 10:21:12 +0100 (CET)
Subject: Use of condy + MH.invokevirtual instead of indy Was Deconstruction
 patterns
In-Reply-To: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
Message-ID: <1409285726.5174584.1678180872430.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Monday, March 6, 2023 7:24:54 PM
> Subject: Deconstruction patterns

> Time to look ahead to the next installment of pattern matching:
> deconstruction patterns, which generalize record patterns.? This
> document does an end-to-end walkthrough (at a sketchy level of detail)
> through declaration, overloading, use, translation, and reflection of
> deconstruction patterns.
> 
> I would like to *not* discuss syntax at this time.? There's a lengthy
> discussion to be had about syntax, and we'll have that, but let's nail
> down model, semantics, and translation first.
> 
> As usual, I would prefer that people either (a) post a single reply
> addressing the totality of this sketch or (b) start _new threads_ if you
> want to discuss a specific aspect.? A quick "I'll just reply to this
> minor detail" seems to often derail the conversation in such a way that
> it never comes back.? If this all looks fine to you, a quick "no
> surprises here" will keep us from suspensefully waiting for feedback.
> 
> 
> # Deconstruction patterns -- translation, use, and reflection
> 
> As we are wrapping up record patterns, it's time to look ahead to the
> next major
> part of the pattern matching story -- extending the capabilities of record
> patterns to all classes that want to support destructuring. Record
> patterns are
> simply a special case of _deconstruction patterns_ or _deconstructors_,
> where we
> derive the deconstructor API, implementation, and use from the state
> description
> of the record.? For an arbitrary class, a deconstruction patterns will
> require
> an explicit member declaration, with a header identifying the names and
> types of
> the bindings and a body that extracts the bindings from the representation.
> 
> ## Deconstructors
> 
> Just as constructors are special cases of methods, deconstruction
> patterns are
> special cases of a more general notion of declared pattern, which also
> includes
> static matchers (the dual of static methods) and instance matchers (the
> dual of
> instance methods.)? Specifically, unlike the more general notion of
> matcher, a
> deconstructor must be _total_; it must always match.? This document will
> focus
> exclusively on deconstructors, and we'll come back to static and instance
> matchers in due time.? (But note that some of the design choices in the
> simple
> case of deconstructors may be constrained by the more general case.)
> 
> There are a number of choices for how we might syntactically represent a
> deconstructor (or more generally, a declared pattern.)? For purposes of
> illustration, this document picks one possible syntactic expression of
> deconstructors, but it is premature to devolve into a syntax discussion
> at this
> time.
> 
> ```
> class Point {
> ??? final double x, y;
> 
> ??? public Point(double x, double y) {
> ??????? this.x = x;
> ??????? this.y = y;
> ??? }
> 
> ??? public matcher Point(double x, double y) {
> ??????? x = this.x;
> ??????? y = this.y;
> ??? }
> }
> ```
> 
> This example illustrates two aspects of the duality between constructors and
> their corresponding deconstructors.? Their APIs are duals: a constructor
> takes N
> parameters containing the desired description of the object state and
> produces a
> constructed object; a deconstructor starts from the constructed object
> and has N
> bindings (outputs) that receive the desired state components. Similarly,
> their
> implementations are duals: the body of the constructor initializes the
> object
> representation from the description, and the body of the deconstructor
> extracts
> the description from the representation.? A deconstructor is best
> understood as
> a _co-constructor_.
> 
> The `Point` example above is special in two ways.? First, the internal
> representation of a `Point`, and the API of the constructor and
> deconstructor,
> are the same: `(double x, double y)`.? We can call the API implied by the
> constructor and deconstructor the _external representation_, and for
> `Point`,
> both the internal and external representations are the same. (This is one of
> the requirements for being a candidate to be a record.)? And second, the
> constructor is _total_; it does not reject any combinations of arguments.
> 
> Here's another version of `Point` which does not have these special
> aspects; it
> uses the same internal representation as before, but chooses a pair of
> strings
> as the external representation:
> 
> ```
> class Point2 {
> ??? final double x, y;
> 
> ??? public Point2(String x, String y) {
> ??????? this.x = Double.parseDouble(x);
> ??????? this.y = Double.parseDouble(y);
> ??? }
> 
> ??? public matcher Point2(String x, String y) {
> ??????? x = Double.toString(this.x);
> ??????? y = Double.toSTring(this.y);
> ??? }
> }
> ```
> 
> The method `Double::parseDouble` will throw `NumberFormatException` if its
> argument does not describe a suitable value, so unlike the `Point`
> constructor,
> the `Point2` constructor is partial: it will reject `new Double("foo",
> "bar")`.
> And the internal representation is no longer the same as the external
> representation.? Less obviously, there are valid string values that we can
> provide to the constructor, but which cannot be represented exactly as
> `double`,
> and which will be approximated; the string value
> `"3.22222222222222222222222222222222222222"` will be approximated with the
> double value `3.2222222222222223`.
> 
> This example highlights more clearly how the constructor and
> deconstructor form
> an _embedding-projection pair_ between the internal and external
> representations.? While some external representations might be invalid,
> and some
> might result in approximation, deconstruct-then-construct is always an
> identity
> transformation.? Indeed, the specification of `java.lang.Record`
> requires that
> if we deconstruct a record with its accessors, and pass the resulting values
> back to the constructor, we should get a new record that is `equals` to the
> original.
> 
> The fact that constructor and deconstructor (and eventually, factory and
> static
> matcher) form an embedding-projection pair is why we are able to derive
> higher-level language features, such as [safer
> serialization](https://openjdk.org/projects/amber/design-notes/towards-better-serialization)
> and [functional transformation of immutable
> objects](https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md),
> from a matched set of constructor and deconstructor.
> 
> Of course, users are free to implement constructors without
> deconstructors, or
> constructors and deconstructors whose external representations don't
> match up,
> or even matching constructors and deconstructors that are not
> behaviorally dual.
> But providing a matched set (or several) of constructors and deconstructors
> enables reliably reversible aggregation, and allows us to mechanically
> derive
> useful higher-level features such as withers.
> 
> #### Overloading
> 
> Just as constructors can be overloaded, deconstructors can be overloaded
> for the
> same reason: multiple constructors can expose multiple external
> representations
> for aggregation, and corresponding deconstructors can recover those multiple
> external representations.? Any matching pair of
> constructor-deconstructor (and
> eventually, factory-deconstructor) is a candidate for use in higher-level
> features based on the embedding-projection nature of the
> constructor-deconstructor pair.
> 
> Just as deconstruction is dual to construction, overloading of
> deconstructors is
> dual to that of constructors: rather than restricting which sets of
> parameters
> can be overloaded against each other, we do so with the bindings
> instead.? For
> constructors of a given arity, we require that their signatures not be
> override-equivalent; for deconstructors of a given arity, we require the
> same of
> their bindings.
> 
> For a deconstructor (and declared patterns in general), we derive a _binding
> signature_ (and an erased _binding descriptor_) which treats the binding
> list as
> a parameter list.? The overload rule outlined above requires that binding
> signatures for two deconstructors of the same arity not be
> override-equivalent.
> (We will find it useful later to derive a `MethodType` for the binding
> descriptor; this is a `MethodType` whose return type is `V` and whose
> parameter
> types are the erased types of the bindings.)
> 
> #### Digression: embedding-projection pairs
> 
> Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B
> -> A`,
> forms an _embedding-projection pair_ if `p . e` (embed then project) is an
> identity function, and `e . p` (project then embed) _approximates_ the input
> according to a domain-specific approximation metric (which is a complete
> partial
> ordering on `B`.)
> 
> When applied to constructor-deconstructor pairs, this says that
> deconstructing
> an object and then reconstructing it with the resulting bindings should
> result
> in an equivalent object, and constructing an object from an external
> representation and then deconstructing it back into that external
> representation
> should result in an approximation of the original external
> representation.? (A
> complete partial ordering models constructor failure as the non-terminating
> bottom value, which is considered an infinitely bad approximation to
> everything.)
> 
> Embedding-projection pairs have a number of desirable properties, such
> as the
> composition of two e-p pairs is an e-p pair; this property is at the
> heart of
> using constructor-deconstructor pairs for improved serialization and
> functional
> transformation.
> 
> ## Invoking deconstructors
> 
> We've already seen how to "invoke" deconstructors: through pattern matching.
> What we've been calling "record patterns" are merely deconstruction patterns
> derived mechanically from the state description, just as we do with
> constructors
> and accessors; there is little difference between record patterns and
> deconstruction patterns other than the ability to declare them explicitly.
> (There is an accidental difference in the translation, in that we currently
> implement record patterns by appealing to individual accessors rather than a
> single deconstructor, but this may eventually converge as well.)
> 
> The use-site syntax of deconstruction bears a deliberate similarity to
> that of
> construction; `new Point(x, y)` is deconstructed by `case Point(var x,
> var y)`.
> 
> #### Overload selection
> 
> In the presence of overloaded deconstructors, we need to figure out which
> deconstructor a deconstruction pattern `C(P*)` is referring to. The
> details are
> similar to overload selection for methods, except that we operate on the
> bindings rather than the parameters.? We first search for _applicable
> matchers_,
> using increasingly loose criteria (first excluding boxing, unboxing, and
> varargs; then including boxing and unboxing but not varargs; and
> finally, all
> candidates) and then selecting the most applicable.
> 
> It is tempting to try and bypass the three-phase selection process and use a
> simpler notion of applicability (perhaps noting that we got this process for
> compatibility with existing overload selection decisions when autoboxing and
> varargs were added, and that there are few deconstructor invocations to be
> compatible with yet.)? But because existing overloaded constructors use this
> mechanism, and there is significant value in pairing constructors and
> deconstructors, attempting to invent a simpler-but-different overload
> selection
> mechanism for deconstructors would inevitably undermine the duality between
> matching constructor-deconstructor pairs. So compatibility (this time, with
> existing overloaded constructors) once again forces our hand.
> 
> The specification for overload selection is complicated significantly by
> poly
> expressions (e.g., lambdas); fortunately, there are no "poly patterns",
> and so,
> while the structure of JLS 15.12.2 is retained for overload selection of
> deconstruction patterns, much of the detail is left behind.
> 
> ## Translation
> 
> We translate patterns into synthetic methods with a `Matcher` attribute;
> this
> method implements the matcher behavior.? The translation scheme derives
> from a
> number of requirements, only some of which are in play for deconstructors.
> 
> The matcher method for a deconstructor is a final instance method that
> takes no
> parameters and returns `Object`, perhaps with a special name (just as
> constructors are called `<init>`.)
> 
> #### Carriers
> 
> Because the matcher methods implements the matcher behavior, but a
> matcher may
> "return" multiple bindings (or failure), we must encode the bindings in some
> way.? For this, we use a _carrier object_.? The choice of carrier is
> largely a
> footprint/specificity tradeoff.? One could imagine a carrier class per
> matcher,
> or a carrier class per matcher descriptor, or using `Object[]` as a
> carrier for
> everything, or caching some number of common shapes (e.g, three ints and two
> refs).? This sort of tuning should be separate from the protocol encoded
> in the
> bytecode of the pattern method and its clients.
> 
> We use a small _carrier runtime_ to decouple pattern translation from
> carrier
> selection.? (This same carrier runtime is used by string templates as well.)
> This allows tradeoffs in runtime characteristics (e.g., carrier per
> matcher vs
> sharing carriers across matchers, dropping carrier identity with value types
> later, etc) without affecting the translation. The carrier API consists
> of condy
> bootstraps like:
> 
> ```
> static MethodHandle carrierFactory(MethodType matcherDescriptor) { ... }
> static MethodHandle carrierAccessor(MethodType matcherDescriptor, int
> bindingNo) { ... }
> ```
> 
> The `matcherDescriptor` is a `MethodType` describing the binding types.? The
> `carrierFactory` method returns a method handle which takes the bindings and
> produces a carrier object; the `carrierAccessor` method returns method
> handles
> that take the carrier object and return the corresponding binding.? To
> indicate
> success, the matcher method invokes the carrier factory method handle and
> returns the result; to indicate failure (deconstructors cannot fail, but
> other
> matchers can) the matcher method returns null.
> 
> We would translate the XY deconstructor from `Point` as follows
> (pseudo-code):
> 
> ```
> #100: MethodType[(II)V]
> #101: Condy[bsm=Carriers::carrierFactory, args=[#100]]
> 
> final synthetic Object Point$MANGLE() {
> ??? aload_0
> ??? getfield Point::x
> ??? aload_0
> ??? getfield Point::y
> ??? LDC #101
> ??? invokevirtual MethodHandle::invoke(II)V
> ??? areturn
> }
> ```
> 
> Constant `#100` contains a `MethodType` holding the binding descriptor;
> constant
> `#101` holds a method handle whose parameters are the parameter types of the
> binding descriptor and returns `Object`.
> 
> At the use site, matching a deconstruction pattern is performed by
> invoking the
> matcher method on the appropriate target object, and then extracting the
> components with the carrier accessor method handles if the match is
> successful.
> (Deconstructors are total, so are always successful, but for other patterns,
> null is returned from the matcher method on failure to match.)


First, there is a small typo, 
  invokevirtual MethodHandle::invoke(II)V

should be
  invokevirtual MethodHandle::invoke(II)Ljava/lang/Object;


I believe that indy is better than ldc condy + MH.invokevirtual because the main difference between these two approaches is that in the case of indy the bootstrap method receive the Lookup and I believe you need the Lookup at runtime (exactly the ClassLoader) to decode the field "patternDescr" of the attribute Matcher (to transform it to a MethodType).

regards,
R?mi

From heidinga at redhat.com  Tue Mar  7 13:21:47 2023
From: heidinga at redhat.com (Dan Heidinga)
Date: Tue, 7 Mar 2023 08:21:47 -0500
Subject: Matcher method name mangling Was: Deconstruction patterns
In-Reply-To: <1219316046.5174614.1678180875926.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <1219316046.5174614.1678180875926.JavaMail.zimbra@u-pem.fr>
Message-ID: <CAJq4Gi45An1yQVewUDED52UnB8zfOHAq7J=zBB-UyETty93LUw@mail.gmail.com>

On Tue, Mar 7, 2023 at 4:21?AM Remi Forax <forax at univ-mlv.fr> wrote:

> ----- Original Message -----
> > From: "Brian Goetz" <brian.goetz at oracle.com>
> > To: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> > Sent: Monday, March 6, 2023 7:24:54 PM
> > Subject: Deconstruction patterns
>
> > Time to look ahead to the next installment of pattern matching:
> > deconstruction patterns, which generalize record patterns.  This
> > document does an end-to-end walkthrough (at a sketchy level of detail)
> > through declaration, overloading, use, translation, and reflection of
> > deconstruction patterns.
> >
> > I would like to *not* discuss syntax at this time.  There's a lengthy
> > discussion to be had about syntax, and we'll have that, but let's nail
> > down model, semantics, and translation first.
> >
> > As usual, I would prefer that people either (a) post a single reply
> > addressing the totality of this sketch or (b) start _new threads_ if you
> > want to discuss a specific aspect.  A quick "I'll just reply to this
> > minor detail" seems to often derail the conversation in such a way that
> > it never comes back.  If this all looks fine to you, a quick "no
> > surprises here" will keep us from suspensefully waiting for feedback.
> >
> >
> > # Deconstruction patterns -- translation, use, and reflection
> >
> > As we are wrapping up record patterns, it's time to look ahead to the
> > next major
> > part of the pattern matching story -- extending the capabilities of
> record
> > patterns to all classes that want to support destructuring. Record
> > patterns are
> > simply a special case of _deconstruction patterns_ or _deconstructors_,
> > where we
> > derive the deconstructor API, implementation, and use from the state
> > description
> > of the record.  For an arbitrary class, a deconstruction patterns will
> > require
> > an explicit member declaration, with a header identifying the names and
> > types of
> > the bindings and a body that extracts the bindings from the
> representation.
> >
> > ## Deconstructors
> >
> > Just as constructors are special cases of methods, deconstruction
> > patterns are
> > special cases of a more general notion of declared pattern, which also
> > includes
> > static matchers (the dual of static methods) and instance matchers (the
> > dual of
> > instance methods.)  Specifically, unlike the more general notion of
> > matcher, a
> > deconstructor must be _total_; it must always match.  This document will
> > focus
> > exclusively on deconstructors, and we'll come back to static and instance
> > matchers in due time.  (But note that some of the design choices in the
> > simple
> > case of deconstructors may be constrained by the more general case.)
> >
> > There are a number of choices for how we might syntactically represent a
> > deconstructor (or more generally, a declared pattern.)  For purposes of
> > illustration, this document picks one possible syntactic expression of
> > deconstructors, but it is premature to devolve into a syntax discussion
> > at this
> > time.
> >
> > ```
> > class Point {
> >     final double x, y;
> >
> >     public Point(double x, double y) {
> >         this.x = x;
> >         this.y = y;
> >     }
> >
> >     public matcher Point(double x, double y) {
> >         x = this.x;
> >         y = this.y;
> >     }
> > }
> > ```
> >
> > This example illustrates two aspects of the duality between constructors
> and
> > their corresponding deconstructors.  Their APIs are duals: a constructor
> > takes N
> > parameters containing the desired description of the object state and
> > produces a
> > constructed object; a deconstructor starts from the constructed object
> > and has N
> > bindings (outputs) that receive the desired state components. Similarly,
> > their
> > implementations are duals: the body of the constructor initializes the
> > object
> > representation from the description, and the body of the deconstructor
> > extracts
> > the description from the representation.  A deconstructor is best
> > understood as
> > a _co-constructor_.
> >
> > The `Point` example above is special in two ways.  First, the internal
> > representation of a `Point`, and the API of the constructor and
> > deconstructor,
> > are the same: `(double x, double y)`.  We can call the API implied by the
> > constructor and deconstructor the _external representation_, and for
> > `Point`,
> > both the internal and external representations are the same. (This is
> one of
> > the requirements for being a candidate to be a record.)  And second, the
> > constructor is _total_; it does not reject any combinations of arguments.
> >
> > Here's another version of `Point` which does not have these special
> > aspects; it
> > uses the same internal representation as before, but chooses a pair of
> > strings
> > as the external representation:
> >
> > ```
> > class Point2 {
> >     final double x, y;
> >
> >     public Point2(String x, String y) {
> >         this.x = Double.parseDouble(x);
> >         this.y = Double.parseDouble(y);
> >     }
> >
> >     public matcher Point2(String x, String y) {
> >         x = Double.toString(this.x);
> >         y = Double.toSTring(this.y);
> >     }
> > }
> > ```
> >
> > The method `Double::parseDouble` will throw `NumberFormatException` if
> its
> > argument does not describe a suitable value, so unlike the `Point`
> > constructor,
> > the `Point2` constructor is partial: it will reject `new Double("foo",
> > "bar")`.
> > And the internal representation is no longer the same as the external
> > representation.  Less obviously, there are valid string values that we
> can
> > provide to the constructor, but which cannot be represented exactly as
> > `double`,
> > and which will be approximated; the string value
> > `"3.22222222222222222222222222222222222222"` will be approximated with
> the
> > double value `3.2222222222222223`.
> >
> > This example highlights more clearly how the constructor and
> > deconstructor form
> > an _embedding-projection pair_ between the internal and external
> > representations.  While some external representations might be invalid,
> > and some
> > might result in approximation, deconstruct-then-construct is always an
> > identity
> > transformation.  Indeed, the specification of `java.lang.Record`
> > requires that
> > if we deconstruct a record with its accessors, and pass the resulting
> values
> > back to the constructor, we should get a new record that is `equals` to
> the
> > original.
> >
> > The fact that constructor and deconstructor (and eventually, factory and
> > static
> > matcher) form an embedding-projection pair is why we are able to derive
> > higher-level language features, such as [safer
> > serialization](
> https://openjdk.org/projects/amber/design-notes/towards-better-serialization
> )
> > and [functional transformation of immutable
> > objects](
> https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md
> ),
> > from a matched set of constructor and deconstructor.
> >
> > Of course, users are free to implement constructors without
> > deconstructors, or
> > constructors and deconstructors whose external representations don't
> > match up,
> > or even matching constructors and deconstructors that are not
> > behaviorally dual.
> > But providing a matched set (or several) of constructors and
> deconstructors
> > enables reliably reversible aggregation, and allows us to mechanically
> > derive
> > useful higher-level features such as withers.
> >
> > #### Overloading
> >
> > Just as constructors can be overloaded, deconstructors can be overloaded
> > for the
> > same reason: multiple constructors can expose multiple external
> > representations
> > for aggregation, and corresponding deconstructors can recover those
> multiple
> > external representations.  Any matching pair of
> > constructor-deconstructor (and
> > eventually, factory-deconstructor) is a candidate for use in higher-level
> > features based on the embedding-projection nature of the
> > constructor-deconstructor pair.
> >
> > Just as deconstruction is dual to construction, overloading of
> > deconstructors is
> > dual to that of constructors: rather than restricting which sets of
> > parameters
> > can be overloaded against each other, we do so with the bindings
> > instead.  For
> > constructors of a given arity, we require that their signatures not be
> > override-equivalent; for deconstructors of a given arity, we require the
> > same of
> > their bindings.
> >
> > For a deconstructor (and declared patterns in general), we derive a
> _binding
> > signature_ (and an erased _binding descriptor_) which treats the binding
> > list as
> > a parameter list.  The overload rule outlined above requires that binding
> > signatures for two deconstructors of the same arity not be
> > override-equivalent.
> > (We will find it useful later to derive a `MethodType` for the binding
> > descriptor; this is a `MethodType` whose return type is `V` and whose
> > parameter
> > types are the erased types of the bindings.)
> >
> > #### Digression: embedding-projection pairs
> >
> > Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B
> > -> A`,
> > forms an _embedding-projection pair_ if `p . e` (embed then project) is
> an
> > identity function, and `e . p` (project then embed) _approximates_ the
> input
> > according to a domain-specific approximation metric (which is a complete
> > partial
> > ordering on `B`.)
> >
> > When applied to constructor-deconstructor pairs, this says that
> > deconstructing
> > an object and then reconstructing it with the resulting bindings should
> > result
> > in an equivalent object, and constructing an object from an external
> > representation and then deconstructing it back into that external
> > representation
> > should result in an approximation of the original external
> > representation.  (A
> > complete partial ordering models constructor failure as the
> non-terminating
> > bottom value, which is considered an infinitely bad approximation to
> > everything.)
> >
> > Embedding-projection pairs have a number of desirable properties, such
> > as the
> > composition of two e-p pairs is an e-p pair; this property is at the
> > heart of
> > using constructor-deconstructor pairs for improved serialization and
> > functional
> > transformation.
> >
> > ## Invoking deconstructors
> >
> > We've already seen how to "invoke" deconstructors: through pattern
> matching.
> > What we've been calling "record patterns" are merely deconstruction
> patterns
> > derived mechanically from the state description, just as we do with
> > constructors
> > and accessors; there is little difference between record patterns and
> > deconstruction patterns other than the ability to declare them
> explicitly.
> > (There is an accidental difference in the translation, in that we
> currently
> > implement record patterns by appealing to individual accessors rather
> than a
> > single deconstructor, but this may eventually converge as well.)
> >
> > The use-site syntax of deconstruction bears a deliberate similarity to
> > that of
> > construction; `new Point(x, y)` is deconstructed by `case Point(var x,
> > var y)`.
> >
> > #### Overload selection
> >
> > In the presence of overloaded deconstructors, we need to figure out which
> > deconstructor a deconstruction pattern `C(P*)` is referring to. The
> > details are
> > similar to overload selection for methods, except that we operate on the
> > bindings rather than the parameters.  We first search for _applicable
> > matchers_,
> > using increasingly loose criteria (first excluding boxing, unboxing, and
> > varargs; then including boxing and unboxing but not varargs; and
> > finally, all
> > candidates) and then selecting the most applicable.
> >
> > It is tempting to try and bypass the three-phase selection process and
> use a
> > simpler notion of applicability (perhaps noting that we got this process
> for
> > compatibility with existing overload selection decisions when autoboxing
> and
> > varargs were added, and that there are few deconstructor invocations to
> be
> > compatible with yet.)  But because existing overloaded constructors use
> this
> > mechanism, and there is significant value in pairing constructors and
> > deconstructors, attempting to invent a simpler-but-different overload
> > selection
> > mechanism for deconstructors would inevitably undermine the duality
> between
> > matching constructor-deconstructor pairs. So compatibility (this time,
> with
> > existing overloaded constructors) once again forces our hand.
> >
> > The specification for overload selection is complicated significantly by
> > poly
> > expressions (e.g., lambdas); fortunately, there are no "poly patterns",
> > and so,
> > while the structure of JLS 15.12.2 is retained for overload selection of
> > deconstruction patterns, much of the detail is left behind.
> >
> > ## Translation
> >
> > We translate patterns into synthetic methods with a `Matcher` attribute;
> > this
> > method implements the matcher behavior.  The translation scheme derives
> > from a
> > number of requirements, only some of which are in play for
> deconstructors.
> >
> > The matcher method for a deconstructor is a final instance method that
> > takes no
> > parameters and returns `Object`, perhaps with a special name (just as
> > constructors are called `<init>`.)
> >
> > #### Carriers
> >
> > Because the matcher methods implements the matcher behavior, but a
> > matcher may
> > "return" multiple bindings (or failure), we must encode the bindings in
> some
> > way.  For this, we use a _carrier object_.  The choice of carrier is
> > largely a
> > footprint/specificity tradeoff.  One could imagine a carrier class per
> > matcher,
> > or a carrier class per matcher descriptor, or using `Object[]` as a
> > carrier for
> > everything, or caching some number of common shapes (e.g, three ints and
> two
> > refs).  This sort of tuning should be separate from the protocol encoded
> > in the
> > bytecode of the pattern method and its clients.
> >
> > We use a small _carrier runtime_ to decouple pattern translation from
> > carrier
> > selection.  (This same carrier runtime is used by string templates as
> well.)
> > This allows tradeoffs in runtime characteristics (e.g., carrier per
> > matcher vs
> > sharing carriers across matchers, dropping carrier identity with value
> types
> > later, etc) without affecting the translation. The carrier API consists
> > of condy
> > bootstraps like:
> >
> > ```
> > static MethodHandle carrierFactory(MethodType matcherDescriptor) { ... }
> > static MethodHandle carrierAccessor(MethodType matcherDescriptor, int
> > bindingNo) { ... }
> > ```
> >
> > The `matcherDescriptor` is a `MethodType` describing the binding types.
> The
> > `carrierFactory` method returns a method handle which takes the bindings
> and
> > produces a carrier object; the `carrierAccessor` method returns method
> > handles
> > that take the carrier object and return the corresponding binding.  To
> > indicate
> > success, the matcher method invokes the carrier factory method handle and
> > returns the result; to indicate failure (deconstructors cannot fail, but
> > other
> > matchers can) the matcher method returns null.
> >
> > We would translate the XY deconstructor from `Point` as follows
> > (pseudo-code):
> >
> > ```
> > #100: MethodType[(II)V]
> > #101: Condy[bsm=Carriers::carrierFactory, args=[#100]]
> >
> > final synthetic Object Point$MANGLE() {
> >     aload_0
> >     getfield Point::x
> >     aload_0
> >     getfield Point::y
> >     LDC #101
> >     invokevirtual MethodHandle::invoke(II)V
> >     areturn
> > }
> > ```
> >
> > Constant `#100` contains a `MethodType` holding the binding descriptor;
> > constant
> > `#101` holds a method handle whose parameters are the parameter types of
> the
> > binding descriptor and returns `Object`.
> >
> > At the use site, matching a deconstruction pattern is performed by
> > invoking the
> > matcher method on the appropriate target object, and then extracting the
> > components with the carrier accessor method handles if the match is
> > successful.
> > (Deconstructors are total, so are always successful, but for other
> patterns,
> > null is returned from the matcher method on failure to match.)
> >
> > #### Method names
> >
> > The name of the matcher method is mangled to support overloading. The JVM
> > permits overloading on parameter types, but not return types (and
> overloaded
> > matchers are effectively overloaded on return types.)  We take the
> > approach of
> > encoding the erasure of the matcher descriptor in the name of the
> > pattern.  This
> > has several desirable properties: it is stable (the name is derived
> > solely from
> > stable aspects of the declaration), for matchers with override-equivalent
> > signatures (deconstructors can't be overridden, but other patterns can
> be),
> > these map to true overrides in the translation, and valid overloads of
> > matchers
> > will always have distinct names.
> >
> > We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
> > descriptor as the mangled disambiguator, which is exactly as stable as
> > any other
> > method descriptor derived from source declarations.
>
> I do not think you need mangling, you need different name but matcher$0,
> matcher$1, etc or something similar with a counter should be enough.
>
> The Matcher attribute contains enough information to do the linking both
> at compile time and at runtime.
>
> The idea is that in the bytecode, you can derive the MethodType from the
> invokeVirtual (or invokedynamic) and then ask the VM the method that has a
> Matcher attribute that matches that MethodType.
> So the exact name of the matcher method should not be present in the
> bytecode of a switch that uses a deconstructor, only the corresponding
> MethodType should be present.
>
> Mangling method names appear in stack trace, if we can have "light
> mangling", i.e. a counter likes with lambdas, instead of a C++ like
> mangling, i think it's a win.
>

A couple of thoughts:
* counter based names aren't very friendly to the broader ecosystem,
especially to native image / precompilation projects.  Best to avoid them
when possible. i.e. There was a recent PR [0] that removed the counter from
the hidden classes created by the LambdaMetafactory to make the build more
reproducible for native image generators.

* embedding the mangled descriptor in the name is a big aid to debugging
for users and service engineers.  It's much easier to tell which matcher
was called from a stacktrace when you only have the stacktrace and the
source code than when you need to figure out which matcher got numbered as
matcher$1.

* stable names allow for existing override mechanisms (ie: vtables) to
work.  Not sure this is the intended strategy but if it is, then we need a
name that ensures that matchers implemented in subclasses correctly
override the ones in superclasses.  While this can be done without the
names, the VM is already really good at matching names and descriptors.

I haven't thought through the proposal to avoid having the names in the
bytecode enough to comment on that yet.

--Dan

[0] https://github.com/openjdk/jdk/pull/12579


>
> R?mi
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230307/7f53db55/attachment-0001.htm>

From forax at univ-mlv.fr  Tue Mar  7 14:02:45 2023
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Tue, 7 Mar 2023 15:02:45 +0100 (CET)
Subject: Matcher method name mangling Was: Deconstruction patterns
In-Reply-To: <CAJq4Gi45An1yQVewUDED52UnB8zfOHAq7J=zBB-UyETty93LUw@mail.gmail.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <1219316046.5174614.1678180875926.JavaMail.zimbra@u-pem.fr>
 <CAJq4Gi45An1yQVewUDED52UnB8zfOHAq7J=zBB-UyETty93LUw@mail.gmail.com>
Message-ID: <1205375024.5404024.1678197765922.JavaMail.zimbra@u-pem.fr>

The message from this sender included one or more files
which could not be scanned for virus detection; do not
open these files unless you are certain of the sender's intent.

----------------------------------------------------------------------
> From: "Dan Heidinga" <heidinga at redhat.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "Brian Goetz" <brian.goetz at oracle.com>, "amber-spec-experts"
> <amber-spec-experts at openjdk.java.net>
> Sent: Tuesday, March 7, 2023 2:21:47 PM
> Subject: Re: Matcher method name mangling Was: Deconstruction patterns

> On Tue, Mar 7, 2023 at 4:21 AM Remi Forax < [ mailto:forax at univ-mlv.fr |
> forax at univ-mlv.fr ] > wrote:

>> ----- Original Message -----
>>> From: "Brian Goetz" < [ mailto:brian.goetz at oracle.com | brian.goetz at oracle.com ]
>> > >
>>> To: "amber-spec-experts" < [ mailto:amber-spec-experts at openjdk.java.net |
>> > amber-spec-experts at openjdk.java.net ] >
>> > Sent: Monday, March 6, 2023 7:24:54 PM
>> > Subject: Deconstruction patterns

>> > Time to look ahead to the next installment of pattern matching:
>> > deconstruction patterns, which generalize record patterns. This
>> > document does an end-to-end walkthrough (at a sketchy level of detail)
>> > through declaration, overloading, use, translation, and reflection of
>> > deconstruction patterns.

>> > I would like to *not* discuss syntax at this time. There's a lengthy
>> > discussion to be had about syntax, and we'll have that, but let's nail
>> > down model, semantics, and translation first.

>> > As usual, I would prefer that people either (a) post a single reply
>> > addressing the totality of this sketch or (b) start _new threads_ if you
>> > want to discuss a specific aspect. A quick "I'll just reply to this
>> > minor detail" seems to often derail the conversation in such a way that
>> > it never comes back. If this all looks fine to you, a quick "no
>> > surprises here" will keep us from suspensefully waiting for feedback.


>> > # Deconstruction patterns -- translation, use, and reflection

>> > As we are wrapping up record patterns, it's time to look ahead to the
>> > next major
>> > part of the pattern matching story -- extending the capabilities of record
>> > patterns to all classes that want to support destructuring. Record
>> > patterns are
>> > simply a special case of _deconstruction patterns_ or _deconstructors_,
>> > where we
>> > derive the deconstructor API, implementation, and use from the state
>> > description
>> > of the record. For an arbitrary class, a deconstruction patterns will
>> > require
>> > an explicit member declaration, with a header identifying the names and
>> > types of
>> > the bindings and a body that extracts the bindings from the representation.

>> > ## Deconstructors

>> > Just as constructors are special cases of methods, deconstruction
>> > patterns are
>> > special cases of a more general notion of declared pattern, which also
>> > includes
>> > static matchers (the dual of static methods) and instance matchers (the
>> > dual of
>> > instance methods.) Specifically, unlike the more general notion of
>> > matcher, a
>> > deconstructor must be _total_; it must always match. This document will
>> > focus
>> > exclusively on deconstructors, and we'll come back to static and instance
>> > matchers in due time. (But note that some of the design choices in the
>> > simple
>> > case of deconstructors may be constrained by the more general case.)

>> > There are a number of choices for how we might syntactically represent a
>> > deconstructor (or more generally, a declared pattern.) For purposes of
>> > illustration, this document picks one possible syntactic expression of
>> > deconstructors, but it is premature to devolve into a syntax discussion
>> > at this
>> > time.

>> > ```
>> > class Point {
>> > final double x, y;

>> > public Point(double x, double y) {
>> > this.x = x;
>> > this.y = y;
>> > }

>> > public matcher Point(double x, double y) {
>> > x = this.x;
>> > y = this.y;
>> > }
>> > }
>> > ```

>> > This example illustrates two aspects of the duality between constructors and
>> > their corresponding deconstructors. Their APIs are duals: a constructor
>> > takes N
>> > parameters containing the desired description of the object state and
>> > produces a
>> > constructed object; a deconstructor starts from the constructed object
>> > and has N
>> > bindings (outputs) that receive the desired state components. Similarly,
>> > their
>> > implementations are duals: the body of the constructor initializes the
>> > object
>> > representation from the description, and the body of the deconstructor
>> > extracts
>> > the description from the representation. A deconstructor is best
>> > understood as
>> > a _co-constructor_.

>> > The `Point` example above is special in two ways. First, the internal
>> > representation of a `Point`, and the API of the constructor and
>> > deconstructor,
>> > are the same: `(double x, double y)`. We can call the API implied by the
>> > constructor and deconstructor the _external representation_, and for
>> > `Point`,
>> > both the internal and external representations are the same. (This is one of
>> > the requirements for being a candidate to be a record.) And second, the
>> > constructor is _total_; it does not reject any combinations of arguments.

>> > Here's another version of `Point` which does not have these special
>> > aspects; it
>> > uses the same internal representation as before, but chooses a pair of
>> > strings
>> > as the external representation:

>> > ```
>> > class Point2 {
>> > final double x, y;

>> > public Point2(String x, String y) {
>> > this.x = Double.parseDouble(x);
>> > this.y = Double.parseDouble(y);
>> > }

>> > public matcher Point2(String x, String y) {
>> > x = Double.toString(this.x);
>> > y = Double.toSTring(this.y);
>> > }
>> > }
>> > ```

>> > The method `Double::parseDouble` will throw `NumberFormatException` if its
>> > argument does not describe a suitable value, so unlike the `Point`
>> > constructor,
>> > the `Point2` constructor is partial: it will reject `new Double("foo",
>> > "bar")`.
>> > And the internal representation is no longer the same as the external
>> > representation. Less obviously, there are valid string values that we can
>> > provide to the constructor, but which cannot be represented exactly as
>> > `double`,
>> > and which will be approximated; the string value
>> > `"3.22222222222222222222222222222222222222"` will be approximated with the
>> > double value `3.2222222222222223`.

>> > This example highlights more clearly how the constructor and
>> > deconstructor form
>> > an _embedding-projection pair_ between the internal and external
>> > representations. While some external representations might be invalid,
>> > and some
>> > might result in approximation, deconstruct-then-construct is always an
>> > identity
>> > transformation. Indeed, the specification of `java.lang.Record`
>> > requires that
>> > if we deconstruct a record with its accessors, and pass the resulting values
>> > back to the constructor, we should get a new record that is `equals` to the
>> > original.

>> > The fact that constructor and deconstructor (and eventually, factory and
>> > static
>> > matcher) form an embedding-projection pair is why we are able to derive
>> > higher-level language features, such as [safer
>>> serialization]( [
>>> https://openjdk.org/projects/amber/design-notes/towards-better-serialization |
>>> https://openjdk.org/projects/amber/design-notes/towards-better-serialization ]
>> > )
>> > and [functional transformation of immutable
>>> objects]( [
>>> https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md
>>> |
>>> https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md
>> > ] ),
>> > from a matched set of constructor and deconstructor.

>> > Of course, users are free to implement constructors without
>> > deconstructors, or
>> > constructors and deconstructors whose external representations don't
>> > match up,
>> > or even matching constructors and deconstructors that are not
>> > behaviorally dual.
>> > But providing a matched set (or several) of constructors and deconstructors
>> > enables reliably reversible aggregation, and allows us to mechanically
>> > derive
>> > useful higher-level features such as withers.

>> > #### Overloading

>> > Just as constructors can be overloaded, deconstructors can be overloaded
>> > for the
>> > same reason: multiple constructors can expose multiple external
>> > representations
>> > for aggregation, and corresponding deconstructors can recover those multiple
>> > external representations. Any matching pair of
>> > constructor-deconstructor (and
>> > eventually, factory-deconstructor) is a candidate for use in higher-level
>> > features based on the embedding-projection nature of the
>> > constructor-deconstructor pair.

>> > Just as deconstruction is dual to construction, overloading of
>> > deconstructors is
>> > dual to that of constructors: rather than restricting which sets of
>> > parameters
>> > can be overloaded against each other, we do so with the bindings
>> > instead. For
>> > constructors of a given arity, we require that their signatures not be
>> > override-equivalent; for deconstructors of a given arity, we require the
>> > same of
>> > their bindings.

>> > For a deconstructor (and declared patterns in general), we derive a _binding
>> > signature_ (and an erased _binding descriptor_) which treats the binding
>> > list as
>> > a parameter list. The overload rule outlined above requires that binding
>> > signatures for two deconstructors of the same arity not be
>> > override-equivalent.
>> > (We will find it useful later to derive a `MethodType` for the binding
>> > descriptor; this is a `MethodType` whose return type is `V` and whose
>> > parameter
>> > types are the erased types of the bindings.)

>> > #### Digression: embedding-projection pairs

>> > Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B
>> > -> A`,
>> > forms an _embedding-projection pair_ if `p . e` (embed then project) is an
>> > identity function, and `e . p` (project then embed) _approximates_ the input
>> > according to a domain-specific approximation metric (which is a complete
>> > partial
>> > ordering on `B`.)

>> > When applied to constructor-deconstructor pairs, this says that
>> > deconstructing
>> > an object and then reconstructing it with the resulting bindings should
>> > result
>> > in an equivalent object, and constructing an object from an external
>> > representation and then deconstructing it back into that external
>> > representation
>> > should result in an approximation of the original external
>> > representation. (A
>> > complete partial ordering models constructor failure as the non-terminating
>> > bottom value, which is considered an infinitely bad approximation to
>> > everything.)

>> > Embedding-projection pairs have a number of desirable properties, such
>> > as the
>> > composition of two e-p pairs is an e-p pair; this property is at the
>> > heart of
>> > using constructor-deconstructor pairs for improved serialization and
>> > functional
>> > transformation.

>> > ## Invoking deconstructors

>> > We've already seen how to "invoke" deconstructors: through pattern matching.
>> > What we've been calling "record patterns" are merely deconstruction patterns
>> > derived mechanically from the state description, just as we do with
>> > constructors
>> > and accessors; there is little difference between record patterns and
>> > deconstruction patterns other than the ability to declare them explicitly.
>> > (There is an accidental difference in the translation, in that we currently
>> > implement record patterns by appealing to individual accessors rather than a
>> > single deconstructor, but this may eventually converge as well.)

>> > The use-site syntax of deconstruction bears a deliberate similarity to
>> > that of
>> > construction; `new Point(x, y)` is deconstructed by `case Point(var x,
>> > var y)`.

>> > #### Overload selection

>> > In the presence of overloaded deconstructors, we need to figure out which
>> > deconstructor a deconstruction pattern `C(P*)` is referring to. The
>> > details are
>> > similar to overload selection for methods, except that we operate on the
>> > bindings rather than the parameters. We first search for _applicable
>> > matchers_,
>> > using increasingly loose criteria (first excluding boxing, unboxing, and
>> > varargs; then including boxing and unboxing but not varargs; and
>> > finally, all
>> > candidates) and then selecting the most applicable.

>> > It is tempting to try and bypass the three-phase selection process and use a
>> > simpler notion of applicability (perhaps noting that we got this process for
>> > compatibility with existing overload selection decisions when autoboxing and
>> > varargs were added, and that there are few deconstructor invocations to be
>> > compatible with yet.) But because existing overloaded constructors use this
>> > mechanism, and there is significant value in pairing constructors and
>> > deconstructors, attempting to invent a simpler-but-different overload
>> > selection
>> > mechanism for deconstructors would inevitably undermine the duality between
>> > matching constructor-deconstructor pairs. So compatibility (this time, with
>> > existing overloaded constructors) once again forces our hand.

>> > The specification for overload selection is complicated significantly by
>> > poly
>> > expressions (e.g., lambdas); fortunately, there are no "poly patterns",
>> > and so,
>> > while the structure of JLS 15.12.2 is retained for overload selection of
>> > deconstruction patterns, much of the detail is left behind.

>> > ## Translation

>> > We translate patterns into synthetic methods with a `Matcher` attribute;
>> > this
>> > method implements the matcher behavior. The translation scheme derives
>> > from a
>> > number of requirements, only some of which are in play for deconstructors.

>> > The matcher method for a deconstructor is a final instance method that
>> > takes no
>> > parameters and returns `Object`, perhaps with a special name (just as
>> > constructors are called `<init>`.)

>> > #### Carriers

>> > Because the matcher methods implements the matcher behavior, but a
>> > matcher may
>> > "return" multiple bindings (or failure), we must encode the bindings in some
>> > way. For this, we use a _carrier object_. The choice of carrier is
>> > largely a
>> > footprint/specificity tradeoff. One could imagine a carrier class per
>> > matcher,
>> > or a carrier class per matcher descriptor, or using `Object[]` as a
>> > carrier for
>> > everything, or caching some number of common shapes (e.g, three ints and two
>> > refs). This sort of tuning should be separate from the protocol encoded
>> > in the
>> > bytecode of the pattern method and its clients.

>> > We use a small _carrier runtime_ to decouple pattern translation from
>> > carrier
>> > selection. (This same carrier runtime is used by string templates as well.)
>> > This allows tradeoffs in runtime characteristics (e.g., carrier per
>> > matcher vs
>> > sharing carriers across matchers, dropping carrier identity with value types
>> > later, etc) without affecting the translation. The carrier API consists
>> > of condy
>> > bootstraps like:

>> > ```
>> > static MethodHandle carrierFactory(MethodType matcherDescriptor) { ... }
>> > static MethodHandle carrierAccessor(MethodType matcherDescriptor, int
>> > bindingNo) { ... }
>> > ```

>> > The `matcherDescriptor` is a `MethodType` describing the binding types. The
>> > `carrierFactory` method returns a method handle which takes the bindings and
>> > produces a carrier object; the `carrierAccessor` method returns method
>> > handles
>> > that take the carrier object and return the corresponding binding. To
>> > indicate
>> > success, the matcher method invokes the carrier factory method handle and
>> > returns the result; to indicate failure (deconstructors cannot fail, but
>> > other
>> > matchers can) the matcher method returns null.

>> > We would translate the XY deconstructor from `Point` as follows
>> > (pseudo-code):

>> > ```
>> > #100: MethodType[(II)V]
>> > #101: Condy[bsm=Carriers::carrierFactory, args=[#100]]

>> > final synthetic Object Point$MANGLE() {
>> > aload_0
>> > getfield Point::x
>> > aload_0
>> > getfield Point::y
>> > LDC #101
>> > invokevirtual MethodHandle::invoke(II)V
>> > areturn
>> > }
>> > ```

>> > Constant `#100` contains a `MethodType` holding the binding descriptor;
>> > constant
>> > `#101` holds a method handle whose parameters are the parameter types of the
>> > binding descriptor and returns `Object`.

>> > At the use site, matching a deconstruction pattern is performed by
>> > invoking the
>> > matcher method on the appropriate target object, and then extracting the
>> > components with the carrier accessor method handles if the match is
>> > successful.
>> > (Deconstructors are total, so are always successful, but for other patterns,
>> > null is returned from the matcher method on failure to match.)

>> > #### Method names

>> > The name of the matcher method is mangled to support overloading. The JVM
>> > permits overloading on parameter types, but not return types (and overloaded
>> > matchers are effectively overloaded on return types.) We take the
>> > approach of
>> > encoding the erasure of the matcher descriptor in the name of the
>> > pattern. This
>> > has several desirable properties: it is stable (the name is derived
>> > solely from
>> > stable aspects of the declaration), for matchers with override-equivalent
>> > signatures (deconstructors can't be overridden, but other patterns can be),
>> > these map to true overrides in the translation, and valid overloads of
>> > matchers
>> > will always have distinct names.

>> > We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
>> > descriptor as the mangled disambiguator, which is exactly as stable as
>> > any other
>> > method descriptor derived from source declarations.

>> I do not think you need mangling, you need different name but matcher$0,
>> matcher$1, etc or something similar with a counter should be enough.

>> The Matcher attribute contains enough information to do the linking both at
>> compile time and at runtime.

>> The idea is that in the bytecode, you can derive the MethodType from the
>> invokeVirtual (or invokedynamic) and then ask the VM the method that has a
>> Matcher attribute that matches that MethodType.
>> So the exact name of the matcher method should not be present in the bytecode of
>> a switch that uses a deconstructor, only the corresponding MethodType should be
>> present.

>> Mangling method names appear in stack trace, if we can have "light mangling",
>> i.e. a counter likes with lambdas, instead of a C++ like mangling, i think it's
>> a win.

> A couple of thoughts:
> * counter based names aren't very friendly to the broader ecosystem, especially
> to native image / precompilation projects. Best to avoid them when possible.
> i.e. There was a recent PR [0] that removed the counter from the hidden classes
> created by the LambdaMetafactory to make the build more reproducible for native
> image generators.

Counter based class name *at runtime* (where the thread scheduling change from run to run) is not the same issue as counter based method name at *compile time*. 
I think the problem is not the same here. 

> * embedding the mangled descriptor in the name is a big aid to debugging for
> users and service engineers. It's much easier to tell which matcher was called
> from a stacktrace when you only have the stacktrace and the source code than
> when you need to figure out which matcher got numbered as matcher$1.

Yes, it helps VM/JDK engineers, not necessary end users. We have kept the name of the desugared lambdas as simple as possible even if having the type of the parameters would have helped people that knows exactly how the compiler desugars things for the runtime. 

> * stable names allow for existing override mechanisms (ie: vtables) to work. Not
> sure this is the intended strategy but if it is, then we need a name that
> ensures that matchers implemented in subclasses correctly override the ones in
> superclasses. While this can be done without the names, the VM is already
> really good at matching names and descriptors.

That a stronger argument. Not having stable name will cause trouble if the base class and the subclass are compiled separately. Like you, i am not sure of the exact strategy around overriding of matcher methods. During previous discussions, we talked about overridable deconstructors but it seems Brian has now rule them out. 

> I haven't thought through the proposal to avoid having the names in the bytecode
> enough to comment on that yet.

> --Dan

> [0] [ https://github.com/openjdk/jdk/pull/12579 |
> https://github.com/openjdk/jdk/pull/12579 ]

R?mi 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230307/a843e32f/attachment-0001.htm>

From brian.goetz at oracle.com  Tue Mar  7 14:20:25 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 09:20:25 -0500
Subject: Overloading of matcher method Was: Deconstruction patterns
In-Reply-To: <969212125.5143862.1678179347294.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <969212125.5143862.1678179347294.JavaMail.zimbra@u-pem.fr>
Message-ID: <0a144b91-13ab-6af9-bce5-f5916c05bf73@oracle.com>


>> For a deconstructor (and declared patterns in general), we derive a _binding
>> signature_ (and an erased _binding descriptor_) which treats the binding
>> list as
>> a parameter list.? The overload rule outlined above requires that binding
>> signatures for two deconstructors of the same arity not be
>> override-equivalent.
>> (We will find it useful later to derive a `MethodType` for the binding
>> descriptor; this is a `MethodType` whose return type is `V` and whose
>> parameter
>> types are the erased types of the bindings.)
>
> I am still not able to compute how it is supposed to work.
> Given that inside a record pattern you can have type patterns, the type of bindings can not be used to select matcher methods, because the type of the bindings may be a subtypes of the type of the matcher method.
>
> class Foo {
>    public Foo(String s) { ... }
>    public Foo(CharSequence seq) { ... }
>
>    public matcher Foo(String s) { ... }
>    public matcher Foo(CharSequence seq) { ... }
> }
>
> If i want to call Foo(CharSequence) with a String, i can use a cast, new Foo((CharSequence) "foo") and the compiler selects the right overload.
>
> How i can do the same to select the right matcher method inside a deconstructor pattern ?

Let's start without overloads.? Suppose I have

 ??? record Foo(String s) { }

and a client does

 ??? anObject instanceof Foo(String s)

This is a nested pattern; a record pattern Foo(...) with a type pattern 
`String s` nested in it.? The first question the compiler asks is: is 
the pattern `Foo(String s)` applicable to Object?? And answering this 
question has two parts:

 ?- Is the type of the record pattern (Foo) cast-convertible to Object 
without unchecked conversion?? (Yes)
 ?- The first component type of Foo is String.? Is the nested pattern 
`String s` applicable to String?? (Yes)

So since both of these are yes, `Foo(String s)` is applicable to Object.

Now imagine I have overloaded matchers:

 ??? matcher Foo(String s)
 ??? matcher Foo(Integer i)
 ??? matcher Foo(CharSequence c)

and the same use site.? First, we filter for applicable matchers. The 
first and third are applicable (same logic as above), the second is not 
(`String s` is not applicable to Integer, because they are not 
cast-convertible.)

Now, as with overloads of methods, we have to pick a "more specific".? 
Is matcher Foo(String s) or matcher Foo(CharSequence c) more specific 
given `Foo(String s)` at the use site?? The term "more specific" is a 
little confusing, so a better way to think about this is minimizing the 
gap between what is at the declaration site and use site.? And 
`Foo(String s)` is a closer match here, so we prefer that one.? (The 
details will be formalized, but they are pretty similar to 15.12.2.5, 
minus all the goop about lambdas.)





From brian.goetz at oracle.com  Tue Mar  7 14:23:29 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 09:23:29 -0500
Subject: Deconstruction patterns
In-Reply-To: <1819623456.5154827.1678180001685.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <1819623456.5154827.1678180001685.JavaMail.zimbra@u-pem.fr>
Message-ID: <e0ea7a11-ae25-2c3d-c86b-85a5c18a9cd4@oracle.com>


> Can we discuss about the semantics of this syntax, given that it has an effect on the translation ?

No.? If you want to have a abstract conversation about "what about 
side-effects in matchers" and order of effects, we can have that (though 
I think that is kind of premature too.)


From brian.goetz at oracle.com  Tue Mar  7 14:31:26 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 09:31:26 -0500
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <667851490.5158494.1678180165559.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <667851490.5158494.1678180165559.JavaMail.zimbra@u-pem.fr>
Message-ID: <07386d40-3cdf-17d0-e5ae-09fb79891f88@oracle.com>


> Why deconstructor can not be overriden ?
>
>
Deconstructors are not inherited, just as constructors are not 
inherited.? However, just as with constructors, a subclass is free to 
declare (or not!) a deconstructor with the same descriptor.

However, deconstructors are still _applicable_ to subtypes:

 ??? class A {
 ??????? public matcher A(String s) { ... }
 ??? }

 ??? class B extends A {
 ??????? // no matchers
 ??? }

 ??? B b = new B(monkey);

 ??? switch (b) {
 ??????? case A(var s): ...? // exhaustive on B
 ??? }

Since a B is-a A, the A pattern is applicable and unconditional on all 
instances of A, including instances of B.? This allows the client to 
discriminate more carefully (assuming B has a similar dtor):

 ??? switch (a) {
 ??????? case B(var s): ...? B logic ...
 ??????? case A(var s): ...? catch-all A logic ...
 ??? }

But, your question raises a good translation question: should we 
translate a dtor as an instance member (like <init>) and then ensure 
that it is only invoked by `invokespecial`, or should we translate it as 
a static?

> Does it means that we can not have a deconstructor on an interface ?

It does not mean that.? Indeed, deconstructors in interfaces are great:

 ??? Map<String, String> kvStore = ...
 ??? for (Map.Entry(var k, var v) : kvStore.entrySet()) { ... }



From brian.goetz at oracle.com  Tue Mar  7 14:33:00 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 09:33:00 -0500
Subject: Use of condy + MH.invokevirtual instead of indy Was
 Deconstruction patterns
In-Reply-To: <1409285726.5174584.1678180872430.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <1409285726.5174584.1678180872430.JavaMail.zimbra@u-pem.fr>
Message-ID: <173c6082-3360-99f2-7b66-4ea53c68e66f@oracle.com>


> First, there is a small typo,
>    invokevirtual MethodHandle::invoke(II)V
>
> should be
>    invokevirtual MethodHandle::invoke(II)Ljava/lang/Object;

Good catch.

> I believe that indy is better than ldc condy + MH.invokevirtual because the main difference between these two approaches is that in the case of indy the bootstrap method receive the Lookup and I believe you need the Lookup at runtime (exactly the ClassLoader) to decode the field "patternDescr" of the attribute Matcher (to transform it to a MethodType).

Condy receives the Lookup as well (there is also an effort underway, 
slowly, to make the metadata (Lookup/name/type) optional for condy, so 
ordinary static methods can be used as bootstraps.)? So this is not a 
difference.



From brian.goetz at oracle.com  Tue Mar  7 14:39:02 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 09:39:02 -0500
Subject: Matcher method name mangling Was: Deconstruction patterns
In-Reply-To: <1219316046.5174614.1678180875926.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <1219316046.5174614.1678180875926.JavaMail.zimbra@u-pem.fr>
Message-ID: <37a6c336-b7ff-bc44-4fe5-41c20cc23e8d@oracle.com>


>> We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
>> descriptor as the mangled disambiguator, which is exactly as stable as
>> any other
>> method descriptor derived from source declarations.
> I do not think you need mangling, you need different name but matcher$0, matcher$1, etc or something similar with a counter should be enough.

This might be OK if the pattern train ended permanently at 
deconstructors (and even then, it is iffy.)

Recall from "Pattern Matching in the Java Object model" that instance 
patterns are a thing, and instance patterns can be overridden.? (This is 
how, for example, `Map` would specify that `get` is really a pattern.)? 
While deconstructors are not inherited and therefore can't be 
overridden, instance patterns can, and you want the translated method 
name in the superclass and subclass to be the same, so that ordinary 
virtual dispatch does the right thing.

Further, even in the absence of inheritance, you want to be able to 
stably select the name of a pattern method in a client at compile time, 
with confidence that, absent a source-incompatible change in the 
declaration, it will select the right method at run time.? Using a 
counter has all the same brittleness as inner class names; if you 
rearrange a source file, Foo$1 will mean something else.

The mangling proposed here is selected to be stable with respect to 
recompilation, reordering members in a source file, adding and removing 
other members, etc, for both selection and overriding.

> The Matcher attribute contains enough information to do the linking both at compile time and at runtime.
>
> The idea is that in the bytecode, you can derive the MethodType from the invokeVirtual (or invokedynamic) and then ask the VM the method that has a Matcher attribute that matches that MethodType.
> So the exact name of the matcher method should not be present in the bytecode of a switch that uses a deconstructor, only the corresponding MethodType should be present.
>
> Mangling method names appear in stack trace, if we can have "light mangling", i.e. a counter likes with lambdas, instead of a C++ like mangling, i think it's a win.

This sounds like "let's invent a new kind of VM dispatch so method names 
can be slightly less ugly."? This doesn't seem like a win to me?

From brian.goetz at oracle.com  Tue Mar  7 14:41:46 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 09:41:46 -0500
Subject: Matcher method name mangling Was: Deconstruction patterns
In-Reply-To: <1205375024.5404024.1678197765922.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <1219316046.5174614.1678180875926.JavaMail.zimbra@u-pem.fr>
 <CAJq4Gi45An1yQVewUDED52UnB8zfOHAq7J=zBB-UyETty93LUw@mail.gmail.com>
 <1205375024.5404024.1678197765922.JavaMail.zimbra@u-pem.fr>
Message-ID: <c2be08d0-3c9c-6715-72e6-36512a3fdc8f@oracle.com>


>
> That a stronger argument. Not having stable name will cause trouble if 
> the base class and the subclass are compiled separately. Like you, i 
> am not sure of the exact strategy around overriding of matcher 
> methods. During previous discussions, we talked about overridable 
> deconstructors but it seems Brian has now rule them out.
>

The model (as outlined in "Pattern Matching in the Java Object Model") 
hasn't changed.? And it's pretty simple.? There are three kinds of 
"methods": constructors, static methods, and instance methods.? 
Constructors are a weird hybrid of static and instance (instance but not 
inherited, etc).

Similarly, there are three kinds of matchers: deconstructors, static 
matchers, and instance matchers.? Again, deconstructors are a weird 
hybrid: instance but not inherited.? (Also, must be total.)

This hasn't changed, the only thing that is different is we're focusing 
on deconstructors now, but we clearly have to come back for the others 
later (and the design has, as stated, taken into account the needs of 
the others as well.)

From forax at univ-mlv.fr  Tue Mar  7 17:53:52 2023
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Tue, 7 Mar 2023 18:53:52 +0100 (CET)
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <07386d40-3cdf-17d0-e5ae-09fb79891f88@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <667851490.5158494.1678180165559.JavaMail.zimbra@u-pem.fr>
 <07386d40-3cdf-17d0-e5ae-09fb79891f88@oracle.com>
Message-ID: <1048964381.5570219.1678211632395.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Tuesday, March 7, 2023 3:31:26 PM
> Subject: Re: Deconstructor can not be overriden ? Was: Deconstruction patterns

>> Why deconstructor can not be overriden ?
>>
>>
> Deconstructors are not inherited, just as constructors are not
> inherited.? However, just as with constructors, a subclass is free to
> declare (or not!) a deconstructor with the same descriptor.
> 
> However, deconstructors are still _applicable_ to subtypes:
> 
> ??? class A {
> ??????? public matcher A(String s) { ... }
> ??? }
> 
> ??? class B extends A {
> ??????? // no matchers
> ??? }
> 
> ??? B b = new B(monkey);
> 
> ??? switch (b) {
> ??????? case A(var s): ...? // exhaustive on B
> ??? }
> 
> Since a B is-a A, the A pattern is applicable and unconditional on all
> instances of A, including instances of B.? This allows the client to
> discriminate more carefully (assuming B has a similar dtor):
> 
> ??? switch (a) {
> ??????? case B(var s): ...? B logic ...
> ??????? case A(var s): ...? catch-all A logic ...
> ??? }


Conceptually that violate the principle of encapsulation, you can access to the state of B without using a method of B.

You talk about de-constructors as the dual of constructors. It can be a useful metaphor but it does not mean that a de-constructor mirrors all the features of a constructor.

A de-constructor is also a bundle of accessors, like if you were able to call all the accessors at once.
Accessors are overridable for a good reason, otherwise you can leak the internal state.

By example,

class A {
  int value;

  A(int value) { this.value = value; }

  int getValue() { return value; }

  public matcher A(int value) { ... }
} 

class B extends A {
  B(int value) {
   super(-value);
  }

  int getValue() { return -value; }

  // here we need to override the matcher otherwise the user code can see the internal state !
}


regards,
R?mi

From brian.goetz at oracle.com  Tue Mar  7 18:02:56 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 13:02:56 -0500
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <1048964381.5570219.1678211632395.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <667851490.5158494.1678180165559.JavaMail.zimbra@u-pem.fr>
 <07386d40-3cdf-17d0-e5ae-09fb79891f88@oracle.com>
 <1048964381.5570219.1678211632395.JavaMail.zimbra@u-pem.fr>
Message-ID: <737a9e58-7e97-942d-8358-9ef22160b74a@oracle.com>


> Conceptually that violate the principle of encapsulation, you can access to the state of B without using a method of B.

No, it is an application of LSP; if a B is-a A, I should be able to use 
a B wherever I currently use an A.? B doesn't necessarily get to 
interpose itself everywhere; it can't interpose against instanceof 
testing for A, or casting to A, and it can't override final methods from 
A.? That's what "extends" means.

> A de-constructor is also a bundle of accessors, like if you were able to call all the accessors at once.
> Accessors are overridable for a good reason, otherwise you can leak the internal state.

Consider A's dtor as a set of final accessors that B cannot override.? 
It can provide other accessors in addition.

If you have a point here, I'm not seeing it?

From forax at univ-mlv.fr  Tue Mar  7 18:40:47 2023
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Tue, 7 Mar 2023 19:40:47 +0100 (CET)
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <737a9e58-7e97-942d-8358-9ef22160b74a@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <667851490.5158494.1678180165559.JavaMail.zimbra@u-pem.fr>
 <07386d40-3cdf-17d0-e5ae-09fb79891f88@oracle.com>
 <1048964381.5570219.1678211632395.JavaMail.zimbra@u-pem.fr>
 <737a9e58-7e97-942d-8358-9ef22160b74a@oracle.com>
Message-ID: <307215629.5587646.1678214447102.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Tuesday, March 7, 2023 7:02:56 PM
> Subject: Re: Deconstructor can not be overriden ? Was: Deconstruction patterns

>> Conceptually that violate the principle of encapsulation, you can access to the
>> state of B without using a method of B.
> 
> No, it is an application of LSP; if a B is-a A, I should be able to use
> a B wherever I currently use an A.? B doesn't necessarily get to
> interpose itself everywhere; it can't interpose against instanceof
> testing for A, or casting to A, and it can't override final methods from
> A.? That's what "extends" means.

"extends" is 3 things,
- subtyping (LSP),
- get all the members of the super class
- MUST override the methods that have the wrong semantics.

Your design miss the point 3. 
If you do not allow overriding, you do not allow the subclass to have a different semantics than the superclass.

It's maybe clearer with an interface

interface Figure {
  public abstract matcher Figure(int surface);
}

class Rectangle implements Figure {
  int width, height;

  public matcher Rectangle(int surface) {
    surface = width * height;
  } 
}


var figure = new Rectangle(4, 5);
switch(figure) {
  case Figure(int surface) -> System.out.println("Surface is " + surface);
}


> 
>> A de-constructor is also a bundle of accessors, like if you were able to call
>> all the accessors at once.
>> Accessors are overridable for a good reason, otherwise you can leak the internal
>> state.
> 
> Consider A's dtor as a set of final accessors that B cannot override.
> It can provide other accessors in addition.
> 
> If you have a point here, I'm not seeing it?

My point is why "final" !

R?mi

From brian.goetz at oracle.com  Tue Mar  7 19:34:02 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 14:34:02 -0500
Subject: Overloading of matcher method Was: Deconstruction patterns
In-Reply-To: <0a144b91-13ab-6af9-bce5-f5916c05bf73@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <969212125.5143862.1678179347294.JavaMail.zimbra@u-pem.fr>
 <0a144b91-13ab-6af9-bce5-f5916c05bf73@oracle.com>
Message-ID: <dced2f23-6925-8d96-85a7-df58efc5f500@oracle.com>

Remi pointed out privately that I didn't answer this question directly 
enough for his satisfaction.

On 3/7/2023 9:20 AM, Brian Goetz wrote:
> class Foo {
> ?? public Foo(String s) { ... }
> ?? public Foo(CharSequence seq) { ... }
>
> ?? public matcher Foo(String s) { ... }
> ?? public matcher Foo(CharSequence seq) { ... }
> }
>
> If i want to call Foo(CharSequence) with a String, i can use a cast, 
> new Foo((CharSequence) "foo") and the compiler selects the right 
> overload.
>
> How i can do the same to select the right matcher method inside a 
> deconstructor pattern ? 

For the constructor, you can guide (but not force) overload selection to 
the answer you want by providing more type information, which in turn 
will steer the "most specific" selection:

 ??? new Foo((String) s)
vs
 ??? new Foo((CharSequence) s)

For the deconstructor, we can do something similar with type patterns.? 
We can say:

 ??? case Foo(String s):
vs
 ??? case Foo(CharSequence s):

For both of these use sites, both matchers are applicable; we're left 
with a question of which is "more specific" to the types present at the 
use site.? As I have mentioned, the details are TBD, but we will draw 
inspiration from dualizing 15.12.2.5.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230307/f3ed20f2/attachment.htm>

From brian.goetz at oracle.com  Tue Mar  7 19:42:37 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 14:42:37 -0500
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <307215629.5587646.1678214447102.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <667851490.5158494.1678180165559.JavaMail.zimbra@u-pem.fr>
 <07386d40-3cdf-17d0-e5ae-09fb79891f88@oracle.com>
 <1048964381.5570219.1678211632395.JavaMail.zimbra@u-pem.fr>
 <737a9e58-7e97-942d-8358-9ef22160b74a@oracle.com>
 <307215629.5587646.1678214447102.JavaMail.zimbra@u-pem.fr>
Message-ID: <bf9e6838-afe9-67b3-f1f7-96a3bb4b0397@oracle.com>


>> No, it is an application of LSP; if a B is-a A, I should be able to use
>> a B wherever I currently use an A.? B doesn't necessarily get to
>> interpose itself everywhere; it can't interpose against instanceof
>> testing for A, or casting to A, and it can't override final methods from
>> A.? That's what "extends" means.
> "extends" is 3 things,
> - subtyping (LSP),
> - get all the members of the super class
> - MUST override the methods that have the wrong semantics.
>
> Your design miss the point 3.

When the B author says "B extends A", they are consenting to the 
contract of A, in all its manifest glory.? A may have final methods.? It 
may have deconstructors.? It may have methods with specific contracts.? 
It only provides a limited menu of constructors that B can delegate to.? 
And B has to ensure that B is substitutible for A in all these ways, or 
it is doing a bad job of extension.

Final methods (and related) are a fact of life; if B cannot be an A 
without overriding these, then it shouldn't extend A.? (This seems so 
obvious I don't even understand why we're talking about it.) Extension 
carries responsibility.

Again, I cannot figure out what your are trying to get at, but it sounds 
to me like you are pointing at a run-of-the-mill interaction but are 
concluding "so the whole design is garbage".? I don't see the line that 
runs from the "this is how it works" to "so it doesn't work", so I'm 
having a hard time understanding what you're getting at.

From heidinga at redhat.com  Tue Mar  7 19:51:59 2023
From: heidinga at redhat.com (Dan Heidinga)
Date: Tue, 7 Mar 2023 14:51:59 -0500
Subject: Deconstructor reflection Was: Re: Deconstruction patterns
In-Reply-To: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
Message-ID: <CAJq4Gi4Qw-fxVijAwquJHJ-8-GM5dP2BOLvRudkr9k+uYuSKKw@mail.gmail.com>

Some questions about the reflective design in line below.

>
>
> #### Carriers
>
> Because the matcher methods implements the matcher behavior, but a
> matcher may
> "return" multiple bindings (or failure), we must encode the bindings in
> some
> way.  For this, we use a _carrier object_.  The choice of carrier is
> largely a
> footprint/specificity tradeoff.  One could imagine a carrier class per
> matcher,
> or a carrier class per matcher descriptor, or using `Object[]` as a
> carrier for
> everything, or caching some number of common shapes (e.g, three ints and
> two
> refs).  This sort of tuning should be separate from the protocol encoded
> in the
> bytecode of the pattern method and its clients.
>
> We use a small _carrier runtime_ to decouple pattern translation from
> carrier
> selection.  (This same carrier runtime is used by string templates as
> well.)
> This allows tradeoffs in runtime characteristics (e.g., carrier per
> matcher vs
> sharing carriers across matchers, dropping carrier identity with value
> types
> later, etc) without affecting the translation. The carrier API consists
> of condy
> bootstraps like:
>
> ```
> static MethodHandle carrierFactory(MethodType matcherDescriptor) { ... }
> static MethodHandle carrierAccessor(MethodType matcherDescriptor, int
> bindingNo) { ... }
> ```
>
> The `matcherDescriptor` is a `MethodType` describing the binding types.
> The
> `carrierFactory` method returns a method handle which takes the bindings
> and
> produces a carrier object; the `carrierAccessor` method returns method
> handles
> that take the carrier object and return the corresponding binding.  To
> indicate
> success, the matcher method invokes the carrier factory method handle and
> returns the result; to indicate failure (deconstructors cannot fail, but
> other
> matchers can) the matcher method returns null.
>
> We would translate the XY deconstructor from `Point` as follows
> (pseudo-code):
>
> ```
> #100: MethodType[(II)V]
> #101: Condy[bsm=Carriers::carrierFactory, args=[#100]]
>
> final synthetic Object Point$MANGLE() {
>      aload_0
>      getfield Point::x
>      aload_0
>      getfield Point::y
>      LDC #101
>      invokevirtual MethodHandle::invoke(II)V
>      areturn
> }
> ```
>
> Constant `#100` contains a `MethodType` holding the binding descriptor;
> constant
> `#101` holds a method handle whose parameters are the parameter types of
> the
> binding descriptor and returns `Object`.
>
> At the use site, matching a deconstruction pattern is performed by
> invoking the
> matcher method on the appropriate target object, and then extracting the
> components with the carrier accessor method handles if the match is
> successful.
> (Deconstructors are total, so are always successful, but for other
> patterns,
> null is returned from the matcher method on failure to match.)
>
> #### Method names
>
> The name of the matcher method is mangled to support overloading. The JVM
> permits overloading on parameter types, but not return types (and
> overloaded
> matchers are effectively overloaded on return types.)  We take the
> approach of
> encoding the erasure of the matcher descriptor in the name of the
> pattern.  This
> has several desirable properties: it is stable (the name is derived
> solely from
> stable aspects of the declaration), for matchers with override-equivalent
> signatures (deconstructors can't be overridden, but other patterns can be),
> these map to true overrides in the translation, and valid overloads of
> matchers
> will always have distinct names.
>
> We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
> descriptor as the mangled disambiguator, which is exactly as stable as
> any other
> method descriptor derived from source declarations.
>
> #### Attributes
>
> Because patterns are methods, we can take advantage of all the
> affordances of
> methods.  We can use access bits to control accessibility; we can use the
> attributes that carry annotations, method parameter metadata, and generics
> signatures to carry information about the pattern declaration (and its
> (input)
> parameters, when we get to those kinds of matchers).  What's missing is
> the fact
> that this is a pattern implementation and not an ordinary method, and a
> place to
> put metadata for bindings.  To address the first, we can add the following
> attribute on matcher methods:
>
>      Matcher {
>          u2 name;                            // "Matcher"
>          u4 length;
>          u2 patternFlags;
>          u2 patternName;                     // UTF8
>          u2 patternDescr;                    // MethodType
>          u2 attributes_count;
>          attribute_info attributes[attributes_count];
>      }
>
> This says that "this method is a pattern".  The source name of the pattern
> declaration is reified as `patternName`, and the matcher descriptor, which
> encodes the types of the bindings, is reified as a `MethodType` in
> `patternDescr`.  The `flags` word can carry matcher-specific information
> such as
> "this matcher is a deconstructor" or "this matcher is total".
>
> A matcher method may have the usual variety of method attributes, such as
> `RuntimeInvisibleAnnotations` for annotations on the matcher declaration
> itself.
>
> If we wish to encode information about the matcher _bindings_, we do so
> with
> attributes inside the `Matcher` annotation itself.  Attributes such as
> `Signature`, `ParameterNames`, `RuntimeVisibleParameterAnnotations`,
> etc, can
> appear in a `Matcher` and are interpreted relative to the matcher
> signature or
> descriptor.  So if we had a matcher:
>
> ```
> matcher Foo(@Bar List<String> list) { ... }
> ```
>
> then the `Matcher` would contain the signature attribute corresponding to
> `(List<String>)` and a `RuntimeXxxParameterAnnotations` attribute
> describing the
> `@Bar` annotation on the first "parameter".
>
> #### Reflection
>
> Since matchers are a new kind of class member, they will need a new kind of
> reflective object, and a method that is analogous to
> `Class::getConstructors`.
> The reflective object should extend `Executable`, as all of the existing
> methods
> on `Executable` make sense for patterns (using `Object` as the return
> type.)  If
> the pattern is reflectively invoked, it returns `null` for no match, or an
> `Object[]` which is the boxing of the values in the carrier.
>
>
This surprised me slightly and I'm not sure I follow the reasoning on why
the return value would be boxed and collected into an Object[]?

The design says matcher methods return Object as an opaque descriptor for
the actual implementation carrier object.  Yet, reflection will take that
carrier object, and replace it with an equivalent Object[] resulting in
more allocations and (potentially) boxing on the return path.

I get it's a developer friendly approach but I wonder if it encourages the
wrong mental model about what matchers return?

Would it make sense to add an extra step in that process so the
java.lang.reflect.Matcher instance has a `Object[] resultToArray(Object)`
method?  It's more ceremony (yuck) but allows avoiding the array creation
and maybe the boxing on the return path for allocation-sensitive callers?

I may be overly concerned with the array creation on the return path due to
historical work with MethodHandles which sought to remove the equivalent
argument boxing / array creation from Method::invoke calls.

And speaking of MethodHandles, will there be new
MethodHandles.Lookup.findMatcher , findDeconstructor, etc methods?  Or do
you see them being looked up with the existing find* methods?


> We will then need some additional methods to describe the bindings, so the
> subtype of `Executable` has methods like `getBindings`,
> `getAnnotatedBindings`,
> `getGenericBindings`, `isDeconstructor`, `isPartial`, etc.  These
> methods will
> decode the `Matcher` attribute and its embedded attributes.
>

What does `getBindings` return?  The MethodType describing the bindings?  A
Class[] describing the types of the bindings?  Something else?

--Dan


>
> ## Summary
>
> This design borrows from previous rounds, but makes a number of
> simplifications.
>
>   - The bindings of a pattern are captured in a `MethodType`, called the
> _matcher
>     descriptor_.  The parameters of the matcher descriptor are the types
> of the
>     bindings; the return type is either `V` or the minimal type that
> will match
>     (but is not as important as the bindings.)
>   - Matchers are translated as methods whose names are derived
> deterministically
>     from the name of the matcher and the erasure of the pattern
> descriptor. These
>     are called _matcher methods_.  Matcher methods take as parameters
> the input
>     parameters of the pattern (if any), and return `Object`.
>   - The returned object is an opaque carrier.  Null means the pattern
> didn't
>     match.  A non-null value is the carrier type (from the carrier
> runtime) which
>     is derived from the pattern descriptor.
>   - Matcher methods are not directly invocable from the source language;
> they are
>     invoked indirectly through pattern matching or reflection.
>   - Generated code invokes the matcher method and interprets the
> returned value
>     according to the protocol, using MHs from the carrier runtime to
> access the
>     bindings.
>   - Matcher methods have a `Matcher` attribute, which captures
> information about
>     the matcher as a whole (is a total/partial, a deconstructor, etc) and
>     parameter-related attributes which describe the bindings.
>   - Matchers are reflected through a new subtype of `Executable`, which
> exposes
>     new methods to reflect over bindings.
>   - When invoking a matcher reflectively, the carrier is boxed to an
> Object[].
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230307/09314093/attachment-0001.htm>

From brian.goetz at oracle.com  Tue Mar  7 20:16:32 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 15:16:32 -0500
Subject: Deconstructor reflection Was: Re: Deconstruction patterns
In-Reply-To: <CAJq4Gi4Qw-fxVijAwquJHJ-8-GM5dP2BOLvRudkr9k+uYuSKKw@mail.gmail.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <CAJq4Gi4Qw-fxVijAwquJHJ-8-GM5dP2BOLvRudkr9k+uYuSKKw@mail.gmail.com>
Message-ID: <9c5e6ca2-ec31-edf3-4d10-be18d797fe18@oracle.com>



On 3/7/2023 2:51 PM, Dan Heidinga wrote:
>
>
>     #### Reflection
>
>     Since matchers are a new kind of class member, they will need a
>     new kind of
>     reflective object, and a method that is analogous to
>     `Class::getConstructors`.
>     The reflective object should extend `Executable`, as all of the
>     existing
>     methods
>     on `Executable` make sense for patterns (using `Object` as the return
>     type.)? If
>     the pattern is reflectively invoked, it returns `null` for no
>     match, or an
>     `Object[]` which is the boxing of the values in the carrier.
>
>
> This surprised me slightly and I'm not sure I follow the reasoning on 
> why the return value would be boxed and collected into an Object[]?
>
> The design says matcher methods return Object as an opaque descriptor 
> for the actual implementation carrier object.? Yet, reflection will 
> take that carrier object, and replace it with an equivalent Object[] 
> resulting in more allocations and (potentially) boxing on the return path.
>
> I get it's a developer friendly approach but I wonder if it encourages 
> the wrong mental model about what matchers return?
>
> Would it make sense to add an extra step in that process so the 
> java.lang.reflect.Matcher instance has a `Object[] 
> resultToArray(Object)` method?? It's more ceremony (yuck) but allows 
> avoiding the array creation and maybe the boxing on the return path 
> for allocation-sensitive?callers?

There's a few things here, let's try to unpack them.

I agree that the actual classfile descriptor of the synthetic method 
need not be all that related to what reflection does, though it is nice 
to minimize that difference if we can.

Reflection routinely boxes everything; if you're working reflectively, 
this is just the price of entry?? MethodHandles are a different story, 
of course, and we should be able to unreflect a Matcher to something 
that can be invoked directly.

I had been assuming that `invoke` was a method on Executable, but now 
that I look, I realize that invocation lives on the subclasses Method 
and Constructor.? So we have more latitude than I thought. Which nudges 
me a little towards

 ??? Object[] match(Object matchTarget, Object... additionalArgs)

where match failure is reflected as null.

This also nudges me a bit towards hiding matchers from getMethods() and 
friends (as we do with constructors).

> And speaking of MethodHandles, will there be new 
> MethodHandles.Lookup.findMatcher , findDeconstructor, etc methods?? Or 
> do you see them being looked up with the existing find* methods?

I had been assuming that we would use MethodHandle::findStatic for this.

>     We will then need some additional methods to describe the
>     bindings, so the
>     subtype of `Executable` has methods like `getBindings`,
>     `getAnnotatedBindings`,
>     `getGenericBindings`, `isDeconstructor`, `isPartial`, etc. These
>     methods will
>     decode the `Matcher` attribute and its embedded attributes.
>
>
> What does `getBindings` return?? The MethodType describing the 
> bindings?? A Class[] describing the types of the bindings? Something else?
>

 ??? Class[] getBindings();
 ??? Type[] getGenericBindings();
 ??? AnnotatedType[] getAnnotatedBindings();


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230307/8e8ac24c/attachment.htm>

From forax at univ-mlv.fr  Tue Mar  7 20:23:04 2023
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Tue, 7 Mar 2023 21:23:04 +0100 (CET)
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <bf9e6838-afe9-67b3-f1f7-96a3bb4b0397@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <667851490.5158494.1678180165559.JavaMail.zimbra@u-pem.fr>
 <07386d40-3cdf-17d0-e5ae-09fb79891f88@oracle.com>
 <1048964381.5570219.1678211632395.JavaMail.zimbra@u-pem.fr>
 <737a9e58-7e97-942d-8358-9ef22160b74a@oracle.com>
 <307215629.5587646.1678214447102.JavaMail.zimbra@u-pem.fr>
 <bf9e6838-afe9-67b3-f1f7-96a3bb4b0397@oracle.com>
Message-ID: <586494199.5716777.1678220584645.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Tuesday, March 7, 2023 8:42:37 PM
> Subject: Re: Deconstructor can not be overriden ? Was: Deconstruction patterns

>>> No, it is an application of LSP; if a B is-a A, I should be able to use
>>> a B wherever I currently use an A.? B doesn't necessarily get to
>>> interpose itself everywhere; it can't interpose against instanceof
>>> testing for A, or casting to A, and it can't override final methods from
>>> A.? That's what "extends" means.
>> "extends" is 3 things,
>> - subtyping (LSP),
>> - get all the members of the super class
>> - MUST override the methods that have the wrong semantics.
>>
>> Your design miss the point 3.
> 
> When the B author says "B extends A", they are consenting to the
> contract of A, in all its manifest glory.? A may have final methods.? It
> may have deconstructors.? It may have methods with specific contracts.
> It only provides a limited menu of constructors that B can delegate to.
> And B has to ensure that B is substitutible for A in all these ways, or
> it is doing a bad job of extension.
> 
...
> 
> Again, I cannot figure out what your are trying to get at, ...

My question is why a deconstructor should be always final ? Why a de-constructor can not be polymorphic ? Why it's not up to the user to choose if a de-constructor should be final or not ?

regards,
R?mi

From forax at univ-mlv.fr  Tue Mar  7 20:26:53 2023
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Tue, 7 Mar 2023 21:26:53 +0100 (CET)
Subject: Overloading of matcher method Was: Deconstruction patterns
In-Reply-To: <dced2f23-6925-8d96-85a7-df58efc5f500@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <969212125.5143862.1678179347294.JavaMail.zimbra@u-pem.fr>
 <0a144b91-13ab-6af9-bce5-f5916c05bf73@oracle.com>
 <dced2f23-6925-8d96-85a7-df58efc5f500@oracle.com>
Message-ID: <317692261.5717481.1678220813727.JavaMail.zimbra@u-pem.fr>

> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Tuesday, March 7, 2023 8:34:02 PM
> Subject: Re: Overloading of matcher method Was: Deconstruction patterns

> Remi pointed out privately that I didn't answer this question directly enough
> for his satisfaction.

> On 3/7/2023 9:20 AM, Brian Goetz wrote:

>> class Foo {
>> public Foo(String s) { ... }
>> public Foo(CharSequence seq) { ... }

>> public matcher Foo(String s) { ... }
>> public matcher Foo(CharSequence seq) { ... }
>> }

>> If i want to call Foo(CharSequence) with a String, i can use a cast, new
>> Foo((CharSequence) "foo") and the compiler selects the right overload.

>> How i can do the same to select the right matcher method inside a deconstructor
>> pattern ?
> For the constructor, you can guide (but not force) overload selection to the
> answer you want by providing more type information, which in turn will steer
> the "most specific" selection:

> new Foo((String) s)
> vs
> new Foo((CharSequence) s)

> For the deconstructor, we can do something similar with type patterns. We can
> say:

> case Foo(String s):
> vs
> case Foo(CharSequence s):

> For both of these use sites, both matchers are applicable; we're left with a
> question of which is "more specific" to the types present at the use site. As I
> have mentioned, the details are TBD, but we will draw inspiration from
> dualizing 15.12.2.5.
It's not exactly what i ask, the question is how to call the deconstructor that takes a CharSequence with a type pattern on a String. 
The inverse of 
String s = ... 
new Foo((CharSequence) s) 

should be 
case [Call the destructor Foo(CharSequence)] + Type Pattern (String) 

R?mi 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230307/59731cf8/attachment-0001.htm>

From brian.goetz at oracle.com  Tue Mar  7 20:48:11 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 15:48:11 -0500
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <586494199.5716777.1678220584645.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <667851490.5158494.1678180165559.JavaMail.zimbra@u-pem.fr>
 <07386d40-3cdf-17d0-e5ae-09fb79891f88@oracle.com>
 <1048964381.5570219.1678211632395.JavaMail.zimbra@u-pem.fr>
 <737a9e58-7e97-942d-8358-9ef22160b74a@oracle.com>
 <307215629.5587646.1678214447102.JavaMail.zimbra@u-pem.fr>
 <bf9e6838-afe9-67b3-f1f7-96a3bb4b0397@oracle.com>
 <586494199.5716777.1678220584645.JavaMail.zimbra@u-pem.fr>
Message-ID: <b6600da0-d893-1e62-b7e4-408a65d07f8a@oracle.com>


>> Again, I cannot figure out what your are trying to get at, ...
> My question is why a deconstructor should be always final ? Why a de-constructor can not be polymorphic ? Why it's not up to the user to choose if a de-constructor should be final or not ?
>

It's a fair question, but I think framing this as a "restriction" is 
misleading.? It emerges as a consequence of duality with constructors, 
and constructors have particular characteristics (such as not being 
inherited.)

If you want to declare a polymorphic, overridable, inheritable matcher, 
you can -- that's "instance matchers" (just as you can do so with 
instance methods.)? The "Object Model" doc outlines the distinction 
between constructor-like, static-like, and instance-like matchers.

It's not a "restriction" that B cannot "override" `x instanceof A` or 
`(A) x`; it's simply what these things mean. And since every B is an A, 
and that means construction flowed through an A constructor, it is 
reasonable to ask A-specific questions of a B, without necessarily 
giving B the chance to interpose.

Could we define the model differently, where all matchers are partial, 
or all matchers are instance methods?? Yes, of course we could define an 
alternate language with those characteristics. Would it be better?? 
Maybe, depending on your metric of better. Does the existence of 
alternate models mean the model proposed here is wrong?? No, of course not.

The central organizing principle here is that aggregation and 
destructuring are duals.? This means deconstructors are best understood 
as co-constructors, and static and instance patterns are co-factories.? 
By defining them as duals to the corresponding members that are already 
in the language, we guide users towards the right mental model, and set 
ourselves up for defining aggregation-deconstruction pairs with the 
desired properties -- specifically, that they form embedding-projection 
pairs.? This allows us to not only use pattern matching directly, but 
also to build things like withers on top of it.

Could we define deconstructors but make them gratutiously different from 
constructors?? Of course we could, but that's probably not helpful to 
users.

I think what's going on here is that you have a vague, alternate mental 
model of what patterns are supposed to be, and you're comparing to that 
and so any different feels like a "restriction", but unless you can put 
your model on the table, we can't compare the pros and cons.



From brian.goetz at oracle.com  Tue Mar  7 20:51:48 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 15:51:48 -0500
Subject: Overloading of matcher method Was: Deconstruction patterns
In-Reply-To: <317692261.5717481.1678220813727.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <969212125.5143862.1678179347294.JavaMail.zimbra@u-pem.fr>
 <0a144b91-13ab-6af9-bce5-f5916c05bf73@oracle.com>
 <dced2f23-6925-8d96-85a7-df58efc5f500@oracle.com>
 <317692261.5717481.1678220813727.JavaMail.zimbra@u-pem.fr>
Message-ID: <62d7114b-17c5-a470-860f-600d8c0ea91e@oracle.com>



On 3/7/2023 3:26 PM, forax at univ-mlv.fr wrote:
> It's not exactly what i ask, the question is how to call the 
> deconstructor that takes a CharSequence with a type pattern on a String.
> The inverse of
> ? String s = ...
> ? new Foo((CharSequence) s)
>
> should be
> ? case [Call the destructor Foo(CharSequence)] + Type Pattern (String)
>

Well, remember that you cannot really "name" a method either, you can 
only eliminate the ambiguity that would cause a different method to be 
selected.? And its the same with patterns.

If you really want to do this (and its not clear to me why this is so 
important) you can do this:

 ??? if (target instanceof Foo(CharSequence cs) && cs instanceof String 
s) { ... }

The first clause ensures that you select the CS pattern, and the second 
refines the binding as if you had nested `String s`.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230307/dbd045c9/attachment.htm>

From guy.steele at oracle.com  Tue Mar  7 22:42:02 2023
From: guy.steele at oracle.com (Guy Steele)
Date: Tue, 7 Mar 2023 22:42:02 +0000
Subject: Deconstruction patterns
In-Reply-To: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
Message-ID: <A9FF8557-64E8-4949-A579-DF123F98AAD7@oracle.com>

I like this topic and overall organization, BUT:

Here is my initial big-picture reaction to this document: it is much harder to read than necessary because of the chiasmus induced by use of the terms ?constructor-deconstructor pair? and ?embedding-projection? pair. Very close reading of the ?Digression? section reveals that the _deconstructor_ is the embedding and the _constructor_ is the projection. Constantly having to mentally reverse this mismatched order is gratuitously taxing.

Please do one of two things: either change ?constructor-deconstructor? to ?deconstructor-constructor? everywhere, or change ?embedding-projection? to ?projection-embedding? everywhere.

Another thing you could do in the ?Digression? section is to avoid the function-composition notation, which induces a similar reversal. You don?t really make extensive use of the result anyway (such as by  applying such a composition as a function in a formula). My advice for a Java audience would be to use a pseudo-Java method notation:

x.embed().project().equals(x)  should always be true
x.project().embed().approx(x) should be true if nothing bad happens during ?project()?

Alos, I?m not sure exactly what you mean by a ?complete partial order?: Wikipedia notes that the term has three distinct meanings in common use, and is also sometimes confused with ?compete lattice?. https://en.wikipedia.org/wiki/Complete_partial_order Furthermore it is unclear how you indeed to use that partial order to define the notion of approximate equivalence (which notorioiusly has difficulties associated with lack of transitivity).

Please remove these obstacles to readability and I will gladly take another look.  :-)

?Guy





On Mar 6, 2023, at 1:24 PM, Brian Goetz <brian.goetz at oracle.com<mailto:brian.goetz at oracle.com>> wrote:

Time to look ahead to the next installment of pattern matching: deconstruction patterns, which generalize record patterns.  This document does an end-to-end walkthrough (at a sketchy level of detail) through declaration, overloading, use, translation, and reflection of deconstruction patterns.

I would like to *not* discuss syntax at this time.  There's a lengthy discussion to be had about syntax, and we'll have that, but let's nail down model, semantics, and translation first.

As usual, I would prefer that people either (a) post a single reply addressing the totality of this sketch or (b) start _new threads_ if you want to discuss a specific aspect.  A quick "I'll just reply to this minor detail" seems to often derail the conversation in such a way that it never comes back.  If this all looks fine to you, a quick "no surprises here" will keep us from suspensefully waiting for feedback.


# Deconstruction patterns -- translation, use, and reflection

As we are wrapping up record patterns, it's time to look ahead to the next major
part of the pattern matching story -- extending the capabilities of record
patterns to all classes that want to support destructuring. Record patterns are
simply a special case of _deconstruction patterns_ or _deconstructors_, where we
derive the deconstructor API, implementation, and use from the state description
of the record.  For an arbitrary class, a deconstruction patterns will require
an explicit member declaration, with a header identifying the names and types of
the bindings and a body that extracts the bindings from the representation.

## Deconstructors

Just as constructors are special cases of methods, deconstruction patterns are
special cases of a more general notion of declared pattern, which also includes
static matchers (the dual of static methods) and instance matchers (the dual of
instance methods.)  Specifically, unlike the more general notion of matcher, a
deconstructor must be _total_; it must always match.  This document will focus
exclusively on deconstructors, and we'll come back to static and instance
matchers in due time.  (But note that some of the design choices in the simple
case of deconstructors may be constrained by the more general case.)

There are a number of choices for how we might syntactically represent a
deconstructor (or more generally, a declared pattern.)  For purposes of
illustration, this document picks one possible syntactic expression of
deconstructors, but it is premature to devolve into a syntax discussion at this
time.

```
class Point {
    final double x, y;

    public Point(double x, double y) {
        this.x = x;
        this.y = y;
    }

    public matcher Point(double x, double y) {
        x = this.x;
        y = this.y;
    }
}
```

This example illustrates two aspects of the duality between constructors and
their corresponding deconstructors.  Their APIs are duals: a constructor takes N
parameters containing the desired description of the object state and produces a
constructed object; a deconstructor starts from the constructed object and has N
bindings (outputs) that receive the desired state components. Similarly, their
implementations are duals: the body of the constructor initializes the object
representation from the description, and the body of the deconstructor extracts
the description from the representation.  A deconstructor is best understood as
a _co-constructor_.

The `Point` example above is special in two ways.  First, the internal
representation of a `Point`, and the API of the constructor and deconstructor,
are the same: `(double x, double y)`.  We can call the API implied by the
constructor and deconstructor the _external representation_, and for `Point`,
both the internal and external representations are the same. (This is one of
the requirements for being a candidate to be a record.)  And second, the
constructor is _total_; it does not reject any combinations of arguments.

Here's another version of `Point` which does not have these special aspects; it
uses the same internal representation as before, but chooses a pair of strings
as the external representation:

```
class Point2 {
    final double x, y;

    public Point2(String x, String y) {
        this.x = Double.parseDouble(x);
        this.y = Double.parseDouble(y);
    }

    public matcher Point2(String x, String y) {
        x = Double.toString(this.x);
        y = Double.toSTring(this.y);
    }
}
```

The method `Double::parseDouble` will throw `NumberFormatException` if its
argument does not describe a suitable value, so unlike the `Point` constructor,
the `Point2` constructor is partial: it will reject `new Double("foo", "bar")`.
And the internal representation is no longer the same as the external
representation.  Less obviously, there are valid string values that we can
provide to the constructor, but which cannot be represented exactly as `double`,
and which will be approximated; the string value
`"3.22222222222222222222222222222222222222"` will be approximated with the
double value `3.2222222222222223`.

This example highlights more clearly how the constructor and deconstructor form
an _embedding-projection pair_ between the internal and external
representations.  While some external representations might be invalid, and some
might result in approximation, deconstruct-then-construct is always an identity
transformation.  Indeed, the specification of `java.lang.Record` requires that
if we deconstruct a record with its accessors, and pass the resulting values
back to the constructor, we should get a new record that is `equals` to the
original.

The fact that constructor and deconstructor (and eventually, factory and static
matcher) form an embedding-projection pair is why we are able to derive
higher-level language features, such as [safer
serialization](https://openjdk.org/projects/amber/design-notes/towards-better-serialization)
and [functional transformation of immutable
objects](https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md),
from a matched set of constructor and deconstructor.

Of course, users are free to implement constructors without deconstructors, or
constructors and deconstructors whose external representations don't match up,
or even matching constructors and deconstructors that are not behaviorally dual.
But providing a matched set (or several) of constructors and deconstructors
enables reliably reversible aggregation, and allows us to mechanically derive
useful higher-level features such as withers.

#### Overloading

Just as constructors can be overloaded, deconstructors can be overloaded for the
same reason: multiple constructors can expose multiple external representations
for aggregation, and corresponding deconstructors can recover those multiple
external representations.  Any matching pair of constructor-deconstructor (and
eventually, factory-deconstructor) is a candidate for use in higher-level
features based on the embedding-projection nature of the
constructor-deconstructor pair.

Just as deconstruction is dual to construction, overloading of deconstructors is
dual to that of constructors: rather than restricting which sets of parameters
can be overloaded against each other, we do so with the bindings instead.  For
constructors of a given arity, we require that their signatures not be
override-equivalent; for deconstructors of a given arity, we require the same of
their bindings.

For a deconstructor (and declared patterns in general), we derive a _binding
signature_ (and an erased _binding descriptor_) which treats the binding list as
a parameter list.  The overload rule outlined above requires that binding
signatures for two deconstructors of the same arity not be override-equivalent.
(We will find it useful later to derive a `MethodType` for the binding
descriptor; this is a `MethodType` whose return type is `V` and whose parameter
types are the erased types of the bindings.)

#### Digression: embedding-projection pairs

Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B -> A`,
forms an _embedding-projection pair_ if `p . e` (embed then project) is an
identity function, and `e . p` (project then embed) _approximates_ the input
according to a domain-specific approximation metric (which is a complete partial
ordering on `B`.)

When applied to constructor-deconstructor pairs, this says that deconstructing
an object and then reconstructing it with the resulting bindings should result
in an equivalent object, and constructing an object from an external
representation and then deconstructing it back into that external representation
should result in an approximation of the original external representation.  (A
complete partial ordering models constructor failure as the non-terminating
bottom value, which is considered an infinitely bad approximation to
everything.)

Embedding-projection pairs have a number of desirable properties, such as the
composition of two e-p pairs is an e-p pair; this property is at the heart of
using constructor-deconstructor pairs for improved serialization and functional
transformation.

## Invoking deconstructors

We've already seen how to "invoke" deconstructors: through pattern matching.
What we've been calling "record patterns" are merely deconstruction patterns
derived mechanically from the state description, just as we do with constructors
and accessors; there is little difference between record patterns and
deconstruction patterns other than the ability to declare them explicitly.
(There is an accidental difference in the translation, in that we currently
implement record patterns by appealing to individual accessors rather than a
single deconstructor, but this may eventually converge as well.)

The use-site syntax of deconstruction bears a deliberate similarity to that of
construction; `new Point(x, y)` is deconstructed by `case Point(var x, var y)`.

#### Overload selection

In the presence of overloaded deconstructors, we need to figure out which
deconstructor a deconstruction pattern `C(P*)` is referring to. The details are
similar to overload selection for methods, except that we operate on the
bindings rather than the parameters.  We first search for _applicable matchers_,
using increasingly loose criteria (first excluding boxing, unboxing, and
varargs; then including boxing and unboxing but not varargs; and finally, all
candidates) and then selecting the most applicable.

It is tempting to try and bypass the three-phase selection process and use a
simpler notion of applicability (perhaps noting that we got this process for
compatibility with existing overload selection decisions when autoboxing and
varargs were added, and that there are few deconstructor invocations to be
compatible with yet.)  But because existing overloaded constructors use this
mechanism, and there is significant value in pairing constructors and
deconstructors, attempting to invent a simpler-but-different overload selection
mechanism for deconstructors would inevitably undermine the duality between
matching constructor-deconstructor pairs. So compatibility (this time, with
existing overloaded constructors) once again forces our hand.

The specification for overload selection is complicated significantly by poly
expressions (e.g., lambdas); fortunately, there are no "poly patterns", and so,
while the structure of JLS 15.12.2 is retained for overload selection of
deconstruction patterns, much of the detail is left behind.

## Translation

We translate patterns into synthetic methods with a `Matcher` attribute; this
method implements the matcher behavior.  The translation scheme derives from a
number of requirements, only some of which are in play for deconstructors.

The matcher method for a deconstructor is a final instance method that takes no
parameters and returns `Object`, perhaps with a special name (just as
constructors are called `<init>`.)

#### Carriers

Because the matcher methods implements the matcher behavior, but a matcher may
"return" multiple bindings (or failure), we must encode the bindings in some
way.  For this, we use a _carrier object_.  The choice of carrier is largely a
footprint/specificity tradeoff.  One could imagine a carrier class per matcher,
or a carrier class per matcher descriptor, or using `Object[]` as a carrier for
everything, or caching some number of common shapes (e.g, three ints and two
refs).  This sort of tuning should be separate from the protocol encoded in the
bytecode of the pattern method and its clients.

We use a small _carrier runtime_ to decouple pattern translation from carrier
selection.  (This same carrier runtime is used by string templates as well.)
This allows tradeoffs in runtime characteristics (e.g., carrier per matcher vs
sharing carriers across matchers, dropping carrier identity with value types
later, etc) without affecting the translation. The carrier API consists of condy
bootstraps like:

```
static MethodHandle carrierFactory(MethodType matcherDescriptor) { ... }
static MethodHandle carrierAccessor(MethodType matcherDescriptor, int bindingNo) { ... }
```

The `matcherDescriptor` is a `MethodType` describing the binding types.  The
`carrierFactory` method returns a method handle which takes the bindings and
produces a carrier object; the `carrierAccessor` method returns method handles
that take the carrier object and return the corresponding binding.  To indicate
success, the matcher method invokes the carrier factory method handle and
returns the result; to indicate failure (deconstructors cannot fail, but other
matchers can) the matcher method returns null.

We would translate the XY deconstructor from `Point` as follows (pseudo-code):

```
#100: MethodType[(II)V]
#101: Condy[bsm=Carriers::carrierFactory, args=[#100]]

final synthetic Object Point$MANGLE() {
    aload_0
    getfield Point::x
    aload_0
    getfield Point::y
    LDC #101
    invokevirtual MethodHandle::invoke(II)V
    areturn
}
```

Constant `#100` contains a `MethodType` holding the binding descriptor; constant
`#101` holds a method handle whose parameters are the parameter types of the
binding descriptor and returns `Object`.

At the use site, matching a deconstruction pattern is performed by invoking the
matcher method on the appropriate target object, and then extracting the
components with the carrier accessor method handles if the match is successful.
(Deconstructors are total, so are always successful, but for other patterns,
null is returned from the matcher method on failure to match.)

#### Method names

The name of the matcher method is mangled to support overloading. The JVM
permits overloading on parameter types, but not return types (and overloaded
matchers are effectively overloaded on return types.)  We take the approach of
encoding the erasure of the matcher descriptor in the name of the pattern.  This
has several desirable properties: it is stable (the name is derived solely from
stable aspects of the declaration), for matchers with override-equivalent
signatures (deconstructors can't be overridden, but other patterns can be),
these map to true overrides in the translation, and valid overloads of matchers
will always have distinct names.

We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
descriptor as the mangled disambiguator, which is exactly as stable as any other
method descriptor derived from source declarations.

#### Attributes

Because patterns are methods, we can take advantage of all the affordances of
methods.  We can use access bits to control accessibility; we can use the
attributes that carry annotations, method parameter metadata, and generics
signatures to carry information about the pattern declaration (and its (input)
parameters, when we get to those kinds of matchers).  What's missing is the fact
that this is a pattern implementation and not an ordinary method, and a place to
put metadata for bindings.  To address the first, we can add the following
attribute on matcher methods:

    Matcher {
        u2 name;                            // "Matcher"
        u4 length;
        u2 patternFlags;
        u2 patternName;                     // UTF8
        u2 patternDescr;                    // MethodType
        u2 attributes_count;
        attribute_info attributes[attributes_count];
    }

This says that "this method is a pattern".  The source name of the pattern
declaration is reified as `patternName`, and the matcher descriptor, which
encodes the types of the bindings, is reified as a `MethodType` in
`patternDescr`.  The `flags` word can carry matcher-specific information such as
"this matcher is a deconstructor" or "this matcher is total".

A matcher method may have the usual variety of method attributes, such as
`RuntimeInvisibleAnnotations` for annotations on the matcher declaration itself.

If we wish to encode information about the matcher _bindings_, we do so with
attributes inside the `Matcher` annotation itself.  Attributes such as
`Signature`, `ParameterNames`, `RuntimeVisibleParameterAnnotations`, etc, can
appear in a `Matcher` and are interpreted relative to the matcher signature or
descriptor.  So if we had a matcher:

```
matcher Foo(@Bar List<String> list) { ... }
```

then the `Matcher` would contain the signature attribute corresponding to
`(List<String>)` and a `RuntimeXxxParameterAnnotations` attribute describing the
`@Bar` annotation on the first "parameter".

#### Reflection

Since matchers are a new kind of class member, they will need a new kind of
reflective object, and a method that is analogous to `Class::getConstructors`.
The reflective object should extend `Executable`, as all of the existing methods
on `Executable` make sense for patterns (using `Object` as the return type.)  If
the pattern is reflectively invoked, it returns `null` for no match, or an
`Object[]` which is the boxing of the values in the carrier.

We will then need some additional methods to describe the bindings, so the
subtype of `Executable` has methods like `getBindings`, `getAnnotatedBindings`,
`getGenericBindings`, `isDeconstructor`, `isPartial`, etc.  These methods will
decode the `Matcher` attribute and its embedded attributes.

## Summary

This design borrows from previous rounds, but makes a number of simplifications.

 - The bindings of a pattern are captured in a `MethodType`, called the _matcher
   descriptor_.  The parameters of the matcher descriptor are the types of the
   bindings; the return type is either `V` or the minimal type that will match
   (but is not as important as the bindings.)
 - Matchers are translated as methods whose names are derived deterministically
   from the name of the matcher and the erasure of the pattern descriptor. These
   are called _matcher methods_.  Matcher methods take as parameters the input
   parameters of the pattern (if any), and return `Object`.
 - The returned object is an opaque carrier.  Null means the pattern didn't
   match.  A non-null value is the carrier type (from the carrier runtime) which
   is derived from the pattern descriptor.
 - Matcher methods are not directly invocable from the source language; they are
   invoked indirectly through pattern matching or reflection.
 - Generated code invokes the matcher method and interprets the returned value
   according to the protocol, using MHs from the carrier runtime to access the
   bindings.
 - Matcher methods have a `Matcher` attribute, which captures information about
   the matcher as a whole (is a total/partial, a deconstructor, etc) and
   parameter-related attributes which describe the bindings.
 - Matchers are reflected through a new subtype of `Executable`, which exposes
   new methods to reflect over bindings.
 - When invoking a matcher reflectively, the carrier is boxed to an Object[].




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230307/f0cc17d3/attachment-0001.htm>

From brian.goetz at oracle.com  Tue Mar  7 23:26:55 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Tue, 7 Mar 2023 18:26:55 -0500
Subject: Deconstruction patterns
In-Reply-To: <A9FF8557-64E8-4949-A579-DF123F98AAD7@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <A9FF8557-64E8-4949-A579-DF123F98AAD7@oracle.com>
Message-ID: <193fe032-c57f-995c-ccef-326452c6cd8a@oracle.com>



> Please do one of two things: either change ?constructor-deconstructor? 
> to ?deconstructor-constructor? everywhere, or change 
> ?embedding-projection? to ?projection-embedding? everywhere.

Changing to d-c pair is preferable, because e-p pair is already (I 
believe) the more standard term of art?

> Another thing you could do in the ?Digression? section is to avoid the 
> function-composition notation, which induces a similar reversal. You 
> don?t really make extensive use of the result anyway (such as by 
> ?applying such a composition as a function in a formula). My advice 
> for a Java audience would be to use a pseudo-Java method notation:
>
> x.embed().project().equals(x) ?should always be true
> x.project().embed().approx(x) should be true if nothing bad happens 
> during ?project()?

Yes, or just f-then-g.

> Alos, I?m not sure exactly what you mean by a ?complete partial 
> order?: Wikipedia notes that the term has three distinct meanings in 
> common use, and is also sometimes confused with ?compete lattice?. 
> https://en.wikipedia.org/wiki/Complete_partial_order?Furthermore it is 
> unclear how you indeed to use that partial order to define the notion 
> of approximate equivalence (which notorioiusly has difficulties 
> associated with lack of transitivity).

You're right that this should all go in a separate document where there 
is more room for the gory details.? These should be pretty easy changes 
to make.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230307/0725a318/attachment.htm>

From brian.goetz at oracle.com  Wed Mar  8 15:07:52 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Wed, 8 Mar 2023 10:07:52 -0500
Subject: Deconstruction patterns
In-Reply-To: <A9FF8557-64E8-4949-A579-DF123F98AAD7@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <A9FF8557-64E8-4949-A579-DF123F98AAD7@oracle.com>
Message-ID: <d1cfb279-63f3-ff5f-3b3e-74d930e86ee4@oracle.com>

Updated version, addressing Guy's concerns and fixing/clarifying a few 
other things that came up in earlier discussions.


# Deconstruction patterns -- translation, use, and reflection

As we are wrapping up record patterns, it's time to look ahead to the 
next major
part of the pattern matching story -- extending the capabilities of record
patterns to all classes that want to support destructuring. Record 
patterns are
simply a special case of _deconstruction patterns_ or _deconstructors_, 
where we
derive the deconstructor API, implementation, and use from the state 
description
of the record.? For an arbitrary class, a deconstruction patterns will 
require
an explicit member declaration, with a header identifying the names and 
types of
the bindings and a body that extracts the bindings from the representation.

## Deconstructors

Just as constructors are special cases of methods, deconstruction 
patterns are
special cases of a more general notion of declared pattern, which also 
includes
static matchers (the dual of static methods) and instance matchers (the 
dual of
instance methods.)? Specifically, unlike the more general notion of 
matcher, a
deconstructor must be _total_; it must always match.? This document will 
focus
exclusively on deconstructors, and we'll come back to static and instance
matchers in due time.? (But note that some of the design choices in the 
simple
case of deconstructors may be constrained by the more general case.)

There are a number of choices for how we might syntactically represent a
deconstructor (or more generally, a declared pattern.)? For purposes of
illustration, this document picks one possible syntactic expression of
deconstructors, but it is premature to devolve into a syntax discussion 
at this
time.

```
class Point {
 ??? final double x, y;

 ??? public Point(double x, double y) {
 ??????? this.x = x;
 ??????? this.y = y;
 ??? }

 ??? public matcher Point(double x, double y) {
 ??????? x = this.x;
 ??????? y = this.y;
 ??? }
}
```

This example illustrates two aspects of the duality between constructors and
their corresponding deconstructors.? Their APIs are duals: a constructor 
takes N
parameters containing the desired description of the object state and 
produces a
constructed object; a deconstructor starts from the constructed object 
and has N
bindings (outputs) that receive the desired state components. Similarly, 
their
implementations are duals: the body of the constructor initializes the 
object
representation from the description, and the body of the deconstructor 
extracts
the description from the representation.? A deconstructor is best 
understood as
a _co-constructor_.

The `Point` example above is special in two ways.? First, the internal
representation of a `Point`, and the API of the constructor and 
deconstructor,
are the same: `(double x, double y)`.? We can call the API implied by the
constructor and deconstructor the _external representation_, and for 
`Point`,
both the internal and external representations are the same. (This is one of
the requirements for being a candidate to be a record.)? And second, the
constructor is _total_; it does not reject any combinations of arguments.

Here's another version of `Point` which does not have these special 
aspects; it
uses the same internal representation as before, but chooses a pair of 
strings
as the external representation:

```
class Point2 {
 ??? final double x, y;

 ??? public Point2(String x, String y) {
 ??????? this.x = Double.parseDouble(x);
 ??????? this.y = Double.parseDouble(y);
 ??? }

 ??? public matcher Point2(String x, String y) {
 ??????? x = Double.toString(this.x);
 ??????? y = Double.toSTring(this.y);
 ??? }
}
```

The method `Double::parseDouble` will throw `NumberFormatException` if its
argument does not describe a suitable value, so unlike the `Point` 
constructor,
the `Point2` constructor is partial: it will reject `new Double("foo", 
"bar")`.
And the internal representation is no longer the same as the external
representation.? Less obviously, there are valid string values that we can
provide to the constructor, but which cannot be represented exactly as 
`double`,
and which will be approximated; the string value
`"3.22222222222222222222222222222222222222"` will be approximated with the
double value `3.2222222222222223`.

This example highlights more clearly how the deconstructor and 
constructor form
an _embedding-projection pair_ between the external and internal
representations.? While some external representations might be invalid, 
and some
might result in approximation, deconstruct-then-construct is always an 
identity
transformation.? (Indeed, the specification of `java.lang.Record` 
requires that
if we deconstruct a record with its accessors, and pass the resulting values
back to the constructor, we should get a new record that is `equals` to the
original.)

The fact that deconstructor and constructor (and eventually, static 
matcher and
factory) form an embedding-projection pair is why we are able to derive
higher-level language features, such as [safer
serialization](https://openjdk.org/projects/amber/design-notes/towards-better-serialization)
and [functional transformation of immutable
objects](https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md),
from a matched set of constructor and deconstructor.

Of course, users are free to implement constructors without 
deconstructors, or
constructors and deconstructors whose external representations don't 
match up,
or even matching constructors and deconstructors that are not 
behaviorally dual.
But providing a matched set (or several) of constructors and deconstructors
enables reliably reversible aggregation, and allows us to mechanically 
derive
useful higher-level features such as withers.

#### Overloading

Just as constructors can be overloaded, deconstructors can be overloaded 
for the
same reason: multiple constructors can expose multiple external 
representations
for aggregation, and corresponding deconstructors can recover those multiple
external representations.? Any matching pair of 
deconstructor-constructor (and
eventually, deconstructor-factory) is a candidate for use in higher-level
features based on the embedding-projection nature of the
deconstructor-constructor pair.

Just as deconstruction is dual to construction, overloading of 
deconstructors is
dual to that of constructors: rather than restricting which sets of 
parameters
can be overloaded against each other, we do so with the bindings 
instead.? For
constructors we require that their signatures not be 
override-equivalent; for
deconstructors, we require the same of their bindings.

For a deconstructor (and declared patterns in general), we derive a _binding
signature_ (and an erased _binding descriptor_) which treats the binding 
list as
if it were a parameter list.? The overload rule outlined above requires that
binding signatures for two deconstructors of the same arity not be
override-equivalent. (We will find it useful later to derive a 
`MethodType` for
the binding descriptor; this is a `MethodType` whose parameter types are the
erased types of the bindings.)

#### Digression: embedding-projection pairs

Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B 
-> A`,
forms an _embedding-projection pair_ if embed-then-project is an identity
function, and project-then-embed _approximates_ the input according to a
domain-specific approximation metric.? In our example which used strings
as an external representation, projection rejected some inputs (`"foo"`) and
approximated others.

When applied to deconstructor-constructor pairs, this says that 
deconstructing
an object and then reconstructing it with the resulting bindings should 
result
in an equivalent object, and constructing an object from an external
representation and then deconstructing it back into that external 
representation
should result in an approximation of the original external representation,
assuming nothing bad happens during construction such as rejecting a bad 
input.

Embedding-projection pairs have a number of desirable properties, such 
as the
composition of two e-p pairs is an e-p pair; this property is at the 
heart of
using deconstructor-constructor pairs for improved serialization and 
functional
transformation.

## Invoking deconstructors

We've already seen how to "invoke" deconstructors: through pattern matching.
What we've been calling "record patterns" are merely deconstruction patterns
derived mechanically from the state description of a record, just as we 
do with
constructors and accessors; there is little difference between record 
patterns
and deconstruction patterns other than the ability to declare them 
explicitly.
(There is an accidental difference in the translation, in that we currently
implement record patterns by appealing to individual accessors rather than a
single deconstructor, but this may eventually converge as well.)

The use-site syntax of deconstruction bears a deliberate similarity to 
that of
construction; `new Point(x, y)` is deconstructed by `case Point(var x, 
var y)`.

#### Overload selection

In the presence of overloaded deconstructors, we need to figure out which
deconstructor a deconstruction pattern `C(P*)` is referring to. The 
details are
similar to overload selection for methods, except that we operate on the
bindings rather than the parameters.? We first search for _applicable 
matchers_,
using increasingly loose criteria (first excluding boxing, unboxing, and
varargs; then including boxing and unboxing but not varargs; and 
finally, all
candidates) and then selecting the most applicable.

It is tempting to try and bypass the three-phase selection process and use a
simpler notion of applicability (perhaps noting that we got this process for
compatibility with existing overload selection decisions when autoboxing and
varargs were added, and that there are few deconstructor invocations to be
compatible with yet.)? But because existing overloaded constructors use this
mechanism, and there is significant value in pairing constructors and
deconstructors, attempting to invent a simpler-but-different overload 
selection
mechanism for deconstructors would inevitably undermine the duality between
matching deconstructor-constructor pairs. So compatibility (this time, with
existing overloaded constructors) once again forces our hand.

The specification for overload selection is complicated significantly by 
poly
expressions (e.g., lambdas); fortunately, there are no "poly patterns", 
and so,
while the structure of JLS 15.12.2 is retained for overload selection of
deconstruction patterns, much of the detail is left behind.

## Translation

We translate patterns into synthetic methods with a `Matcher` attribute; 
this
method implements the matcher behavior.? The translation scheme derives 
from a
number of requirements, only some of which are in play for deconstructors.

The matcher method for a deconstructor is a final instance method that 
takes no
parameters and returns `Object`, perhaps with a special name (just as
constructors are called `<init>`.)

#### Carriers

Because the matcher methods implements the matcher behavior, but a 
matcher may
"return" multiple bindings (or failure), we must encode the bindings in some
way.? For this, we use a _carrier object_.? The choice of carrier is 
largely a
footprint/specificity tradeoff.? One could imagine a carrier class per 
matcher,
or a carrier class per matcher descriptor, or using `Object[]` as a 
carrier for
everything, or caching some number of common shapes (e.g, three ints and two
refs).? This sort of tuning should be separate from the protocol encoded 
in the
bytecode of the pattern method and its clients.

We use a small _carrier runtime_ to decouple pattern translation from 
carrier
selection.? (This same carrier runtime is used by string templates as well.)
This allows tradeoffs in runtime characteristics (e.g., carrier per 
matcher vs
sharing carriers across matchers, dropping carrier identity with value types
later, etc) without affecting the translation. The carrier API consists 
of condy
bootstraps like:

```
static MethodHandle carrierFactory(METADATA,
 ?????????????????????????????????? MethodType matcherDescriptor) { ... }
static MethodHandle carrierAccessor(METADATA,
 ??????????????????????????????????? MethodType matcherDescriptor,
 ??????????????????????????????????? int bindingNo) { ... }
```

where `METADATA` are the standard condy metadata arguments (lookup, 
name, type.)
The `matcherDescriptor` is a `MethodType` describing the binding types.? The
`carrierFactory` method returns a method handle which takes the bindings and
produces a carrier object; the `carrierAccessor` method returns method 
handles
that take the carrier object and return the corresponding binding.? To 
indicate
success, the matcher method invokes the carrier factory method handle and
returns the result; to indicate failure (deconstructors cannot fail, but 
other
matchers can) the matcher method returns null.

We would translate the XY deconstructor from `Point` as follows
(pseudo-bytecode):

```
#100: MethodType[(II)V]
#101: Condy[bsm=Carriers::carrierFactory, args=[#100]]

final synthetic Object Point$MANGLE() {
 ??? aload_0
 ??? getfield Point::x
 ??? aload_0
 ??? getfield Point::y
 ??? LDC #101
 ??? invokevirtual MethodHandle::invoke(II)Object
 ??? areturn
}
```

Constant `#100` contains a `MethodType` holding the binding descriptor; 
constant
`#101` holds a method handle whose parameters are the parameter types of the
binding descriptor and returns `Object`.

At the use site, matching a deconstruction pattern is performed by 
invoking the
matcher method on the appropriate target object, and then extracting the
components with the carrier accessor method handles if the match is 
successful.
(Deconstructors are total, so are always successful, but for other patterns,
null is returned from the matcher method on failure to match.)

#### Method names

The name of the matcher method is mangled to support overloading. The JVM
permits overloading on parameter types, but not return types (and overloaded
matchers are effectively overloaded on return types.)? We take the 
approach of
encoding the erasure of the matcher descriptor in the name of the 
pattern.? This
has several desirable properties: it is stable (the name is derived 
solely from
stable aspects of the declaration), for matchers with override-equivalent
signatures (deconstructors can't be overridden, but other patterns can be),
these map to true overrides in the translation, and valid overloads of 
matchers
will always have distinct names.

We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
descriptor as the mangled disambiguator, which is exactly as stable as 
any other
method descriptor derived from source declarations.

#### Attributes

Because patterns are methods, we can take advantage of all the 
affordances of
methods.? We can use access bits to control accessibility; we can use the
attributes that carry annotations, method parameter metadata, and generics
signatures to carry information about the pattern declaration (and its 
(input)
parameters, when we get to those kinds of matchers).? What's missing is 
the fact
that this is a pattern implementation and not an ordinary method, and a 
place to
put metadata for bindings.? To address the first, we can add the following
attribute on matcher methods:

 ??? Matcher {
 ??????? u2 name;??????????????????????????? // "Matcher"
 ??????? u4 length;
 ??????? u2 patternFlags;
 ??????? u2 patternName;???????????????????? // UTF8
 ??????? u2 patternDescr;??????????????????? // MethodType
 ??????? u2 attributes_count;
 ??????? attribute_info attributes[attributes_count];
 ??? }

This says that "this method is a pattern".? The source name of the pattern
declaration is reified as `patternName`, and the matcher descriptor, which
encodes the types of the bindings, is reified as a `MethodType` in
`patternDescr`.? The `flags` word can carry matcher-specific information 
such as
"this matcher is a deconstructor" or "this matcher is total".

A matcher method may have the usual variety of method attributes, such as
`RuntimeInvisibleAnnotations` for annotations on the matcher declaration 
itself.

If we wish to encode information about the matcher _bindings_, we do so with
attributes inside the `Matcher` annotation itself.? Attributes such as
`Signature`, `ParameterNames`, `RuntimeVisibleParameterAnnotations`, 
etc, can
appear in a `Matcher` and are interpreted relative to the matcher 
signature or
descriptor.? So if we had a matcher:

```
matcher Foo(@Bar List<String> list) { ... }
```

then the `Matcher` would contain a signature attribute corresponding to
`(List<String>)` and a `RuntimeXxxParameterAnnotations` attribute 
describing the
`@Bar` annotation on the first "parameter".

#### Reflection

Since matchers are a new kind of class member, they will need a new kind of
reflective object, and a method that is analogous to 
`Class::getConstructors`.
The reflective object for matchers should extend `Executable`, as all of the
existing methods on `Executable` make sense for patterns.? If the pattern is
reflectively invoked, it returns `null` for no match, or an `Object[]` 
which is
the boxing of the values in the carrier.

We will then need some additional methods to describe the bindings, so the
subtype of `Executable` has methods like `getBindings`, 
`getAnnotatedBindings`,
`getGenericBindings`, `isDeconstructor`, `isPartial`, etc.? These 
methods will
decode the `Matcher` attribute and its embedded attributes.

## Summary

This design borrows from previous rounds, but makes a number of 
simplifications.

 ?- The bindings of a pattern are captured in a `MethodType`, called the 
_matcher
 ?? descriptor_.? The parameters of the matcher descriptor are the types 
of the
 ?? bindings; the return type is either `V` or the minimal type that 
will match
 ?? (but is not as important as the bindings.)
 ?- Matchers are translated as methods whose names are derived 
deterministically
 ?? from the name of the matcher and the erasure of the pattern 
descriptor. These
 ?? are called _matcher methods_.? Matcher methods take as parameters 
the input
 ?? parameters of the pattern (if any), and return `Object`.
 ?- The returned object is an opaque carrier.? Null means the pattern didn't
 ?? match.? A non-null value is the carrier type (from the carrier 
runtime) which
 ?? is derived from the pattern descriptor.
 ?- Matcher methods are not directly invocable from the source language; 
they are
 ?? invoked indirectly through pattern matching or reflection.
 ?- Generated code invokes the matcher method and interprets the 
returned value
 ?? according to the protocol, using MHs from the carrier runtime to 
access the
 ?? bindings.
 ?- Matcher methods have a `Matcher` attribute, which captures 
information about
 ?? the matcher as a whole (is a total/partial, a deconstructor, etc) and
 ?? parameter-related attributes which describe the bindings.
 ?- Matchers are reflected through a new subtype of `Executable`, which 
exposes
 ?? new methods to reflect over bindings.
 ?- When invoking a matcher reflectively, the carrier is boxed to an 
`Object[]`.




From forax at univ-mlv.fr  Wed Mar  8 16:16:30 2023
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Wed, 8 Mar 2023 17:16:30 +0100 (CET)
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <b6600da0-d893-1e62-b7e4-408a65d07f8a@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <07386d40-3cdf-17d0-e5ae-09fb79891f88@oracle.com>
 <1048964381.5570219.1678211632395.JavaMail.zimbra@u-pem.fr>
 <737a9e58-7e97-942d-8358-9ef22160b74a@oracle.com>
 <307215629.5587646.1678214447102.JavaMail.zimbra@u-pem.fr>
 <bf9e6838-afe9-67b3-f1f7-96a3bb4b0397@oracle.com>
 <586494199.5716777.1678220584645.JavaMail.zimbra@u-pem.fr>
 <b6600da0-d893-1e62-b7e4-408a65d07f8a@oracle.com>
Message-ID: <1054646865.6547919.1678292190677.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Tuesday, March 7, 2023 9:48:11 PM
> Subject: Re: Deconstructor can not be overriden ? Was: Deconstruction patterns

>>> Again, I cannot figure out what your are trying to get at, ...
>> My question is why a deconstructor should be always final ? Why a de-constructor
>> can not be polymorphic ? Why it's not up to the user to choose if a
>> de-constructor should be final or not ?
>>
> 
> It's a fair question, but I think framing this as a "restriction" is
> misleading.? It emerges as a consequence of duality with constructors,
> and constructors have particular characteristics (such as not being
> inherited.)
> 
> If you want to declare a polymorphic, overridable, inheritable matcher,
> you can -- that's "instance matchers" (just as you can do so with
> instance methods.)? The "Object Model" doc outlines the distinction
> between constructor-like, static-like, and instance-like matchers.
> 
> It's not a "restriction" that B cannot "override" `x instanceof A` or
> `(A) x`; it's simply what these things mean. And since every B is an A,
> and that means construction flowed through an A constructor, it is
> reasonable to ask A-specific questions of a B, without necessarily
> giving B the chance to interpose.


The duality you are talking about is this one

  arguments + constructor => instance
 and
  instance + deconstructor => arguments

If we add types + liskov substitution principle, we get

  A a = new B(arguments...);
 and it's dual
  a instanceof A(arguments...)

As you can see, for the constructors, the arguments are the one of B, so for the deconstructor, it has to be the deconstructor of B, otherwise it only works if the constructor of B do not do any transformation of the arguments when calling the constructor of A.

> 
> The central organizing principle here is that aggregation and
> destructuring are duals.? This means deconstructors are best understood
> as co-constructors, and static and instance patterns are co-factories.
> By defining them as duals to the corresponding members that are already
> in the language, we guide users towards the right mental model, and set
> ourselves up for defining aggregation-deconstruction pairs with the
> desired properties -- specifically, that they form embedding-projection
> pairs.? This allows us to not only use pattern matching directly, but
> also to build things like withers on top of it.
> 
> Could we define deconstructors but make them gratutiously different from
> constructors?? Of course we could, but that's probably not helpful to
> users.

Duality does not means having the same characteristics, + and - are dual, but only + is associative.
As explain above, the deconstructor has to be polymorphic otherwise you go not get the right version of the arguments.

Also if we want to support deconstructors on interface, we need a way for implementations of that interface to choose how to implement such deconstructors.

regards,
R?mi

From brian.goetz at oracle.com  Wed Mar  8 16:35:37 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Wed, 8 Mar 2023 11:35:37 -0500
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <1054646865.6547919.1678292190677.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <07386d40-3cdf-17d0-e5ae-09fb79891f88@oracle.com>
 <1048964381.5570219.1678211632395.JavaMail.zimbra@u-pem.fr>
 <737a9e58-7e97-942d-8358-9ef22160b74a@oracle.com>
 <307215629.5587646.1678214447102.JavaMail.zimbra@u-pem.fr>
 <bf9e6838-afe9-67b3-f1f7-96a3bb4b0397@oracle.com>
 <586494199.5716777.1678220584645.JavaMail.zimbra@u-pem.fr>
 <b6600da0-d893-1e62-b7e4-408a65d07f8a@oracle.com>
 <1054646865.6547919.1678292190677.JavaMail.zimbra@u-pem.fr>
Message-ID: <9b1740c4-be3b-bbb8-e9d3-7157fb7ec812@oracle.com>


> The duality you are talking about is this one
>
>    arguments + constructor => instance
>   and
>    instance + deconstructor => arguments
>
> If we add types + liskov substitution principle, we get
>
>    A a = new B(arguments...);
>   and it's dual
>    a instanceof A(arguments...)
>
> As you can see, for the constructors, the arguments are the one of B, so for the deconstructor, it has to be the deconstructor of B, otherwise it only works if the constructor of B do not do any transformation of the arguments when calling the constructor of A.

No, not correct.? If B extends A, when we instantiate `new B(b-args)`, 
we are also _implicitly_ instantiating `new A(a-args)`, because a 
B-constructor must delegate to an A-constructor. Deconstructing with a B 
deconstructor recovers the b-args; deconstructing with an A 
deconstructor recovers the a-args.

In any case, I think you need to back up -- way up -- and figure out 
what your point really is here; this seems to have devolved into 
unhelpful quibbling over terminology.

> Also if we want to support deconstructors on interface, we need a way for implementations of that interface to choose how to implement such deconstructors.

Of course, but the expression of a dtor in an interface will be no 
different than in a class.? (For this to work, the interface has to 
actually be able to implement it, which means the interface needs to 
have methods to access the required state.? This is true for interfaces 
like Map.Entry, which have getKey/getValue methods, but not true for all 
interfaces.)

Is there a specific problem here that you are worried about?



From forax at univ-mlv.fr  Wed Mar  8 16:59:38 2023
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Wed, 8 Mar 2023 17:59:38 +0100 (CET)
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <9b1740c4-be3b-bbb8-e9d3-7157fb7ec812@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <737a9e58-7e97-942d-8358-9ef22160b74a@oracle.com>
 <307215629.5587646.1678214447102.JavaMail.zimbra@u-pem.fr>
 <bf9e6838-afe9-67b3-f1f7-96a3bb4b0397@oracle.com>
 <586494199.5716777.1678220584645.JavaMail.zimbra@u-pem.fr>
 <b6600da0-d893-1e62-b7e4-408a65d07f8a@oracle.com>
 <1054646865.6547919.1678292190677.JavaMail.zimbra@u-pem.fr>
 <9b1740c4-be3b-bbb8-e9d3-7157fb7ec812@oracle.com>
Message-ID: <632815197.6575991.1678294778823.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Wednesday, March 8, 2023 5:35:37 PM
> Subject: Re: Deconstructor can not be overriden ? Was: Deconstruction patterns

>> The duality you are talking about is this one
>>
>>    arguments + constructor => instance
>>   and
>>    instance + deconstructor => arguments
>>
>> If we add types + liskov substitution principle, we get
>>
>>    A a = new B(arguments...);
>>   and it's dual
>>    a instanceof A(arguments...)
>>
>> As you can see, for the constructors, the arguments are the one of B, so for the
>> deconstructor, it has to be the deconstructor of B, otherwise it only works if
>> the constructor of B do not do any transformation of the arguments when calling
>> the constructor of A.
> 
> No, not correct.? If B extends A, when we instantiate `new B(b-args)`,
> we are also _implicitly_ instantiating `new A(a-args)`, because a
> B-constructor must delegate to an A-constructor. Deconstructing with a B
> deconstructor recovers the b-args; deconstructing with an A
> deconstructor recovers the a-args.
> 

The problem with this semantics is that this is worst that with accessors, because at least you can override an accessor.

It means that once a deconstructor is provided, subclasses can not change it.

If know that a lot of features introduced since Java 5 does not support inheritance but maybe you should be more public about the fact that inheritance is a second class citizen in Java. Personally, i'm fine with that but i want to be sure it's not an unintended side effect.

> 
>> Also if we want to support deconstructors on interface, we need a way for
>> implementations of that interface to choose how to implement such
>> deconstructors.
> 
> Of course, but the expression of a dtor in an interface will be no
> different than in a class.? (For this to work, the interface has to
> actually be able to implement it, which means the interface needs to
> have methods to access the required state.? This is true for interfaces
> like Map.Entry, which have getKey/getValue methods, but not true for all
> interfaces.)
> 
> Is there a specific problem here that you are worried about?

A specific implementation of Map.Entry not being able to fix the default implementation.

By example, if my Map is thread-safe and implemented with two arrays and a lock, i will want the deconstructor to be able to synchronize on the lock, something the default implementation will not do. 

regards,
R?mi

From heidinga at redhat.com  Wed Mar  8 17:05:20 2023
From: heidinga at redhat.com (Dan Heidinga)
Date: Wed, 8 Mar 2023 12:05:20 -0500
Subject: Deconstructor reflection Was: Re: Deconstruction patterns
In-Reply-To: <9c5e6ca2-ec31-edf3-4d10-be18d797fe18@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <CAJq4Gi4Qw-fxVijAwquJHJ-8-GM5dP2BOLvRudkr9k+uYuSKKw@mail.gmail.com>
 <9c5e6ca2-ec31-edf3-4d10-be18d797fe18@oracle.com>
Message-ID: <CAJq4Gi52PYVk_Akkm3Aio33vVW3bu+uC8aSB54Zvn9f1jFqUVg@mail.gmail.com>

Thanks for the response.  This seems reasonable though I still have an
aversion to the Object[] return type for the `match` method though it fits
with the way reflection works today.  I hate wasting the Carrier runtime's
benefits by immediately boxing and collecting to the array for reflection
but maybe anything else is trying to fix the sins of the past?

--Dan

On Tue, Mar 7, 2023 at 3:16?PM Brian Goetz <brian.goetz at oracle.com> wrote:

>
>
> On 3/7/2023 2:51 PM, Dan Heidinga wrote:
>
>
>> #### Reflection
>>
>> Since matchers are a new kind of class member, they will need a new kind
>> of
>> reflective object, and a method that is analogous to
>> `Class::getConstructors`.
>> The reflective object should extend `Executable`, as all of the existing
>> methods
>> on `Executable` make sense for patterns (using `Object` as the return
>> type.)  If
>> the pattern is reflectively invoked, it returns `null` for no match, or an
>> `Object[]` which is the boxing of the values in the carrier.
>>
>>
> This surprised me slightly and I'm not sure I follow the reasoning on why
> the return value would be boxed and collected into an Object[]?
>
> The design says matcher methods return Object as an opaque descriptor for
> the actual implementation carrier object.  Yet, reflection will take that
> carrier object, and replace it with an equivalent Object[] resulting in
> more allocations and (potentially) boxing on the return path.
>
> I get it's a developer friendly approach but I wonder if it encourages the
> wrong mental model about what matchers return?
>
> Would it make sense to add an extra step in that process so the
> java.lang.reflect.Matcher instance has a `Object[] resultToArray(Object)`
> method?  It's more ceremony (yuck) but allows avoiding the array creation
> and maybe the boxing on the return path for allocation-sensitive callers?
>
>
> There's a few things here, let's try to unpack them.
>
> I agree that the actual classfile descriptor of the synthetic method need
> not be all that related to what reflection does, though it is nice to
> minimize that difference if we can.
>
> Reflection routinely boxes everything; if you're working reflectively,
> this is just the price of entry?  MethodHandles are a different story, of
> course, and we should be able to unreflect a Matcher to something that can
> be invoked directly.
>
> I had been assuming that `invoke` was a method on Executable, but now that
> I look, I realize that invocation lives on the subclasses Method and
> Constructor.  So we have more latitude than I thought.  Which nudges me a
> little towards
>
>     Object[] match(Object matchTarget, Object... additionalArgs)
>
> where match failure is reflected as null.
>
> This also nudges me a bit towards hiding matchers from getMethods() and
> friends (as we do with constructors).
>
> And speaking of MethodHandles, will there be new
> MethodHandles.Lookup.findMatcher , findDeconstructor, etc methods?  Or do
> you see them being looked up with the existing find* methods?
>
>
> I had been assuming that we would use MethodHandle::findStatic for this.
>
>
>
>> We will then need some additional methods to describe the bindings, so the
>> subtype of `Executable` has methods like `getBindings`,
>> `getAnnotatedBindings`,
>> `getGenericBindings`, `isDeconstructor`, `isPartial`, etc.  These
>> methods will
>> decode the `Matcher` attribute and its embedded attributes.
>>
>
> What does `getBindings` return?  The MethodType describing the bindings?
> A Class[] describing the types of the bindings?  Something else?
>
>
>     Class[] getBindings();
>     Type[] getGenericBindings();
>     AnnotatedType[] getAnnotatedBindings();
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230308/da92c8f4/attachment.htm>

From brian.goetz at oracle.com  Wed Mar  8 17:18:40 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Wed, 8 Mar 2023 12:18:40 -0500
Subject: Deconstructor reflection Was: Re: Deconstruction patterns
In-Reply-To: <CAJq4Gi52PYVk_Akkm3Aio33vVW3bu+uC8aSB54Zvn9f1jFqUVg@mail.gmail.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <CAJq4Gi4Qw-fxVijAwquJHJ-8-GM5dP2BOLvRudkr9k+uYuSKKw@mail.gmail.com>
 <9c5e6ca2-ec31-edf3-4d10-be18d797fe18@oracle.com>
 <CAJq4Gi52PYVk_Akkm3Aio33vVW3bu+uC8aSB54Zvn9f1jFqUVg@mail.gmail.com>
Message-ID: <354dafb4-56c6-5d52-0fb9-649ea2dedbe2@oracle.com>

That's sort of where I was coming from.? For example, when we added 
Class::getRecordComponents (which, btw, should really be a pattern), it 
returns RecordComponent[] instead of List<RecordComponent>, because 
that's how reflection rolls.? The benefit of using List (which are 
clearly "better" than arrays) is lost by the inconsistency of "some 
methods return arrays, some return lists."

On 3/8/2023 12:05 PM, Dan Heidinga wrote:
> Thanks for the response.? This seems reasonable though I still have an 
> aversion to the Object[] return type for the `match` method though it 
> fits with the way reflection works today.? I hate wasting the Carrier 
> runtime's benefits by immediately boxing and collecting to the array 
> for reflection but maybe anything else is trying to fix the sins of 
> the past?
>
> --Dan
>
> On Tue, Mar 7, 2023 at 3:16?PM Brian Goetz <brian.goetz at oracle.com> wrote:
>
>
>
>     On 3/7/2023 2:51 PM, Dan Heidinga wrote:
>>
>>
>>         #### Reflection
>>
>>         Since matchers are a new kind of class member, they will need
>>         a new kind of
>>         reflective object, and a method that is analogous to
>>         `Class::getConstructors`.
>>         The reflective object should extend `Executable`, as all of
>>         the existing
>>         methods
>>         on `Executable` make sense for patterns (using `Object` as
>>         the return
>>         type.)? If
>>         the pattern is reflectively invoked, it returns `null` for no
>>         match, or an
>>         `Object[]` which is the boxing of the values in the carrier.
>>
>>
>>     This surprised me slightly and I'm not sure I follow the
>>     reasoning on why the return value would be boxed and collected
>>     into an Object[]?
>>
>>     The design says matcher methods return Object as an opaque
>>     descriptor for the actual implementation carrier object.? Yet,
>>     reflection will take that carrier object, and replace it with an
>>     equivalent Object[] resulting in more allocations and
>>     (potentially) boxing on the return path.
>>
>>     I get it's a developer friendly approach but I wonder if it
>>     encourages the wrong mental model about what matchers return?
>>
>>     Would it make sense to add an extra step in that process so the
>>     java.lang.reflect.Matcher instance has a `Object[]
>>     resultToArray(Object)` method?? It's more ceremony (yuck) but
>>     allows avoiding the array creation and maybe the boxing on the
>>     return path for allocation-sensitive?callers?
>
>     There's a few things here, let's try to unpack them.
>
>     I agree that the actual classfile descriptor of the synthetic
>     method need not be all that related to what reflection does,
>     though it is nice to minimize that difference if we can.
>
>     Reflection routinely boxes everything; if you're working
>     reflectively, this is just the price of entry? MethodHandles are a
>     different story, of course, and we should be able to unreflect a
>     Matcher to something that can be invoked directly.
>
>     I had been assuming that `invoke` was a method on Executable, but
>     now that I look, I realize that invocation lives on the subclasses
>     Method and Constructor.? So we have more latitude than I thought.?
>     Which nudges me a little towards
>
>     ??? Object[] match(Object matchTarget, Object... additionalArgs)
>
>     where match failure is reflected as null.
>
>     This also nudges me a bit towards hiding matchers from
>     getMethods() and friends (as we do with constructors).
>
>>     And speaking of MethodHandles, will there be new
>>     MethodHandles.Lookup.findMatcher , findDeconstructor, etc
>>     methods?? Or do you see them being looked up with the existing
>>     find* methods?
>
>     I had been assuming that we would use MethodHandle::findStatic for
>     this.
>
>>         We will then need some additional methods to describe the
>>         bindings, so the
>>         subtype of `Executable` has methods like `getBindings`,
>>         `getAnnotatedBindings`,
>>         `getGenericBindings`, `isDeconstructor`, `isPartial`, etc.?
>>         These
>>         methods will
>>         decode the `Matcher` attribute and its embedded attributes.
>>
>>
>>     What does `getBindings` return?? The MethodType describing the
>>     bindings?? A Class[] describing the types of the bindings??
>>     Something else?
>>
>
>     ??? Class[] getBindings();
>     ??? Type[] getGenericBindings();
>     ??? AnnotatedType[] getAnnotatedBindings();
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230308/9882b2bd/attachment-0001.htm>

From brian.goetz at oracle.com  Wed Mar  8 17:21:13 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Wed, 8 Mar 2023 12:21:13 -0500
Subject: Deconstructor can not be overriden ? Was: Deconstruction patterns
In-Reply-To: <632815197.6575991.1678294778823.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <737a9e58-7e97-942d-8358-9ef22160b74a@oracle.com>
 <307215629.5587646.1678214447102.JavaMail.zimbra@u-pem.fr>
 <bf9e6838-afe9-67b3-f1f7-96a3bb4b0397@oracle.com>
 <586494199.5716777.1678220584645.JavaMail.zimbra@u-pem.fr>
 <b6600da0-d893-1e62-b7e4-408a65d07f8a@oracle.com>
 <1054646865.6547919.1678292190677.JavaMail.zimbra@u-pem.fr>
 <9b1740c4-be3b-bbb8-e9d3-7157fb7ec812@oracle.com>
 <632815197.6575991.1678294778823.JavaMail.zimbra@u-pem.fr>
Message-ID: <dd46b8c3-32f0-82d6-3f15-ba5b5f2f0823@oracle.com>


> It means that once a deconstructor is provided, subclasses can not change it.

Correct.? I don't see this as a problem, or as inconsistent with 
inheritance.

> If know that a lot of features introduced since Java 5 does not support inheritance but maybe you should be more public about the fact that inheritance is a second class citizen in Java. Personally, i'm fine with that but i want to be sure it's not an unintended side effect.

Confirmed: that A's dtor is A's exclusively to define is not an 
unintended side-effect.? "Second class citizen" or "inheritance is dead" 
is unhelpful rhetoric, though.? This is more like `instanceof` or cast; 
B can't override these operators either.? B also can't override the 
parameterization of A's supertypes, but that doesn't bother us either; 
if A extends Foo<A>, we can't make B extends Foo<B>.

>
>> Is there a specific problem here that you are worried about?
> A specific implementation of Map.Entry not being able to fix the default implementation.

Map.Entry has two choices:

 ?- declare a dtor, in which case this is the final word on the subject, or
 ?- declare an abstract instance pattern (perhaps with a default), which 
would allow overriding.

The two are different, and will be used in different situations. Part of 
the API design is choosing between them.? But also, note that the 
matcher implementation can only use functionality that is already 
present in the interface.? This is significant, because:

> By example, if my Map is thread-safe and implemented with two arrays and a lock, i will want the deconstructor to be able to synchronize on the lock, something the default implementation will not do.

Such a Map already has this problem, as Map.Entry already contains 
getKey and getValue methods which can be used independently.? If they 
are allowed to return inconsistent results, it doesn't matter what the 
dtor does, since users are free to go to the source and get an 
unsynchronized view using getKey/getValue.



From paul.sandoz at oracle.com  Wed Mar  8 17:25:42 2023
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Wed, 8 Mar 2023 17:25:42 +0000
Subject: Deconstructor reflection Was: Re: Deconstruction patterns
In-Reply-To: <354dafb4-56c6-5d52-0fb9-649ea2dedbe2@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <CAJq4Gi4Qw-fxVijAwquJHJ-8-GM5dP2BOLvRudkr9k+uYuSKKw@mail.gmail.com>
 <9c5e6ca2-ec31-edf3-4d10-be18d797fe18@oracle.com>
 <CAJq4Gi52PYVk_Akkm3Aio33vVW3bu+uC8aSB54Zvn9f1jFqUVg@mail.gmail.com>
 <354dafb4-56c6-5d52-0fb9-649ea2dedbe2@oracle.com>
Message-ID: <2D120C56-17ED-44F2-B8AA-434BFFEB3685@oracle.com>

I think the current java.lang.reflect model is the path of lease resistance. (Bindings are the dual of method arguments which are also bundled the same way.)

We can leverage the runtime benefits of the carrier with method handles.

Paul.

> On Mar 8, 2023, at 9:18 AM, Brian Goetz <brian.goetz at oracle.com> wrote:
> 
> That's sort of where I was coming from.  For example, when we added Class::getRecordComponents (which, btw, should really be a pattern), it returns RecordComponent[] instead of List<RecordComponent>, because that's how reflection rolls.  The benefit of using List (which are clearly "better" than arrays) is lost by the inconsistency of "some methods return arrays, some return lists."  
> 
> On 3/8/2023 12:05 PM, Dan Heidinga wrote:
>> Thanks for the response.  This seems reasonable though I still have an aversion to the Object[] return type for the `match` method though it fits with the way reflection works today.  I hate wasting the Carrier runtime's benefits by immediately boxing and collecting to the array for reflection but maybe anything else is trying to fix the sins of the past?
>> 
>> --Dan
>> 
>> On Tue, Mar 7, 2023 at 3:16?PM Brian Goetz <brian.goetz at oracle.com> wrote:
>> 
>> 
>> On 3/7/2023 2:51 PM, Dan Heidinga wrote:
>>> 
>>> #### Reflection
>>> 
>>> Since matchers are a new kind of class member, they will need a new kind of
>>> reflective object, and a method that is analogous to 
>>> `Class::getConstructors`.
>>> The reflective object should extend `Executable`, as all of the existing 
>>> methods
>>> on `Executable` make sense for patterns (using `Object` as the return 
>>> type.)  If
>>> the pattern is reflectively invoked, it returns `null` for no match, or an
>>> `Object[]` which is the boxing of the values in the carrier.
>>> 
>>> 
>>> This surprised me slightly and I'm not sure I follow the reasoning on why the return value would be boxed and collected into an Object[]?
>>> 
>>> The design says matcher methods return Object as an opaque descriptor for the actual implementation carrier object.  Yet, reflection will take that carrier object, and replace it with an equivalent Object[] resulting in more allocations and (potentially) boxing on the return path.
>>> 
>>> I get it's a developer friendly approach but I wonder if it encourages the wrong mental model about what matchers return?
>>> 
>>> Would it make sense to add an extra step in that process so the java.lang.reflect.Matcher instance has a `Object[] resultToArray(Object)` method?  It's more ceremony (yuck) but allows avoiding the array creation and maybe the boxing on the return path for allocation-sensitive callers?
>> 
>> There's a few things here, let's try to unpack them.  
>> 
>> I agree that the actual classfile descriptor of the synthetic method need not be all that related to what reflection does, though it is nice to minimize that difference if we can.  
>> 
>> Reflection routinely boxes everything; if you're working reflectively, this is just the price of entry?  MethodHandles are a different story, of course, and we should be able to unreflect a Matcher to something that can be invoked directly.
>> 
>> I had been assuming that `invoke` was a method on Executable, but now that I look, I realize that invocation lives on the subclasses Method and Constructor.  So we have more latitude than I thought.  Which nudges me a little towards 
>> 
>>     Object[] match(Object matchTarget, Object... additionalArgs)
>> 
>> where match failure is reflected as null.
>> 
>> This also nudges me a bit towards hiding matchers from getMethods() and friends (as we do with constructors).  
>> 
>>> And speaking of MethodHandles, will there be new MethodHandles.Lookup.findMatcher , findDeconstructor, etc methods?  Or do you see them being looked up with the existing find* methods?
>> 
>> I had been assuming that we would use MethodHandle::findStatic for this.  
>> 
>>>  
>>> We will then need some additional methods to describe the bindings, so the
>>> subtype of `Executable` has methods like `getBindings`, 
>>> `getAnnotatedBindings`,
>>> `getGenericBindings`, `isDeconstructor`, `isPartial`, etc.  These 
>>> methods will
>>> decode the `Matcher` attribute and its embedded attributes.
>>> 
>>> What does `getBindings` return?  The MethodType describing the bindings?  A Class[] describing the types of the bindings?  Something else?
>>> 
>> 
>>     Class[] getBindings();
>>     Type[] getGenericBindings();
>>     AnnotatedType[] getAnnotatedBindings();
>> 
>> 
>>     
> 


From forax at univ-mlv.fr  Sat Mar 11 13:58:38 2023
From: forax at univ-mlv.fr (Remi Forax)
Date: Sat, 11 Mar 2023 14:58:38 +0100 (CET)
Subject: Deconstruction patterns
In-Reply-To: <d1cfb279-63f3-ff5f-3b3e-74d930e86ee4@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <A9FF8557-64E8-4949-A579-DF123F98AAD7@oracle.com>
 <d1cfb279-63f3-ff5f-3b3e-74d930e86ee4@oracle.com>
Message-ID: <914623224.9120015.1678543118113.JavaMail.zimbra@u-pem.fr>

I've coded a possible implementation here
  https://github.com/forax/amber-deconstructor


- instead of mangling the name, i've followed literally what Dan H said, the VM is good to find a method from a method name and a *descriptor*, so like when you want to have two overloads of a constructor with the same parameter types, you add a fake parameter of type boolean or Void, i've added all the parameters so the algorithm to find the deconstructors in case of overloads is strictly the same as with methods. At call sites, the compiler fill the arguments with zeroes.
- instead of using final instance methods, I use static methods so it also works in an interface (a default method in an interface can not be final). 


By example the class Point,
 class Point {
 ??? final double x, y;
 
 ??? public Point(double x, double y) {
 ??????? this.x = x;
 ??????? this.y = y;
 ??? }
 
 ??? public matcher Point(double x, double y) {
 ??????? x = this.x;
 ??????? y = this.y;
 ??? }

     public matcher Point(int x, int y) {
        x = (int) this.x;
        y = (int) this.y;
     }
 }

is desugared to
 class Point {
 ??? final double x, y;
 
 ??? public Point(double x, double y) {
 ??????? this.x = x;
 ??????? this.y = y;
 ??? }


     // use a static final to emulate a condy
     private static final MethodHandle FACTORY0 =
         RT.carrierFactory(MethodHandles.lookup(), "", MethodHandle.class, methodType(void.class, int.class, int.class));
 
 ??? public static /*synthetic*/ Object Point(Point that, double _, double _) {
 ??????? double x = that.x;
 ??????? double y = that.y;
         return FACTORY0.invokeExact(x, y);
 ??? }

     // use a static final to emulate a condy
     private static final MethodHandle FACTORY1 =
         RT.carrierFactory(MethodHandles.lookup(), "", MethodHandle.class, methodType(void.class, int.class, int.class));

     public static /*synthetic*/ Object Point(Point that, int _, int _) {
        int x = (int) that.x;
        int y = (int) that.y;
        return FACTORY1.invokeExact(x, y);
     }
 }

and at use site

    var point = new Point(2, 3);
    switch (point) {
        case Point(int x, int y) -> {
           ...
        }
    }

is desugared to

  // use a static finals to emulate condies
  private static final MethodHandle POINT_ACCESSOR_0 =
      RT.carrierAccessor(MethodHandles.lookup(), "", MethodHandle.class, methodType(void.class, int.class, int.class), 0);
  private static final MethodHandle POINT_ACCESSOR_1 =
      RT.carrierAccessor(MethodHandles.lookup(), "", MethodHandle.class, methodType(void.class, int.class, int.class), 1);
  ...


    var point = new Point(2, 3);
    switch (point) {
        case Point p -> {
            var carrier = Point.Point(p, 0, 0);   // use zeroes to ask for the right overloads
            var x = (int) POINT_ACCESSOR_0.invokeExact(carrier);
            var y = (int) POINT_ACCESSOR_1.invokeExact(carrier);
           ...
        }
    }

 
The prototype works both with jdk 20 and latest Valhalla early preview. For the later, the generated carrier classes are value classes.

As an implementation detail, the prototype erase all reference types to Object to avoid to generate too many carrier classes at runtime.
Perhaps a better to solution would be to also re-organize the fields to have all objects first and do a permuteArguments when creating the factory method handle.
I did not convert the numeric primitive type (boolean, short, char) to int because if the carrier is a value type, we may want it as compact as possible.

regards,
R?mi

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "Guy Steele" <guy.steele at oracle.com>
> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Wednesday, March 8, 2023 4:07:52 PM
> Subject: Re: Deconstruction patterns

> Updated version, addressing Guy's concerns and fixing/clarifying a few
> other things that came up in earlier discussions.
> 
> 
> # Deconstruction patterns -- translation, use, and reflection
> 
> As we are wrapping up record patterns, it's time to look ahead to the
> next major
> part of the pattern matching story -- extending the capabilities of record
> patterns to all classes that want to support destructuring. Record
> patterns are
> simply a special case of _deconstruction patterns_ or _deconstructors_,
> where we
> derive the deconstructor API, implementation, and use from the state
> description
> of the record.? For an arbitrary class, a deconstruction patterns will
> require
> an explicit member declaration, with a header identifying the names and
> types of
> the bindings and a body that extracts the bindings from the representation.
> 
> ## Deconstructors
> 
> Just as constructors are special cases of methods, deconstruction
> patterns are
> special cases of a more general notion of declared pattern, which also
> includes
> static matchers (the dual of static methods) and instance matchers (the
> dual of
> instance methods.)? Specifically, unlike the more general notion of
> matcher, a
> deconstructor must be _total_; it must always match.? This document will
> focus
> exclusively on deconstructors, and we'll come back to static and instance
> matchers in due time.? (But note that some of the design choices in the
> simple
> case of deconstructors may be constrained by the more general case.)
> 
> There are a number of choices for how we might syntactically represent a
> deconstructor (or more generally, a declared pattern.)? For purposes of
> illustration, this document picks one possible syntactic expression of
> deconstructors, but it is premature to devolve into a syntax discussion
> at this
> time.
> 
> ```
> class Point {
> ??? final double x, y;
> 
> ??? public Point(double x, double y) {
> ??????? this.x = x;
> ??????? this.y = y;
> ??? }
> 
> ??? public matcher Point(double x, double y) {
> ??????? x = this.x;
> ??????? y = this.y;
> ??? }
> }
> ```
> 
> This example illustrates two aspects of the duality between constructors and
> their corresponding deconstructors.? Their APIs are duals: a constructor
> takes N
> parameters containing the desired description of the object state and
> produces a
> constructed object; a deconstructor starts from the constructed object
> and has N
> bindings (outputs) that receive the desired state components. Similarly,
> their
> implementations are duals: the body of the constructor initializes the
> object
> representation from the description, and the body of the deconstructor
> extracts
> the description from the representation.? A deconstructor is best
> understood as
> a _co-constructor_.
> 
> The `Point` example above is special in two ways.? First, the internal
> representation of a `Point`, and the API of the constructor and
> deconstructor,
> are the same: `(double x, double y)`.? We can call the API implied by the
> constructor and deconstructor the _external representation_, and for
> `Point`,
> both the internal and external representations are the same. (This is one of
> the requirements for being a candidate to be a record.)? And second, the
> constructor is _total_; it does not reject any combinations of arguments.
> 
> Here's another version of `Point` which does not have these special
> aspects; it
> uses the same internal representation as before, but chooses a pair of
> strings
> as the external representation:
> 
> ```
> class Point2 {
> ??? final double x, y;
> 
> ??? public Point2(String x, String y) {
> ??????? this.x = Double.parseDouble(x);
> ??????? this.y = Double.parseDouble(y);
> ??? }
> 
> ??? public matcher Point2(String x, String y) {
> ??????? x = Double.toString(this.x);
> ??????? y = Double.toSTring(this.y);
> ??? }
> }
> ```
> 
> The method `Double::parseDouble` will throw `NumberFormatException` if its
> argument does not describe a suitable value, so unlike the `Point`
> constructor,
> the `Point2` constructor is partial: it will reject `new Double("foo",
> "bar")`.
> And the internal representation is no longer the same as the external
> representation.? Less obviously, there are valid string values that we can
> provide to the constructor, but which cannot be represented exactly as
> `double`,
> and which will be approximated; the string value
> `"3.22222222222222222222222222222222222222"` will be approximated with the
> double value `3.2222222222222223`.
> 
> This example highlights more clearly how the deconstructor and
> constructor form
> an _embedding-projection pair_ between the external and internal
> representations.? While some external representations might be invalid,
> and some
> might result in approximation, deconstruct-then-construct is always an
> identity
> transformation.? (Indeed, the specification of `java.lang.Record`
> requires that
> if we deconstruct a record with its accessors, and pass the resulting values
> back to the constructor, we should get a new record that is `equals` to the
> original.)
> 
> The fact that deconstructor and constructor (and eventually, static
> matcher and
> factory) form an embedding-projection pair is why we are able to derive
> higher-level language features, such as [safer
> serialization](https://openjdk.org/projects/amber/design-notes/towards-better-serialization)
> and [functional transformation of immutable
> objects](https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md),
> from a matched set of constructor and deconstructor.
> 
> Of course, users are free to implement constructors without
> deconstructors, or
> constructors and deconstructors whose external representations don't
> match up,
> or even matching constructors and deconstructors that are not
> behaviorally dual.
> But providing a matched set (or several) of constructors and deconstructors
> enables reliably reversible aggregation, and allows us to mechanically
> derive
> useful higher-level features such as withers.
> 
> #### Overloading
> 
> Just as constructors can be overloaded, deconstructors can be overloaded
> for the
> same reason: multiple constructors can expose multiple external
> representations
> for aggregation, and corresponding deconstructors can recover those multiple
> external representations.? Any matching pair of
> deconstructor-constructor (and
> eventually, deconstructor-factory) is a candidate for use in higher-level
> features based on the embedding-projection nature of the
> deconstructor-constructor pair.
> 
> Just as deconstruction is dual to construction, overloading of
> deconstructors is
> dual to that of constructors: rather than restricting which sets of
> parameters
> can be overloaded against each other, we do so with the bindings
> instead.? For
> constructors we require that their signatures not be
> override-equivalent; for
> deconstructors, we require the same of their bindings.
> 
> For a deconstructor (and declared patterns in general), we derive a _binding
> signature_ (and an erased _binding descriptor_) which treats the binding
> list as
> if it were a parameter list.? The overload rule outlined above requires that
> binding signatures for two deconstructors of the same arity not be
> override-equivalent. (We will find it useful later to derive a
> `MethodType` for
> the binding descriptor; this is a `MethodType` whose parameter types are the
> erased types of the bindings.)
> 
> #### Digression: embedding-projection pairs
> 
> Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B
> -> A`,
> forms an _embedding-projection pair_ if embed-then-project is an identity
> function, and project-then-embed _approximates_ the input according to a
> domain-specific approximation metric.? In our example which used strings
> as an external representation, projection rejected some inputs (`"foo"`) and
> approximated others.
> 
> When applied to deconstructor-constructor pairs, this says that
> deconstructing
> an object and then reconstructing it with the resulting bindings should
> result
> in an equivalent object, and constructing an object from an external
> representation and then deconstructing it back into that external
> representation
> should result in an approximation of the original external representation,
> assuming nothing bad happens during construction such as rejecting a bad
> input.
> 
> Embedding-projection pairs have a number of desirable properties, such
> as the
> composition of two e-p pairs is an e-p pair; this property is at the
> heart of
> using deconstructor-constructor pairs for improved serialization and
> functional
> transformation.
> 
> ## Invoking deconstructors
> 
> We've already seen how to "invoke" deconstructors: through pattern matching.
> What we've been calling "record patterns" are merely deconstruction patterns
> derived mechanically from the state description of a record, just as we
> do with
> constructors and accessors; there is little difference between record
> patterns
> and deconstruction patterns other than the ability to declare them
> explicitly.
> (There is an accidental difference in the translation, in that we currently
> implement record patterns by appealing to individual accessors rather than a
> single deconstructor, but this may eventually converge as well.)
> 
> The use-site syntax of deconstruction bears a deliberate similarity to
> that of
> construction; `new Point(x, y)` is deconstructed by `case Point(var x,
> var y)`.
> 
> #### Overload selection
> 
> In the presence of overloaded deconstructors, we need to figure out which
> deconstructor a deconstruction pattern `C(P*)` is referring to. The
> details are
> similar to overload selection for methods, except that we operate on the
> bindings rather than the parameters.? We first search for _applicable
> matchers_,
> using increasingly loose criteria (first excluding boxing, unboxing, and
> varargs; then including boxing and unboxing but not varargs; and
> finally, all
> candidates) and then selecting the most applicable.
> 
> It is tempting to try and bypass the three-phase selection process and use a
> simpler notion of applicability (perhaps noting that we got this process for
> compatibility with existing overload selection decisions when autoboxing and
> varargs were added, and that there are few deconstructor invocations to be
> compatible with yet.)? But because existing overloaded constructors use this
> mechanism, and there is significant value in pairing constructors and
> deconstructors, attempting to invent a simpler-but-different overload
> selection
> mechanism for deconstructors would inevitably undermine the duality between
> matching deconstructor-constructor pairs. So compatibility (this time, with
> existing overloaded constructors) once again forces our hand.
> 
> The specification for overload selection is complicated significantly by
> poly
> expressions (e.g., lambdas); fortunately, there are no "poly patterns",
> and so,
> while the structure of JLS 15.12.2 is retained for overload selection of
> deconstruction patterns, much of the detail is left behind.
> 
> ## Translation
> 
> We translate patterns into synthetic methods with a `Matcher` attribute;
> this
> method implements the matcher behavior.? The translation scheme derives
> from a
> number of requirements, only some of which are in play for deconstructors.
> 
> The matcher method for a deconstructor is a final instance method that
> takes no
> parameters and returns `Object`, perhaps with a special name (just as
> constructors are called `<init>`.)
> 
> #### Carriers
> 
> Because the matcher methods implements the matcher behavior, but a
> matcher may
> "return" multiple bindings (or failure), we must encode the bindings in some
> way.? For this, we use a _carrier object_.? The choice of carrier is
> largely a
> footprint/specificity tradeoff.? One could imagine a carrier class per
> matcher,
> or a carrier class per matcher descriptor, or using `Object[]` as a
> carrier for
> everything, or caching some number of common shapes (e.g, three ints and two
> refs).? This sort of tuning should be separate from the protocol encoded
> in the
> bytecode of the pattern method and its clients.
> 
> We use a small _carrier runtime_ to decouple pattern translation from
> carrier
> selection.? (This same carrier runtime is used by string templates as well.)
> This allows tradeoffs in runtime characteristics (e.g., carrier per
> matcher vs
> sharing carriers across matchers, dropping carrier identity with value types
> later, etc) without affecting the translation. The carrier API consists
> of condy
> bootstraps like:
> 
> ```
> static MethodHandle carrierFactory(METADATA,
> ?????????????????????????????????? MethodType matcherDescriptor) { ... }
> static MethodHandle carrierAccessor(METADATA,
> ??????????????????????????????????? MethodType matcherDescriptor,
> ??????????????????????????????????? int bindingNo) { ... }
> ```
> 
> where `METADATA` are the standard condy metadata arguments (lookup,
> name, type.)
> The `matcherDescriptor` is a `MethodType` describing the binding types.? The
> `carrierFactory` method returns a method handle which takes the bindings and
> produces a carrier object; the `carrierAccessor` method returns method
> handles
> that take the carrier object and return the corresponding binding.? To
> indicate
> success, the matcher method invokes the carrier factory method handle and
> returns the result; to indicate failure (deconstructors cannot fail, but
> other
> matchers can) the matcher method returns null.
> 
> We would translate the XY deconstructor from `Point` as follows
> (pseudo-bytecode):
> 
> ```
> #100: MethodType[(II)V]
> #101: Condy[bsm=Carriers::carrierFactory, args=[#100]]
> 
> final synthetic Object Point$MANGLE() {
> ??? aload_0
> ??? getfield Point::x
> ??? aload_0
> ??? getfield Point::y
> ??? LDC #101
> ??? invokevirtual MethodHandle::invoke(II)Object
> ??? areturn
> }
> ```
> 
> Constant `#100` contains a `MethodType` holding the binding descriptor;
> constant
> `#101` holds a method handle whose parameters are the parameter types of the
> binding descriptor and returns `Object`.
> 
> At the use site, matching a deconstruction pattern is performed by
> invoking the
> matcher method on the appropriate target object, and then extracting the
> components with the carrier accessor method handles if the match is
> successful.
> (Deconstructors are total, so are always successful, but for other patterns,
> null is returned from the matcher method on failure to match.)
> 
> #### Method names
> 
> The name of the matcher method is mangled to support overloading. The JVM
> permits overloading on parameter types, but not return types (and overloaded
> matchers are effectively overloaded on return types.)? We take the
> approach of
> encoding the erasure of the matcher descriptor in the name of the
> pattern.? This
> has several desirable properties: it is stable (the name is derived
> solely from
> stable aspects of the declaration), for matchers with override-equivalent
> signatures (deconstructors can't be overridden, but other patterns can be),
> these map to true overrides in the translation, and valid overloads of
> matchers
> will always have distinct names.
> 
> We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
> descriptor as the mangled disambiguator, which is exactly as stable as
> any other
> method descriptor derived from source declarations.
> 
> #### Attributes
> 
> Because patterns are methods, we can take advantage of all the
> affordances of
> methods.? We can use access bits to control accessibility; we can use the
> attributes that carry annotations, method parameter metadata, and generics
> signatures to carry information about the pattern declaration (and its
> (input)
> parameters, when we get to those kinds of matchers).? What's missing is
> the fact
> that this is a pattern implementation and not an ordinary method, and a
> place to
> put metadata for bindings.? To address the first, we can add the following
> attribute on matcher methods:
> 
> ??? Matcher {
> ??????? u2 name;??????????????????????????? // "Matcher"
> ??????? u4 length;
> ??????? u2 patternFlags;
> ??????? u2 patternName;???????????????????? // UTF8
> ??????? u2 patternDescr;??????????????????? // MethodType
> ??????? u2 attributes_count;
> ??????? attribute_info attributes[attributes_count];
> ??? }
> 
> This says that "this method is a pattern".? The source name of the pattern
> declaration is reified as `patternName`, and the matcher descriptor, which
> encodes the types of the bindings, is reified as a `MethodType` in
> `patternDescr`.? The `flags` word can carry matcher-specific information
> such as
> "this matcher is a deconstructor" or "this matcher is total".
> 
> A matcher method may have the usual variety of method attributes, such as
> `RuntimeInvisibleAnnotations` for annotations on the matcher declaration
> itself.
> 
> If we wish to encode information about the matcher _bindings_, we do so with
> attributes inside the `Matcher` annotation itself.? Attributes such as
> `Signature`, `ParameterNames`, `RuntimeVisibleParameterAnnotations`,
> etc, can
> appear in a `Matcher` and are interpreted relative to the matcher
> signature or
> descriptor.? So if we had a matcher:
> 
> ```
> matcher Foo(@Bar List<String> list) { ... }
> ```
> 
> then the `Matcher` would contain a signature attribute corresponding to
> `(List<String>)` and a `RuntimeXxxParameterAnnotations` attribute
> describing the
> `@Bar` annotation on the first "parameter".
> 
> #### Reflection
> 
> Since matchers are a new kind of class member, they will need a new kind of
> reflective object, and a method that is analogous to
> `Class::getConstructors`.
> The reflective object for matchers should extend `Executable`, as all of the
> existing methods on `Executable` make sense for patterns.? If the pattern is
> reflectively invoked, it returns `null` for no match, or an `Object[]`
> which is
> the boxing of the values in the carrier.
> 
> We will then need some additional methods to describe the bindings, so the
> subtype of `Executable` has methods like `getBindings`,
> `getAnnotatedBindings`,
> `getGenericBindings`, `isDeconstructor`, `isPartial`, etc.? These
> methods will
> decode the `Matcher` attribute and its embedded attributes.
> 
> ## Summary
> 
> This design borrows from previous rounds, but makes a number of
> simplifications.
> 
> ?- The bindings of a pattern are captured in a `MethodType`, called the
> _matcher
> ?? descriptor_.? The parameters of the matcher descriptor are the types
> of the
> ?? bindings; the return type is either `V` or the minimal type that
> will match
> ?? (but is not as important as the bindings.)
> ?- Matchers are translated as methods whose names are derived
> deterministically
> ?? from the name of the matcher and the erasure of the pattern
> descriptor. These
> ?? are called _matcher methods_.? Matcher methods take as parameters
> the input
> ?? parameters of the pattern (if any), and return `Object`.
> ?- The returned object is an opaque carrier.? Null means the pattern didn't
> ?? match.? A non-null value is the carrier type (from the carrier
> runtime) which
> ?? is derived from the pattern descriptor.
> ?- Matcher methods are not directly invocable from the source language;
> they are
> ?? invoked indirectly through pattern matching or reflection.
> ?- Generated code invokes the matcher method and interprets the
> returned value
> ?? according to the protocol, using MHs from the carrier runtime to
> access the
> ?? bindings.
> ?- Matcher methods have a `Matcher` attribute, which captures
> information about
> ?? the matcher as a whole (is a total/partial, a deconstructor, etc) and
> ?? parameter-related attributes which describe the bindings.
> ?- Matchers are reflected through a new subtype of `Executable`, which
> exposes
> ?? new methods to reflect over bindings.
> ?- When invoking a matcher reflectively, the carrier is boxed to an
> `Object[]`.

From brian.goetz at oracle.com  Sat Mar 11 15:52:50 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Sat, 11 Mar 2023 10:52:50 -0500
Subject: Deconstruction patterns
In-Reply-To: <914623224.9120015.1678543118113.JavaMail.zimbra@u-pem.fr>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <A9FF8557-64E8-4949-A579-DF123F98AAD7@oracle.com>
 <d1cfb279-63f3-ff5f-3b3e-74d930e86ee4@oracle.com>
 <914623224.9120015.1678543118113.JavaMail.zimbra@u-pem.fr>
Message-ID: <e9f2307c-a5e7-b5c6-9be7-0cd766f8329f@oracle.com>

I agree that translating dtors as static methods is probably better than 
final instance methods here.

Using fake method arguments instead of mangled names is a very clever 
trick here.? (It won't get you all the way there, though; you'll need 
some more encoding foo to keep

 ??? matcher foo(__in int x)(__out int y)

and

 ??? matcher foo()(__out int x, __out int y)

from having clashing classfile descriptors.? I'm sure you can accomplish 
this with more mangling, but ... )

More importantly, I don't particularly like how this pushes fake work 
onto the client:

     var carrier = Point.Point(p, 0, 0);   // use zeroes to ask for the right overloads

to pass fake arguments for the purposes of overload selection.? (You 
could hide that behind a condy that inserts the default arguments, but 
that doesn't make me like it much better.)

Given this choice, I still prefer name mangling.? (If you dislike name 
mangling so much, perhaps a better cure would be to invest in better 
display of call stacks in exception traces?)


On 3/11/2023 8:58 AM, Remi Forax wrote:
> I've coded a possible implementation here
>    https://urldefense.com/v3/__https://github.com/forax/amber-deconstructor__;!!ACWV5N9M2RV99hQ!M4HQ_Wx5MQZEQwUavg6-Kasr5GKMBzQiac4CoRAmbenb1xcNnDCGwICoGzsX058e1uIwrncE7-Q5Zt7enSZL$
>
>
> - instead of mangling the name, i've followed literally what Dan H said, the VM is good to find a method from a method name and a *descriptor*, so like when you want to have two overloads of a constructor with the same parameter types, you add a fake parameter of type boolean or Void, i've added all the parameters so the algorithm to find the deconstructors in case of overloads is strictly the same as with methods. At call sites, the compiler fill the arguments with zeroes.
> - instead of using final instance methods, I use static methods so it also works in an interface (a default method in an interface can not be final).
>
>
> By example the class Point,
>   class Point {
>   ??? final double x, y;
>   
>   ??? public Point(double x, double y) {
>   ??????? this.x = x;
>   ??????? this.y = y;
>   ??? }
>   
>   ??? public matcher Point(double x, double y) {
>   ??????? x = this.x;
>   ??????? y = this.y;
>   ??? }
>
>       public matcher Point(int x, int y) {
>          x = (int) this.x;
>          y = (int) this.y;
>       }
>   }
>
> is desugared to
>   class Point {
>   ??? final double x, y;
>   
>   ??? public Point(double x, double y) {
>   ??????? this.x = x;
>   ??????? this.y = y;
>   ??? }
>
>
>       // use a static final to emulate a condy
>       private static final MethodHandle FACTORY0 =
>           RT.carrierFactory(MethodHandles.lookup(), "", MethodHandle.class, methodType(void.class, int.class, int.class));
>   
>   ??? public static /*synthetic*/ Object Point(Point that, double _, double _) {
>   ??????? double x = that.x;
>   ??????? double y = that.y;
>           return FACTORY0.invokeExact(x, y);
>   ??? }
>
>       // use a static final to emulate a condy
>       private static final MethodHandle FACTORY1 =
>           RT.carrierFactory(MethodHandles.lookup(), "", MethodHandle.class, methodType(void.class, int.class, int.class));
>
>       public static /*synthetic*/ Object Point(Point that, int _, int _) {
>          int x = (int) that.x;
>          int y = (int) that.y;
>          return FACTORY1.invokeExact(x, y);
>       }
>   }
>
> and at use site
>
>      var point = new Point(2, 3);
>      switch (point) {
>          case Point(int x, int y) -> {
>             ...
>          }
>      }
>
> is desugared to
>
>    // use a static finals to emulate condies
>    private static final MethodHandle POINT_ACCESSOR_0 =
>        RT.carrierAccessor(MethodHandles.lookup(), "", MethodHandle.class, methodType(void.class, int.class, int.class), 0);
>    private static final MethodHandle POINT_ACCESSOR_1 =
>        RT.carrierAccessor(MethodHandles.lookup(), "", MethodHandle.class, methodType(void.class, int.class, int.class), 1);
>    ...
>
>
>      var point = new Point(2, 3);
>      switch (point) {
>          case Point p -> {
>              var carrier = Point.Point(p, 0, 0);   // use zeroes to ask for the right overloads
>              var x = (int) POINT_ACCESSOR_0.invokeExact(carrier);
>              var y = (int) POINT_ACCESSOR_1.invokeExact(carrier);
>             ...
>          }
>      }
>
>   
> The prototype works both with jdk 20 and latest Valhalla early preview. For the later, the generated carrier classes are value classes.
>
> As an implementation detail, the prototype erase all reference types to Object to avoid to generate too many carrier classes at runtime.
> Perhaps a better to solution would be to also re-organize the fields to have all objects first and do a permuteArguments when creating the factory method handle.
> I did not convert the numeric primitive type (boolean, short, char) to int because if the carrier is a value type, we may want it as compact as possible.
>
> regards,
> R?mi
>
> ----- Original Message -----
>> From: "Brian Goetz" <brian.goetz at oracle.com>
>> To: "Guy Steele" <guy.steele at oracle.com>
>> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
>> Sent: Wednesday, March 8, 2023 4:07:52 PM
>> Subject: Re: Deconstruction patterns
>> Updated version, addressing Guy's concerns and fixing/clarifying a few
>> other things that came up in earlier discussions.
>>
>>
>> # Deconstruction patterns -- translation, use, and reflection
>>
>> As we are wrapping up record patterns, it's time to look ahead to the
>> next major
>> part of the pattern matching story -- extending the capabilities of record
>> patterns to all classes that want to support destructuring. Record
>> patterns are
>> simply a special case of _deconstruction patterns_ or _deconstructors_,
>> where we
>> derive the deconstructor API, implementation, and use from the state
>> description
>> of the record.? For an arbitrary class, a deconstruction patterns will
>> require
>> an explicit member declaration, with a header identifying the names and
>> types of
>> the bindings and a body that extracts the bindings from the representation.
>>
>> ## Deconstructors
>>
>> Just as constructors are special cases of methods, deconstruction
>> patterns are
>> special cases of a more general notion of declared pattern, which also
>> includes
>> static matchers (the dual of static methods) and instance matchers (the
>> dual of
>> instance methods.)? Specifically, unlike the more general notion of
>> matcher, a
>> deconstructor must be _total_; it must always match.? This document will
>> focus
>> exclusively on deconstructors, and we'll come back to static and instance
>> matchers in due time.? (But note that some of the design choices in the
>> simple
>> case of deconstructors may be constrained by the more general case.)
>>
>> There are a number of choices for how we might syntactically represent a
>> deconstructor (or more generally, a declared pattern.)? For purposes of
>> illustration, this document picks one possible syntactic expression of
>> deconstructors, but it is premature to devolve into a syntax discussion
>> at this
>> time.
>>
>> ```
>> class Point {
>>  ??? final double x, y;
>>
>>  ??? public Point(double x, double y) {
>>  ??????? this.x = x;
>>  ??????? this.y = y;
>>  ??? }
>>
>>  ??? public matcher Point(double x, double y) {
>>  ??????? x = this.x;
>>  ??????? y = this.y;
>>  ??? }
>> }
>> ```
>>
>> This example illustrates two aspects of the duality between constructors and
>> their corresponding deconstructors.? Their APIs are duals: a constructor
>> takes N
>> parameters containing the desired description of the object state and
>> produces a
>> constructed object; a deconstructor starts from the constructed object
>> and has N
>> bindings (outputs) that receive the desired state components. Similarly,
>> their
>> implementations are duals: the body of the constructor initializes the
>> object
>> representation from the description, and the body of the deconstructor
>> extracts
>> the description from the representation.? A deconstructor is best
>> understood as
>> a _co-constructor_.
>>
>> The `Point` example above is special in two ways.? First, the internal
>> representation of a `Point`, and the API of the constructor and
>> deconstructor,
>> are the same: `(double x, double y)`.? We can call the API implied by the
>> constructor and deconstructor the _external representation_, and for
>> `Point`,
>> both the internal and external representations are the same. (This is one of
>> the requirements for being a candidate to be a record.)? And second, the
>> constructor is _total_; it does not reject any combinations of arguments.
>>
>> Here's another version of `Point` which does not have these special
>> aspects; it
>> uses the same internal representation as before, but chooses a pair of
>> strings
>> as the external representation:
>>
>> ```
>> class Point2 {
>>  ??? final double x, y;
>>
>>  ??? public Point2(String x, String y) {
>>  ??????? this.x = Double.parseDouble(x);
>>  ??????? this.y = Double.parseDouble(y);
>>  ??? }
>>
>>  ??? public matcher Point2(String x, String y) {
>>  ??????? x = Double.toString(this.x);
>>  ??????? y = Double.toSTring(this.y);
>>  ??? }
>> }
>> ```
>>
>> The method `Double::parseDouble` will throw `NumberFormatException` if its
>> argument does not describe a suitable value, so unlike the `Point`
>> constructor,
>> the `Point2` constructor is partial: it will reject `new Double("foo",
>> "bar")`.
>> And the internal representation is no longer the same as the external
>> representation.? Less obviously, there are valid string values that we can
>> provide to the constructor, but which cannot be represented exactly as
>> `double`,
>> and which will be approximated; the string value
>> `"3.22222222222222222222222222222222222222"` will be approximated with the
>> double value `3.2222222222222223`.
>>
>> This example highlights more clearly how the deconstructor and
>> constructor form
>> an _embedding-projection pair_ between the external and internal
>> representations.? While some external representations might be invalid,
>> and some
>> might result in approximation, deconstruct-then-construct is always an
>> identity
>> transformation.? (Indeed, the specification of `java.lang.Record`
>> requires that
>> if we deconstruct a record with its accessors, and pass the resulting values
>> back to the constructor, we should get a new record that is `equals` to the
>> original.)
>>
>> The fact that deconstructor and constructor (and eventually, static
>> matcher and
>> factory) form an embedding-projection pair is why we are able to derive
>> higher-level language features, such as [safer
>> serialization](https://openjdk.org/projects/amber/design-notes/towards-better-serialization)
>> and [functional transformation of immutable
>> objects](https://urldefense.com/v3/__https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md__;!!ACWV5N9M2RV99hQ!M4HQ_Wx5MQZEQwUavg6-Kasr5GKMBzQiac4CoRAmbenb1xcNnDCGwICoGzsX058e1uIwrncE7-Q5ZsjLZu9k$ ),
>> from a matched set of constructor and deconstructor.
>>
>> Of course, users are free to implement constructors without
>> deconstructors, or
>> constructors and deconstructors whose external representations don't
>> match up,
>> or even matching constructors and deconstructors that are not
>> behaviorally dual.
>> But providing a matched set (or several) of constructors and deconstructors
>> enables reliably reversible aggregation, and allows us to mechanically
>> derive
>> useful higher-level features such as withers.
>>
>> #### Overloading
>>
>> Just as constructors can be overloaded, deconstructors can be overloaded
>> for the
>> same reason: multiple constructors can expose multiple external
>> representations
>> for aggregation, and corresponding deconstructors can recover those multiple
>> external representations.? Any matching pair of
>> deconstructor-constructor (and
>> eventually, deconstructor-factory) is a candidate for use in higher-level
>> features based on the embedding-projection nature of the
>> deconstructor-constructor pair.
>>
>> Just as deconstruction is dual to construction, overloading of
>> deconstructors is
>> dual to that of constructors: rather than restricting which sets of
>> parameters
>> can be overloaded against each other, we do so with the bindings
>> instead.? For
>> constructors we require that their signatures not be
>> override-equivalent; for
>> deconstructors, we require the same of their bindings.
>>
>> For a deconstructor (and declared patterns in general), we derive a _binding
>> signature_ (and an erased _binding descriptor_) which treats the binding
>> list as
>> if it were a parameter list.? The overload rule outlined above requires that
>> binding signatures for two deconstructors of the same arity not be
>> override-equivalent. (We will find it useful later to derive a
>> `MethodType` for
>> the binding descriptor; this is a `MethodType` whose parameter types are the
>> erased types of the bindings.)
>>
>> #### Digression: embedding-projection pairs
>>
>> Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B
>> -> A`,
>> forms an _embedding-projection pair_ if embed-then-project is an identity
>> function, and project-then-embed _approximates_ the input according to a
>> domain-specific approximation metric.? In our example which used strings
>> as an external representation, projection rejected some inputs (`"foo"`) and
>> approximated others.
>>
>> When applied to deconstructor-constructor pairs, this says that
>> deconstructing
>> an object and then reconstructing it with the resulting bindings should
>> result
>> in an equivalent object, and constructing an object from an external
>> representation and then deconstructing it back into that external
>> representation
>> should result in an approximation of the original external representation,
>> assuming nothing bad happens during construction such as rejecting a bad
>> input.
>>
>> Embedding-projection pairs have a number of desirable properties, such
>> as the
>> composition of two e-p pairs is an e-p pair; this property is at the
>> heart of
>> using deconstructor-constructor pairs for improved serialization and
>> functional
>> transformation.
>>
>> ## Invoking deconstructors
>>
>> We've already seen how to "invoke" deconstructors: through pattern matching.
>> What we've been calling "record patterns" are merely deconstruction patterns
>> derived mechanically from the state description of a record, just as we
>> do with
>> constructors and accessors; there is little difference between record
>> patterns
>> and deconstruction patterns other than the ability to declare them
>> explicitly.
>> (There is an accidental difference in the translation, in that we currently
>> implement record patterns by appealing to individual accessors rather than a
>> single deconstructor, but this may eventually converge as well.)
>>
>> The use-site syntax of deconstruction bears a deliberate similarity to
>> that of
>> construction; `new Point(x, y)` is deconstructed by `case Point(var x,
>> var y)`.
>>
>> #### Overload selection
>>
>> In the presence of overloaded deconstructors, we need to figure out which
>> deconstructor a deconstruction pattern `C(P*)` is referring to. The
>> details are
>> similar to overload selection for methods, except that we operate on the
>> bindings rather than the parameters.? We first search for _applicable
>> matchers_,
>> using increasingly loose criteria (first excluding boxing, unboxing, and
>> varargs; then including boxing and unboxing but not varargs; and
>> finally, all
>> candidates) and then selecting the most applicable.
>>
>> It is tempting to try and bypass the three-phase selection process and use a
>> simpler notion of applicability (perhaps noting that we got this process for
>> compatibility with existing overload selection decisions when autoboxing and
>> varargs were added, and that there are few deconstructor invocations to be
>> compatible with yet.)? But because existing overloaded constructors use this
>> mechanism, and there is significant value in pairing constructors and
>> deconstructors, attempting to invent a simpler-but-different overload
>> selection
>> mechanism for deconstructors would inevitably undermine the duality between
>> matching deconstructor-constructor pairs. So compatibility (this time, with
>> existing overloaded constructors) once again forces our hand.
>>
>> The specification for overload selection is complicated significantly by
>> poly
>> expressions (e.g., lambdas); fortunately, there are no "poly patterns",
>> and so,
>> while the structure of JLS 15.12.2 is retained for overload selection of
>> deconstruction patterns, much of the detail is left behind.
>>
>> ## Translation
>>
>> We translate patterns into synthetic methods with a `Matcher` attribute;
>> this
>> method implements the matcher behavior.? The translation scheme derives
>> from a
>> number of requirements, only some of which are in play for deconstructors.
>>
>> The matcher method for a deconstructor is a final instance method that
>> takes no
>> parameters and returns `Object`, perhaps with a special name (just as
>> constructors are called `<init>`.)
>>
>> #### Carriers
>>
>> Because the matcher methods implements the matcher behavior, but a
>> matcher may
>> "return" multiple bindings (or failure), we must encode the bindings in some
>> way.? For this, we use a _carrier object_.? The choice of carrier is
>> largely a
>> footprint/specificity tradeoff.? One could imagine a carrier class per
>> matcher,
>> or a carrier class per matcher descriptor, or using `Object[]` as a
>> carrier for
>> everything, or caching some number of common shapes (e.g, three ints and two
>> refs).? This sort of tuning should be separate from the protocol encoded
>> in the
>> bytecode of the pattern method and its clients.
>>
>> We use a small _carrier runtime_ to decouple pattern translation from
>> carrier
>> selection.? (This same carrier runtime is used by string templates as well.)
>> This allows tradeoffs in runtime characteristics (e.g., carrier per
>> matcher vs
>> sharing carriers across matchers, dropping carrier identity with value types
>> later, etc) without affecting the translation. The carrier API consists
>> of condy
>> bootstraps like:
>>
>> ```
>> static MethodHandle carrierFactory(METADATA,
>>  ?????????????????????????????????? MethodType matcherDescriptor) { ... }
>> static MethodHandle carrierAccessor(METADATA,
>>  ??????????????????????????????????? MethodType matcherDescriptor,
>>  ??????????????????????????????????? int bindingNo) { ... }
>> ```
>>
>> where `METADATA` are the standard condy metadata arguments (lookup,
>> name, type.)
>> The `matcherDescriptor` is a `MethodType` describing the binding types.? The
>> `carrierFactory` method returns a method handle which takes the bindings and
>> produces a carrier object; the `carrierAccessor` method returns method
>> handles
>> that take the carrier object and return the corresponding binding.? To
>> indicate
>> success, the matcher method invokes the carrier factory method handle and
>> returns the result; to indicate failure (deconstructors cannot fail, but
>> other
>> matchers can) the matcher method returns null.
>>
>> We would translate the XY deconstructor from `Point` as follows
>> (pseudo-bytecode):
>>
>> ```
>> #100: MethodType[(II)V]
>> #101: Condy[bsm=Carriers::carrierFactory, args=[#100]]
>>
>> final synthetic Object Point$MANGLE() {
>>  ??? aload_0
>>  ??? getfield Point::x
>>  ??? aload_0
>>  ??? getfield Point::y
>>  ??? LDC #101
>>  ??? invokevirtual MethodHandle::invoke(II)Object
>>  ??? areturn
>> }
>> ```
>>
>> Constant `#100` contains a `MethodType` holding the binding descriptor;
>> constant
>> `#101` holds a method handle whose parameters are the parameter types of the
>> binding descriptor and returns `Object`.
>>
>> At the use site, matching a deconstruction pattern is performed by
>> invoking the
>> matcher method on the appropriate target object, and then extracting the
>> components with the carrier accessor method handles if the match is
>> successful.
>> (Deconstructors are total, so are always successful, but for other patterns,
>> null is returned from the matcher method on failure to match.)
>>
>> #### Method names
>>
>> The name of the matcher method is mangled to support overloading. The JVM
>> permits overloading on parameter types, but not return types (and overloaded
>> matchers are effectively overloaded on return types.)? We take the
>> approach of
>> encoding the erasure of the matcher descriptor in the name of the
>> pattern.? This
>> has several desirable properties: it is stable (the name is derived
>> solely from
>> stable aspects of the declaration), for matchers with override-equivalent
>> signatures (deconstructors can't be overridden, but other patterns can be),
>> these map to true overrides in the translation, and valid overloads of
>> matchers
>> will always have distinct names.
>>
>> We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
>> descriptor as the mangled disambiguator, which is exactly as stable as
>> any other
>> method descriptor derived from source declarations.
>>
>> #### Attributes
>>
>> Because patterns are methods, we can take advantage of all the
>> affordances of
>> methods.? We can use access bits to control accessibility; we can use the
>> attributes that carry annotations, method parameter metadata, and generics
>> signatures to carry information about the pattern declaration (and its
>> (input)
>> parameters, when we get to those kinds of matchers).? What's missing is
>> the fact
>> that this is a pattern implementation and not an ordinary method, and a
>> place to
>> put metadata for bindings.? To address the first, we can add the following
>> attribute on matcher methods:
>>
>>  ??? Matcher {
>>  ??????? u2 name;??????????????????????????? // "Matcher"
>>  ??????? u4 length;
>>  ??????? u2 patternFlags;
>>  ??????? u2 patternName;???????????????????? // UTF8
>>  ??????? u2 patternDescr;??????????????????? // MethodType
>>  ??????? u2 attributes_count;
>>  ??????? attribute_info attributes[attributes_count];
>>  ??? }
>>
>> This says that "this method is a pattern".? The source name of the pattern
>> declaration is reified as `patternName`, and the matcher descriptor, which
>> encodes the types of the bindings, is reified as a `MethodType` in
>> `patternDescr`.? The `flags` word can carry matcher-specific information
>> such as
>> "this matcher is a deconstructor" or "this matcher is total".
>>
>> A matcher method may have the usual variety of method attributes, such as
>> `RuntimeInvisibleAnnotations` for annotations on the matcher declaration
>> itself.
>>
>> If we wish to encode information about the matcher _bindings_, we do so with
>> attributes inside the `Matcher` annotation itself.? Attributes such as
>> `Signature`, `ParameterNames`, `RuntimeVisibleParameterAnnotations`,
>> etc, can
>> appear in a `Matcher` and are interpreted relative to the matcher
>> signature or
>> descriptor.? So if we had a matcher:
>>
>> ```
>> matcher Foo(@Bar List<String> list) { ... }
>> ```
>>
>> then the `Matcher` would contain a signature attribute corresponding to
>> `(List<String>)` and a `RuntimeXxxParameterAnnotations` attribute
>> describing the
>> `@Bar` annotation on the first "parameter".
>>
>> #### Reflection
>>
>> Since matchers are a new kind of class member, they will need a new kind of
>> reflective object, and a method that is analogous to
>> `Class::getConstructors`.
>> The reflective object for matchers should extend `Executable`, as all of the
>> existing methods on `Executable` make sense for patterns.? If the pattern is
>> reflectively invoked, it returns `null` for no match, or an `Object[]`
>> which is
>> the boxing of the values in the carrier.
>>
>> We will then need some additional methods to describe the bindings, so the
>> subtype of `Executable` has methods like `getBindings`,
>> `getAnnotatedBindings`,
>> `getGenericBindings`, `isDeconstructor`, `isPartial`, etc.? These
>> methods will
>> decode the `Matcher` attribute and its embedded attributes.
>>
>> ## Summary
>>
>> This design borrows from previous rounds, but makes a number of
>> simplifications.
>>
>>  ?- The bindings of a pattern are captured in a `MethodType`, called the
>> _matcher
>>  ?? descriptor_.? The parameters of the matcher descriptor are the types
>> of the
>>  ?? bindings; the return type is either `V` or the minimal type that
>> will match
>>  ?? (but is not as important as the bindings.)
>>  ?- Matchers are translated as methods whose names are derived
>> deterministically
>>  ?? from the name of the matcher and the erasure of the pattern
>> descriptor. These
>>  ?? are called _matcher methods_.? Matcher methods take as parameters
>> the input
>>  ?? parameters of the pattern (if any), and return `Object`.
>>  ?- The returned object is an opaque carrier.? Null means the pattern didn't
>>  ?? match.? A non-null value is the carrier type (from the carrier
>> runtime) which
>>  ?? is derived from the pattern descriptor.
>>  ?- Matcher methods are not directly invocable from the source language;
>> they are
>>  ?? invoked indirectly through pattern matching or reflection.
>>  ?- Generated code invokes the matcher method and interprets the
>> returned value
>>  ?? according to the protocol, using MHs from the carrier runtime to
>> access the
>>  ?? bindings.
>>  ?- Matcher methods have a `Matcher` attribute, which captures
>> information about
>>  ?? the matcher as a whole (is a total/partial, a deconstructor, etc) and
>>  ?? parameter-related attributes which describe the bindings.
>>  ?- Matchers are reflected through a new subtype of `Executable`, which
>> exposes
>>  ?? new methods to reflect over bindings.
>>  ?- When invoking a matcher reflectively, the carrier is boxed to an
>> `Object[]`.


From forax at univ-mlv.fr  Sat Mar 11 19:52:59 2023
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Sat, 11 Mar 2023 20:52:59 +0100 (CET)
Subject: Deconstruction patterns
In-Reply-To: <e9f2307c-a5e7-b5c6-9be7-0cd766f8329f@oracle.com>
References: <7a131ad4-3dce-7f34-e2b0-8f40380c1105@oracle.com>
 <A9FF8557-64E8-4949-A579-DF123F98AAD7@oracle.com>
 <d1cfb279-63f3-ff5f-3b3e-74d930e86ee4@oracle.com>
 <914623224.9120015.1678543118113.JavaMail.zimbra@u-pem.fr>
 <e9f2307c-a5e7-b5c6-9be7-0cd766f8329f@oracle.com>
Message-ID: <405531273.9173548.1678564379013.JavaMail.zimbra@u-pem.fr>

----- Original Message -----
> From: "Brian Goetz" <brian.goetz at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Saturday, March 11, 2023 4:52:50 PM
> Subject: Re: Deconstruction patterns

> I agree that translating dtors as static methods is probably better than
> final instance methods here.
> 
> Using fake method arguments instead of mangled names is a very clever
> trick here.? (It won't get you all the way there, though; you'll need
> some more encoding foo to keep
> 
> ??? matcher foo(__in int x)(__out int y)
> 
> and
> 
> ??? matcher foo()(__out int x, __out int y)
> 
> from having clashing classfile descriptors.? I'm sure you can accomplish
> this with more mangling, but ... )

Using an empty value type as separator type may solve that issue.

    matcher foo(__in int x, Separator _, __out int y)

> 
> More importantly, I don't particularly like how this pushes fake work
> onto the client:
> 
>     var carrier = Point.Point(p, 0, 0);   // use zeroes to ask for the right
>     overloads
> 
> to pass fake arguments for the purposes of overload selection.? (You
> could hide that behind a condy that inserts the default arguments, but
> that doesn't make me like it much better.)

BTW, from experience, adding several different condy or indy into one method delays, a lot, the time to steady state, because each condy/indy is not optimized at the same time, to the point where all parts are ""never"" fully inlined together. It's better to have either only one condy/indy with a bunch of method handles or to generate the corresponding bytecode as one blob (like with lambdas).

> 
> Given this choice, I still prefer name mangling.

We only need mangling in case of overriding. Otherwise using a stable index like the index of the matcher descriptors once sorted, is enough to make the compilation of a class containing several deconstructors reproducible.

You do not want deconstructors to be polymorphic, because you can always have the deconstructor calling a polymorphic method if necessary (if i understood our discussion about Map.Entry correctly). So i'm not sure that named matchers should be polymorphic too. A syntax, like Optional.of(var value) or Optional.empty() only mentions the base class after all. And like with a deconstructor, a named matcher can call a polymorphic method if necessary.

If nothing is polymoprphic, mangling is not necessary anymore.

>?(If you dislike name mangling so much, perhaps a better cure would be to invest in better display of call stacks in exception traces?)

It's a long awaited feature, i would love to have it but changing all the stacktrace generation codes is not a small task because those codes are perf sensitive.

R?mi

> 
> 
> On 3/11/2023 8:58 AM, Remi Forax wrote:
>> I've coded a possible implementation here
>>    https://urldefense.com/v3/__https://github.com/forax/amber-deconstructor__;!!ACWV5N9M2RV99hQ!M4HQ_Wx5MQZEQwUavg6-Kasr5GKMBzQiac4CoRAmbenb1xcNnDCGwICoGzsX058e1uIwrncE7-Q5Zt7enSZL$
>>
>>
>> - instead of mangling the name, i've followed literally what Dan H said, the VM
>> is good to find a method from a method name and a *descriptor*, so like when
>> you want to have two overloads of a constructor with the same parameter types,
>> you add a fake parameter of type boolean or Void, i've added all the parameters
>> so the algorithm to find the deconstructors in case of overloads is strictly
>> the same as with methods. At call sites, the compiler fill the arguments with
>> zeroes.
>> - instead of using final instance methods, I use static methods so it also works
>> in an interface (a default method in an interface can not be final).
>>
>>
>> By example the class Point,
>>   class Point {
>>   ??? final double x, y;
>>   
>>   ??? public Point(double x, double y) {
>>   ??????? this.x = x;
>>   ??????? this.y = y;
>>   ??? }
>>   
>>   ??? public matcher Point(double x, double y) {
>>   ??????? x = this.x;
>>   ??????? y = this.y;
>>   ??? }
>>
>>       public matcher Point(int x, int y) {
>>          x = (int) this.x;
>>          y = (int) this.y;
>>       }
>>   }
>>
>> is desugared to
>>   class Point {
>>   ??? final double x, y;
>>   
>>   ??? public Point(double x, double y) {
>>   ??????? this.x = x;
>>   ??????? this.y = y;
>>   ??? }
>>
>>
>>       // use a static final to emulate a condy
>>       private static final MethodHandle FACTORY0 =
>>           RT.carrierFactory(MethodHandles.lookup(), "", MethodHandle.class,
>>           methodType(void.class, int.class, int.class));
>>   
>>   ??? public static /*synthetic*/ Object Point(Point that, double _, double _) {
>>   ??????? double x = that.x;
>>   ??????? double y = that.y;
>>           return FACTORY0.invokeExact(x, y);
>>   ??? }
>>
>>       // use a static final to emulate a condy
>>       private static final MethodHandle FACTORY1 =
>>           RT.carrierFactory(MethodHandles.lookup(), "", MethodHandle.class,
>>           methodType(void.class, int.class, int.class));
>>
>>       public static /*synthetic*/ Object Point(Point that, int _, int _) {
>>          int x = (int) that.x;
>>          int y = (int) that.y;
>>          return FACTORY1.invokeExact(x, y);
>>       }
>>   }
>>
>> and at use site
>>
>>      var point = new Point(2, 3);
>>      switch (point) {
>>          case Point(int x, int y) -> {
>>             ...
>>          }
>>      }
>>
>> is desugared to
>>
>>    // use a static finals to emulate condies
>>    private static final MethodHandle POINT_ACCESSOR_0 =
>>        RT.carrierAccessor(MethodHandles.lookup(), "", MethodHandle.class,
>>        methodType(void.class, int.class, int.class), 0);
>>    private static final MethodHandle POINT_ACCESSOR_1 =
>>        RT.carrierAccessor(MethodHandles.lookup(), "", MethodHandle.class,
>>        methodType(void.class, int.class, int.class), 1);
>>    ...
>>
>>
>>      var point = new Point(2, 3);
>>      switch (point) {
>>          case Point p -> {
>>              var carrier = Point.Point(p, 0, 0);   // use zeroes to ask for the right
>>              overloads
>>              var x = (int) POINT_ACCESSOR_0.invokeExact(carrier);
>>              var y = (int) POINT_ACCESSOR_1.invokeExact(carrier);
>>             ...
>>          }
>>      }
>>
>>   
>> The prototype works both with jdk 20 and latest Valhalla early preview. For the
>> later, the generated carrier classes are value classes.
>>
>> As an implementation detail, the prototype erase all reference types to Object
>> to avoid to generate too many carrier classes at runtime.
>> Perhaps a better to solution would be to also re-organize the fields to have all
>> objects first and do a permuteArguments when creating the factory method
>> handle.
>> I did not convert the numeric primitive type (boolean, short, char) to int
>> because if the carrier is a value type, we may want it as compact as possible.
>>
>> regards,
>> R?mi
>>
>> ----- Original Message -----
>>> From: "Brian Goetz" <brian.goetz at oracle.com>
>>> To: "Guy Steele" <guy.steele at oracle.com>
>>> Cc: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
>>> Sent: Wednesday, March 8, 2023 4:07:52 PM
>>> Subject: Re: Deconstruction patterns
>>> Updated version, addressing Guy's concerns and fixing/clarifying a few
>>> other things that came up in earlier discussions.
>>>
>>>
>>> # Deconstruction patterns -- translation, use, and reflection
>>>
>>> As we are wrapping up record patterns, it's time to look ahead to the
>>> next major
>>> part of the pattern matching story -- extending the capabilities of record
>>> patterns to all classes that want to support destructuring. Record
>>> patterns are
>>> simply a special case of _deconstruction patterns_ or _deconstructors_,
>>> where we
>>> derive the deconstructor API, implementation, and use from the state
>>> description
>>> of the record.? For an arbitrary class, a deconstruction patterns will
>>> require
>>> an explicit member declaration, with a header identifying the names and
>>> types of
>>> the bindings and a body that extracts the bindings from the representation.
>>>
>>> ## Deconstructors
>>>
>>> Just as constructors are special cases of methods, deconstruction
>>> patterns are
>>> special cases of a more general notion of declared pattern, which also
>>> includes
>>> static matchers (the dual of static methods) and instance matchers (the
>>> dual of
>>> instance methods.)? Specifically, unlike the more general notion of
>>> matcher, a
>>> deconstructor must be _total_; it must always match.? This document will
>>> focus
>>> exclusively on deconstructors, and we'll come back to static and instance
>>> matchers in due time.? (But note that some of the design choices in the
>>> simple
>>> case of deconstructors may be constrained by the more general case.)
>>>
>>> There are a number of choices for how we might syntactically represent a
>>> deconstructor (or more generally, a declared pattern.)? For purposes of
>>> illustration, this document picks one possible syntactic expression of
>>> deconstructors, but it is premature to devolve into a syntax discussion
>>> at this
>>> time.
>>>
>>> ```
>>> class Point {
>>>  ??? final double x, y;
>>>
>>>  ??? public Point(double x, double y) {
>>>  ??????? this.x = x;
>>>  ??????? this.y = y;
>>>  ??? }
>>>
>>>  ??? public matcher Point(double x, double y) {
>>>  ??????? x = this.x;
>>>  ??????? y = this.y;
>>>  ??? }
>>> }
>>> ```
>>>
>>> This example illustrates two aspects of the duality between constructors and
>>> their corresponding deconstructors.? Their APIs are duals: a constructor
>>> takes N
>>> parameters containing the desired description of the object state and
>>> produces a
>>> constructed object; a deconstructor starts from the constructed object
>>> and has N
>>> bindings (outputs) that receive the desired state components. Similarly,
>>> their
>>> implementations are duals: the body of the constructor initializes the
>>> object
>>> representation from the description, and the body of the deconstructor
>>> extracts
>>> the description from the representation.? A deconstructor is best
>>> understood as
>>> a _co-constructor_.
>>>
>>> The `Point` example above is special in two ways.? First, the internal
>>> representation of a `Point`, and the API of the constructor and
>>> deconstructor,
>>> are the same: `(double x, double y)`.? We can call the API implied by the
>>> constructor and deconstructor the _external representation_, and for
>>> `Point`,
>>> both the internal and external representations are the same. (This is one of
>>> the requirements for being a candidate to be a record.)? And second, the
>>> constructor is _total_; it does not reject any combinations of arguments.
>>>
>>> Here's another version of `Point` which does not have these special
>>> aspects; it
>>> uses the same internal representation as before, but chooses a pair of
>>> strings
>>> as the external representation:
>>>
>>> ```
>>> class Point2 {
>>>  ??? final double x, y;
>>>
>>>  ??? public Point2(String x, String y) {
>>>  ??????? this.x = Double.parseDouble(x);
>>>  ??????? this.y = Double.parseDouble(y);
>>>  ??? }
>>>
>>>  ??? public matcher Point2(String x, String y) {
>>>  ??????? x = Double.toString(this.x);
>>>  ??????? y = Double.toSTring(this.y);
>>>  ??? }
>>> }
>>> ```
>>>
>>> The method `Double::parseDouble` will throw `NumberFormatException` if its
>>> argument does not describe a suitable value, so unlike the `Point`
>>> constructor,
>>> the `Point2` constructor is partial: it will reject `new Double("foo",
>>> "bar")`.
>>> And the internal representation is no longer the same as the external
>>> representation.? Less obviously, there are valid string values that we can
>>> provide to the constructor, but which cannot be represented exactly as
>>> `double`,
>>> and which will be approximated; the string value
>>> `"3.22222222222222222222222222222222222222"` will be approximated with the
>>> double value `3.2222222222222223`.
>>>
>>> This example highlights more clearly how the deconstructor and
>>> constructor form
>>> an _embedding-projection pair_ between the external and internal
>>> representations.? While some external representations might be invalid,
>>> and some
>>> might result in approximation, deconstruct-then-construct is always an
>>> identity
>>> transformation.? (Indeed, the specification of `java.lang.Record`
>>> requires that
>>> if we deconstruct a record with its accessors, and pass the resulting values
>>> back to the constructor, we should get a new record that is `equals` to the
>>> original.)
>>>
>>> The fact that deconstructor and constructor (and eventually, static
>>> matcher and
>>> factory) form an embedding-projection pair is why we are able to derive
>>> higher-level language features, such as [safer
>>> serialization](https://openjdk.org/projects/amber/design-notes/towards-better-serialization)
>>> and [functional transformation of immutable
>>> objects](https://urldefense.com/v3/__https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md__;!!ACWV5N9M2RV99hQ!M4HQ_Wx5MQZEQwUavg6-Kasr5GKMBzQiac4CoRAmbenb1xcNnDCGwICoGzsX058e1uIwrncE7-Q5ZsjLZu9k$
>>> ),
>>> from a matched set of constructor and deconstructor.
>>>
>>> Of course, users are free to implement constructors without
>>> deconstructors, or
>>> constructors and deconstructors whose external representations don't
>>> match up,
>>> or even matching constructors and deconstructors that are not
>>> behaviorally dual.
>>> But providing a matched set (or several) of constructors and deconstructors
>>> enables reliably reversible aggregation, and allows us to mechanically
>>> derive
>>> useful higher-level features such as withers.
>>>
>>> #### Overloading
>>>
>>> Just as constructors can be overloaded, deconstructors can be overloaded
>>> for the
>>> same reason: multiple constructors can expose multiple external
>>> representations
>>> for aggregation, and corresponding deconstructors can recover those multiple
>>> external representations.? Any matching pair of
>>> deconstructor-constructor (and
>>> eventually, deconstructor-factory) is a candidate for use in higher-level
>>> features based on the embedding-projection nature of the
>>> deconstructor-constructor pair.
>>>
>>> Just as deconstruction is dual to construction, overloading of
>>> deconstructors is
>>> dual to that of constructors: rather than restricting which sets of
>>> parameters
>>> can be overloaded against each other, we do so with the bindings
>>> instead.? For
>>> constructors we require that their signatures not be
>>> override-equivalent; for
>>> deconstructors, we require the same of their bindings.
>>>
>>> For a deconstructor (and declared patterns in general), we derive a _binding
>>> signature_ (and an erased _binding descriptor_) which treats the binding
>>> list as
>>> if it were a parameter list.? The overload rule outlined above requires that
>>> binding signatures for two deconstructors of the same arity not be
>>> override-equivalent. (We will find it useful later to derive a
>>> `MethodType` for
>>> the binding descriptor; this is a `MethodType` whose parameter types are the
>>> erased types of the bindings.)
>>>
>>> #### Digression: embedding-projection pairs
>>>
>>> Given two sets _A_ and _B_, a pair of functions `e : A -> B` and `p : B
>>> -> A`,
>>> forms an _embedding-projection pair_ if embed-then-project is an identity
>>> function, and project-then-embed _approximates_ the input according to a
>>> domain-specific approximation metric.? In our example which used strings
>>> as an external representation, projection rejected some inputs (`"foo"`) and
>>> approximated others.
>>>
>>> When applied to deconstructor-constructor pairs, this says that
>>> deconstructing
>>> an object and then reconstructing it with the resulting bindings should
>>> result
>>> in an equivalent object, and constructing an object from an external
>>> representation and then deconstructing it back into that external
>>> representation
>>> should result in an approximation of the original external representation,
>>> assuming nothing bad happens during construction such as rejecting a bad
>>> input.
>>>
>>> Embedding-projection pairs have a number of desirable properties, such
>>> as the
>>> composition of two e-p pairs is an e-p pair; this property is at the
>>> heart of
>>> using deconstructor-constructor pairs for improved serialization and
>>> functional
>>> transformation.
>>>
>>> ## Invoking deconstructors
>>>
>>> We've already seen how to "invoke" deconstructors: through pattern matching.
>>> What we've been calling "record patterns" are merely deconstruction patterns
>>> derived mechanically from the state description of a record, just as we
>>> do with
>>> constructors and accessors; there is little difference between record
>>> patterns
>>> and deconstruction patterns other than the ability to declare them
>>> explicitly.
>>> (There is an accidental difference in the translation, in that we currently
>>> implement record patterns by appealing to individual accessors rather than a
>>> single deconstructor, but this may eventually converge as well.)
>>>
>>> The use-site syntax of deconstruction bears a deliberate similarity to
>>> that of
>>> construction; `new Point(x, y)` is deconstructed by `case Point(var x,
>>> var y)`.
>>>
>>> #### Overload selection
>>>
>>> In the presence of overloaded deconstructors, we need to figure out which
>>> deconstructor a deconstruction pattern `C(P*)` is referring to. The
>>> details are
>>> similar to overload selection for methods, except that we operate on the
>>> bindings rather than the parameters.? We first search for _applicable
>>> matchers_,
>>> using increasingly loose criteria (first excluding boxing, unboxing, and
>>> varargs; then including boxing and unboxing but not varargs; and
>>> finally, all
>>> candidates) and then selecting the most applicable.
>>>
>>> It is tempting to try and bypass the three-phase selection process and use a
>>> simpler notion of applicability (perhaps noting that we got this process for
>>> compatibility with existing overload selection decisions when autoboxing and
>>> varargs were added, and that there are few deconstructor invocations to be
>>> compatible with yet.)? But because existing overloaded constructors use this
>>> mechanism, and there is significant value in pairing constructors and
>>> deconstructors, attempting to invent a simpler-but-different overload
>>> selection
>>> mechanism for deconstructors would inevitably undermine the duality between
>>> matching deconstructor-constructor pairs. So compatibility (this time, with
>>> existing overloaded constructors) once again forces our hand.
>>>
>>> The specification for overload selection is complicated significantly by
>>> poly
>>> expressions (e.g., lambdas); fortunately, there are no "poly patterns",
>>> and so,
>>> while the structure of JLS 15.12.2 is retained for overload selection of
>>> deconstruction patterns, much of the detail is left behind.
>>>
>>> ## Translation
>>>
>>> We translate patterns into synthetic methods with a `Matcher` attribute;
>>> this
>>> method implements the matcher behavior.? The translation scheme derives
>>> from a
>>> number of requirements, only some of which are in play for deconstructors.
>>>
>>> The matcher method for a deconstructor is a final instance method that
>>> takes no
>>> parameters and returns `Object`, perhaps with a special name (just as
>>> constructors are called `<init>`.)
>>>
>>> #### Carriers
>>>
>>> Because the matcher methods implements the matcher behavior, but a
>>> matcher may
>>> "return" multiple bindings (or failure), we must encode the bindings in some
>>> way.? For this, we use a _carrier object_.? The choice of carrier is
>>> largely a
>>> footprint/specificity tradeoff.? One could imagine a carrier class per
>>> matcher,
>>> or a carrier class per matcher descriptor, or using `Object[]` as a
>>> carrier for
>>> everything, or caching some number of common shapes (e.g, three ints and two
>>> refs).? This sort of tuning should be separate from the protocol encoded
>>> in the
>>> bytecode of the pattern method and its clients.
>>>
>>> We use a small _carrier runtime_ to decouple pattern translation from
>>> carrier
>>> selection.? (This same carrier runtime is used by string templates as well.)
>>> This allows tradeoffs in runtime characteristics (e.g., carrier per
>>> matcher vs
>>> sharing carriers across matchers, dropping carrier identity with value types
>>> later, etc) without affecting the translation. The carrier API consists
>>> of condy
>>> bootstraps like:
>>>
>>> ```
>>> static MethodHandle carrierFactory(METADATA,
>>>  ?????????????????????????????????? MethodType matcherDescriptor) { ... }
>>> static MethodHandle carrierAccessor(METADATA,
>>>  ??????????????????????????????????? MethodType matcherDescriptor,
>>>  ??????????????????????????????????? int bindingNo) { ... }
>>> ```
>>>
>>> where `METADATA` are the standard condy metadata arguments (lookup,
>>> name, type.)
>>> The `matcherDescriptor` is a `MethodType` describing the binding types.? The
>>> `carrierFactory` method returns a method handle which takes the bindings and
>>> produces a carrier object; the `carrierAccessor` method returns method
>>> handles
>>> that take the carrier object and return the corresponding binding.? To
>>> indicate
>>> success, the matcher method invokes the carrier factory method handle and
>>> returns the result; to indicate failure (deconstructors cannot fail, but
>>> other
>>> matchers can) the matcher method returns null.
>>>
>>> We would translate the XY deconstructor from `Point` as follows
>>> (pseudo-bytecode):
>>>
>>> ```
>>> #100: MethodType[(II)V]
>>> #101: Condy[bsm=Carriers::carrierFactory, args=[#100]]
>>>
>>> final synthetic Object Point$MANGLE() {
>>>  ??? aload_0
>>>  ??? getfield Point::x
>>>  ??? aload_0
>>>  ??? getfield Point::y
>>>  ??? LDC #101
>>>  ??? invokevirtual MethodHandle::invoke(II)Object
>>>  ??? areturn
>>> }
>>> ```
>>>
>>> Constant `#100` contains a `MethodType` holding the binding descriptor;
>>> constant
>>> `#101` holds a method handle whose parameters are the parameter types of the
>>> binding descriptor and returns `Object`.
>>>
>>> At the use site, matching a deconstruction pattern is performed by
>>> invoking the
>>> matcher method on the appropriate target object, and then extracting the
>>> components with the carrier accessor method handles if the match is
>>> successful.
>>> (Deconstructors are total, so are always successful, but for other patterns,
>>> null is returned from the matcher method on failure to match.)
>>>
>>> #### Method names
>>>
>>> The name of the matcher method is mangled to support overloading. The JVM
>>> permits overloading on parameter types, but not return types (and overloaded
>>> matchers are effectively overloaded on return types.)? We take the
>>> approach of
>>> encoding the erasure of the matcher descriptor in the name of the
>>> pattern.? This
>>> has several desirable properties: it is stable (the name is derived
>>> solely from
>>> stable aspects of the declaration), for matchers with override-equivalent
>>> signatures (deconstructors can't be overridden, but other patterns can be),
>>> these map to true overrides in the translation, and valid overloads of
>>> matchers
>>> will always have distinct names.
>>>
>>> We use the ["Symbolic Freedom"]() encoding of the erasure of the matcher
>>> descriptor as the mangled disambiguator, which is exactly as stable as
>>> any other
>>> method descriptor derived from source declarations.
>>>
>>> #### Attributes
>>>
>>> Because patterns are methods, we can take advantage of all the
>>> affordances of
>>> methods.? We can use access bits to control accessibility; we can use the
>>> attributes that carry annotations, method parameter metadata, and generics
>>> signatures to carry information about the pattern declaration (and its
>>> (input)
>>> parameters, when we get to those kinds of matchers).? What's missing is
>>> the fact
>>> that this is a pattern implementation and not an ordinary method, and a
>>> place to
>>> put metadata for bindings.? To address the first, we can add the following
>>> attribute on matcher methods:
>>>
>>>  ??? Matcher {
>>>  ??????? u2 name;??????????????????????????? // "Matcher"
>>>  ??????? u4 length;
>>>  ??????? u2 patternFlags;
>>>  ??????? u2 patternName;???????????????????? // UTF8
>>>  ??????? u2 patternDescr;??????????????????? // MethodType
>>>  ??????? u2 attributes_count;
>>>  ??????? attribute_info attributes[attributes_count];
>>>  ??? }
>>>
>>> This says that "this method is a pattern".? The source name of the pattern
>>> declaration is reified as `patternName`, and the matcher descriptor, which
>>> encodes the types of the bindings, is reified as a `MethodType` in
>>> `patternDescr`.? The `flags` word can carry matcher-specific information
>>> such as
>>> "this matcher is a deconstructor" or "this matcher is total".
>>>
>>> A matcher method may have the usual variety of method attributes, such as
>>> `RuntimeInvisibleAnnotations` for annotations on the matcher declaration
>>> itself.
>>>
>>> If we wish to encode information about the matcher _bindings_, we do so with
>>> attributes inside the `Matcher` annotation itself.? Attributes such as
>>> `Signature`, `ParameterNames`, `RuntimeVisibleParameterAnnotations`,
>>> etc, can
>>> appear in a `Matcher` and are interpreted relative to the matcher
>>> signature or
>>> descriptor.? So if we had a matcher:
>>>
>>> ```
>>> matcher Foo(@Bar List<String> list) { ... }
>>> ```
>>>
>>> then the `Matcher` would contain a signature attribute corresponding to
>>> `(List<String>)` and a `RuntimeXxxParameterAnnotations` attribute
>>> describing the
>>> `@Bar` annotation on the first "parameter".
>>>
>>> #### Reflection
>>>
>>> Since matchers are a new kind of class member, they will need a new kind of
>>> reflective object, and a method that is analogous to
>>> `Class::getConstructors`.
>>> The reflective object for matchers should extend `Executable`, as all of the
>>> existing methods on `Executable` make sense for patterns.? If the pattern is
>>> reflectively invoked, it returns `null` for no match, or an `Object[]`
>>> which is
>>> the boxing of the values in the carrier.
>>>
>>> We will then need some additional methods to describe the bindings, so the
>>> subtype of `Executable` has methods like `getBindings`,
>>> `getAnnotatedBindings`,
>>> `getGenericBindings`, `isDeconstructor`, `isPartial`, etc.? These
>>> methods will
>>> decode the `Matcher` attribute and its embedded attributes.
>>>
>>> ## Summary
>>>
>>> This design borrows from previous rounds, but makes a number of
>>> simplifications.
>>>
>>>  ?- The bindings of a pattern are captured in a `MethodType`, called the
>>> _matcher
>>>  ?? descriptor_.? The parameters of the matcher descriptor are the types
>>> of the
>>>  ?? bindings; the return type is either `V` or the minimal type that
>>> will match
>>>  ?? (but is not as important as the bindings.)
>>>  ?- Matchers are translated as methods whose names are derived
>>> deterministically
>>>  ?? from the name of the matcher and the erasure of the pattern
>>> descriptor. These
>>>  ?? are called _matcher methods_.? Matcher methods take as parameters
>>> the input
>>>  ?? parameters of the pattern (if any), and return `Object`.
>>>  ?- The returned object is an opaque carrier.? Null means the pattern didn't
>>>  ?? match.? A non-null value is the carrier type (from the carrier
>>> runtime) which
>>>  ?? is derived from the pattern descriptor.
>>>  ?- Matcher methods are not directly invocable from the source language;
>>> they are
>>>  ?? invoked indirectly through pattern matching or reflection.
>>>  ?- Generated code invokes the matcher method and interprets the
>>> returned value
>>>  ?? according to the protocol, using MHs from the carrier runtime to
>>> access the
>>>  ?? bindings.
>>>  ?- Matcher methods have a `Matcher` attribute, which captures
>>> information about
>>>  ?? the matcher as a whole (is a total/partial, a deconstructor, etc) and
>>>  ?? parameter-related attributes which describe the bindings.
>>>  ?- Matchers are reflected through a new subtype of `Executable`, which
>>> exposes
>>>  ?? new methods to reflect over bindings.
>>>  ?- When invoking a matcher reflectively, the carrier is boxed to an
> >> `Object[]`.

From james.laskey at oracle.com  Fri Mar 17 13:24:47 2023
From: james.laskey at oracle.com (Jim Laskey)
Date: Fri, 17 Mar 2023 13:24:47 +0000
Subject: StringTemplates name changes
Message-ID: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>

This is a  heads up about some name changes coming to the string template feature with the intent of eliminating the ?java.lang.template? package along with clarifying the processor hierarchy,

Old     New
java.lang.template.Carriers*    java.lang.runtime.Carriers*
java.lang.template.ReferencedKeyMap*    java.lang.runtime.ReferencedKeyMap*
java.lang.template.ReferenceKey*        java.lang.runtime.ReferenceKey*
java.lang.template.StringTemplateImpl*  java.lang.runtime.StringTemplateImpl*
java.lang.template.StringTemplateImplFactory*   java.lang.runtime.StringTemplateImplFactory*
java.lang.runtime.TemplateRuntime       java.lang.runtime.TemplateRuntime
java.lang.template.TemplateSupport*     java.lang.runtime.TemplateSupport
java.lang.template.StringTemplate       java.lang.StringTemplate
java.lang.template.ValidatingProcessor  java.lang.StringTemplate.Processor
java.lang.template.ProcessorLinkage     java.lang.StringTemplate.Processor.Linkage
java.lang.template.TemplateProcessor    java.lang.StringTemplate.SimpleProcessor
java.lang.template.StringProcessor      java.lang.StringTemplate.StringProcessor

(*) - package private


The new processor hierarchy will be;

interface Processor<R, E>
interface SimpleProcessor<R> extends Processor<R, RuntimeException>
interface StringProcessor extends SimpleProcessor<String>

It will take me a few days to update the JEP, CSRs, PR and JLS, so stay tuned. As always, comments are welcome.

Cheers,

? Jim

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230317/9d56b32b/attachment.htm>

From gavin.bierman at oracle.com  Thu Mar 23 16:29:46 2023
From: gavin.bierman at oracle.com (Gavin Bierman)
Date: Thu, 23 Mar 2023 16:29:46 +0000
Subject: StringTemplates name changes
In-Reply-To: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
References: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
Message-ID: <C02C7A49-3CF8-4F43-9E70-32AD8C830B58@oracle.com>

I have updated the spec change document: https://cr.openjdk.org/~gbierman/jep430/latest/

Thanks,
Gavin

On 17 Mar 2023, at 13:24, Jim Laskey <james.laskey at oracle.com> wrote:

This is a  heads up about some name changes coming to the string template feature with the intent of eliminating the ?java.lang.template? package along with clarifying the processor hierarchy,

Old     New
java.lang.template.Carriers*    java.lang.runtime.Carriers*
java.lang.template.ReferencedKeyMap*    java.lang.runtime.ReferencedKeyMap*
java.lang.template.ReferenceKey*        java.lang.runtime.ReferenceKey*
java.lang.template.StringTemplateImpl*  java.lang.runtime.StringTemplateImpl*
java.lang.template.StringTemplateImplFactory*   java.lang.runtime.StringTemplateImplFactory*
java.lang.runtime.TemplateRuntime       java.lang.runtime.TemplateRuntime
java.lang.template.TemplateSupport*     java.lang.runtime.TemplateSupport
java.lang.template.StringTemplate       java.lang.StringTemplate
java.lang.template.ValidatingProcessor  java.lang.StringTemplate.Processor
java.lang.template.ProcessorLinkage     java.lang.StringTemplate.Processor.Linkage
java.lang.template.TemplateProcessor    java.lang.StringTemplate.SimpleProcessor
java.lang.template.StringProcessor      java.lang.StringTemplate.StringProcessor

(*) - package private


The new processor hierarchy will be;

interface Processor<R, E>
interface SimpleProcessor<R> extends Processor<R, RuntimeException>
interface StringProcessor extends SimpleProcessor<String>

It will take me a few days to update the JEP, CSRs, PR and JLS, so stay tuned. As always, comments are welcome.

Cheers,

? Jim


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230323/37ff0908/attachment-0001.htm>

From james.laskey at oracle.com  Thu Mar 23 16:36:39 2023
From: james.laskey at oracle.com (Jim Laskey)
Date: Thu, 23 Mar 2023 16:36:39 +0000
Subject: StringTemplates name changes
In-Reply-To: <C02C7A49-3CF8-4F43-9E70-32AD8C830B58@oracle.com>
References: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
 <C02C7A49-3CF8-4F43-9E70-32AD8C830B58@oracle.com>
Message-ID: <4BB34B63-6F8C-4475-AE92-F5120676F0EE@oracle.com>

Thank you.

On Mar 23, 2023, at 1:29 PM, Gavin Bierman <gavin.bierman at oracle.com> wrote:

I have updated the spec change document: https://cr.openjdk.org/~gbierman/jep430/latest/

Thanks,
Gavin

On 17 Mar 2023, at 13:24, Jim Laskey <james.laskey at oracle.com> wrote:

This is a  heads up about some name changes coming to the string template feature with the intent of eliminating the ?java.lang.template? package along with clarifying the processor hierarchy,

Old     New
java.lang.template.Carriers*    java.lang.runtime.Carriers*
java.lang.template.ReferencedKeyMap*    java.lang.runtime.ReferencedKeyMap*
java.lang.template.ReferenceKey*        java.lang.runtime.ReferenceKey*
java.lang.template.StringTemplateImpl*  java.lang.runtime.StringTemplateImpl*
java.lang.template.StringTemplateImplFactory*   java.lang.runtime.StringTemplateImplFactory*
java.lang.runtime.TemplateRuntime       java.lang.runtime.TemplateRuntime
java.lang.template.TemplateSupport*     java.lang.runtime.TemplateSupport
java.lang.template.StringTemplate       java.lang.StringTemplate
java.lang.template.ValidatingProcessor  java.lang.StringTemplate.Processor
java.lang.template.ProcessorLinkage     java.lang.StringTemplate.Processor.Linkage
java.lang.template.TemplateProcessor    java.lang.StringTemplate.SimpleProcessor
java.lang.template.StringProcessor      java.lang.StringTemplate.StringProcessor

(*) - package private


The new processor hierarchy will be;

interface Processor<R, E>
interface SimpleProcessor<R> extends Processor<R, RuntimeException>
interface StringProcessor extends SimpleProcessor<String>

It will take me a few days to update the JEP, CSRs, PR and JLS, so stay tuned. As always, comments are welcome.

Cheers,

? Jim



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230323/a9d4e76a/attachment.htm>

From angelos.bimpoudis at oracle.com  Fri Mar 24 14:41:14 2023
From: angelos.bimpoudis at oracle.com (Angelos Bimpoudis)
Date: Fri, 24 Mar 2023 14:41:14 +0000
Subject: Draft JLS Spec about unnamed patterns and variables
In-Reply-To: <ad1cf6ac-7888-6a24-2188-21f9e3156ba7@oracle.com>
References: <SA2PR10MB4667ADD0F0F8D862E22ED17D82CF9@SA2PR10MB4667.namprd10.prod.outlook.com>
 <6a57f58c-6f91-612b-ec48-9213e85e168c@oracle.com>
 <SA2PR10MB46675D8D0413CDB8D7CD009582AA9@SA2PR10MB4667.namprd10.prod.outlook.com>
 <e9ea2e8b-ac1d-5182-271d-19ad38041f11@oracle.com>
 <d0202a80-765d-f9f7-3765-190f3d389332@oracle.com>
 <fbda0407-77b8-39ae-c59a-657ccfa22c30@oracle.com>
 <SA2PR10MB4667E64A6F232DC5E78EA05C82A89@SA2PR10MB4667.namprd10.prod.outlook.com>
 <SA2PR10MB4667FD3C36E5EAB9DAF669B882AC9@SA2PR10MB4667.namprd10.prod.outlook.com>
 <ad1cf6ac-7888-6a24-2188-21f9e3156ba7@oracle.com>
Message-ID: <SA2PR10MB46679B16AFFDEECEDD3AD42E82849@SA2PR10MB4667.namprd10.prod.outlook.com>

Thanks for your comments. I updated our draft with unnamed patterns actually getting resolved now. They weren't before.
`when (Boolean) null` -- yuck :)  Well spotted.
What did you mean by this? ? (Boolean)? is not a constant expression.

Updated version: https://cr.openjdk.org/~abimpoudis/unnamed/latest/

(this points to a new directory that names the JEP number, I updated our "latest" redirect script, but I had to clear the cache of my browser to finally see it! :O, just in case it occurs to others too)
________________________________
From: Brian Goetz <brian.goetz at oracle.com>
Sent: 28 February 2023 21:09
To: Angelos Bimpoudis <angelos.bimpoudis at oracle.com>; amber-spec-experts <amber-spec-experts at openjdk.java.net>
Subject: Re: Draft JLS Spec about unnamed patterns and variables

In 14.11.1, you say:

For a case label with case patterns, it is a compile-time error if any of its case patterns declares one or more pattern variables.

But don't you really mean "For a case label with _more than one_ case patterns, it is ..."?

I could see this being implicit in the "s" at the end of "case patterns", but this would be a pretty subtle distinction.

The following text has a typo: "more than case patterns" -> "more than ONE case pattern".

I see that you've switched horses on domination; now any "dead pattern" is an error.  I am fine with this (though Gavin did make a good argument for the alternative.)

In 14.11.1.2, we talk about resolution of patterns.  If any of the patterns resolves to an any pattern, then it will dominate the others.  I believe this is already handled by the text about "if one dominates the others" so that case is already handled, good.

You commit here to strict left-to-right evaluation of patterns when there are multiple patterns on a case.  This is reasonable, but bear in mind that such L2R commitments do potentially interfere with folding optimizations if any of the patterns involve imperative code such as accessors or deconstructors.

`when (Boolean) null` -- yuck :)  Well spotted.

The text around any patterns and resolution in 14.30.1 seems like it could still be simplified a bit.







On 2/28/2023 11:21 AM, Angelos Bimpoudis wrote:
Updated draft spec.

Includes a small number of fixes and a correct "rebase" on top of the JLS changes resulting from JEP 432 (Record Patterns (Second Preview)) and JEP 433 (Pattern Matching for switch (Fourth Preview)) that covers Maurizio's point.

https://cr.openjdk.org/~abimpoudis/unnamed/latest/

(please note that the based URL has been slightly changed ^^)

Comments are always very much welcomed!

Best,
Angelos

________________________________
From: Angelos Bimpoudis <angelos.bimpoudis at oracle.com><mailto:angelos.bimpoudis at oracle.com>
Sent: 24 February 2023 17:23
To: Maurizio Cimadamore <maurizio.cimadamore at oracle.com><mailto:maurizio.cimadamore at oracle.com>; Brian Goetz <brian.goetz at oracle.com><mailto:brian.goetz at oracle.com>; amber-spec-experts <amber-spec-experts at openjdk.java.net><mailto:amber-spec-experts at openjdk.java.net>
Subject: Re: Draft JLS Spec about unnamed patterns and variables


The main takeaways:

  *   case Number _ can fall out to other patterns since it doesn't introduce any bindings (adjustment in the spec draft is needed)
  *   case Number _ dominates case String _, Integer _ (adjustment is not needed in current draft, but I will double check before I circulate the revised version).

Thanks for all the comments!
Thought experiment: what if we had union type patterns?  Then the case label `case String _, Integer _` would be like matching the the union type pattern `(String|Integer) _`:

    case Number n: ...
    case (String|Integer) _: ...

Would javac then complain that `String|Integer` could be simplified to just `String` on the bsais of flow analysis?  (IntelliJ would, of course.)

I initially thought as Tagir did, but then Gavin turned me around and reminded me that it was not dead code, but unreachable statements that we try to avoid.  So now I am torn...
Would union type patterns imply the existence of union types? If yes, then, the second case could even exist with a binding, correct? In your example the LUB is Object so even the case (String|Integer) x : x.getClass() can work. The difficult scenario would arise with the case (Customer|Human) x : x.getName();

If the first case in your example did not introduce a binding, would both case be equal with Number | (String | Integer)? Union types a la Ceylon support this (http://web.mit.edu/ceylon_v1.3.3/ceylon-1.3.3/doc/en/spec/html_single/#uniontypes). On the other hand in Ceylon, the switch needs to be exhaustive and? all cases need to be disjoint. So this switch would be invalid. hm..

________________________________
From: Maurizio Cimadamore <maurizio.cimadamore at oracle.com><mailto:maurizio.cimadamore at oracle.com>
Sent: 23 February 2023 20:27
To: Brian Goetz <brian.goetz at oracle.com><mailto:brian.goetz at oracle.com>; Angelos Bimpoudis <angelos.bimpoudis at oracle.com><mailto:angelos.bimpoudis at oracle.com>; amber-spec-experts <amber-spec-experts at openjdk.java.net><mailto:amber-spec-experts at openjdk.java.net>
Subject: Re: Draft JLS Spec about unnamed patterns and variables



On 23/02/2023 18:46, Brian Goetz wrote:
but we really wanted the case merging.

Gotcha.

I just wanted to point out that there are two questions here (one about fall-through and one about domination), and when reading the emails it was not obvious to me that a change in how fall-through was defined was being proposed.

If merging unrelated type tests is a goal, I think there should be an example for it in the JEP under "Motivation".

Maurizio

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230324/904176f8/attachment-0001.htm>

From daniel.smith at oracle.com  Fri Mar 24 22:00:21 2023
From: daniel.smith at oracle.com (Dan Smith)
Date: Fri, 24 Mar 2023 22:00:21 +0000
Subject: Record pattern inference & capture
Message-ID: <05E51BDD-F0BA-481E-A73B-5C5A4EC49ECB@oracle.com>

The record pattern inference strategy introduced in 20 attempts to map a pattern's match type to a parameterization of a given generic record class (typically a subclass of the match type).

One of its capabilities is to interpret a "duplicated" type argument?as illustrated in the proposed JLS 18.5.5 text, for example, the type Function<Foo,Foo> can be mapped to a type like UnaryOperator<Foo>.

Another of its capabilities is an inference-based approach to wildcards, so that a match type like Function<? extends Foo, ? extends Bar> can map to something like UnaryOperator<? extends Foo & Bar>:

record Mapper<T>(T in, T out) implements UnaryOperator<T> {
    T apply(T arg) { return in.equals(arg) ? out : null; }
}

void test(Function<? super String, ? extends CharSequence> f) {
    if (f instanceof Mapper(var in, var out)) {
        boolean shorter = out.length() < in.length();
    }
}

Unfortunately, this strategy doesn't account for the fact that the match type should probably be captured before doing anything with it. (JLS is a little fuzzy in this when it comes to variable references, but in general every use/read of a variable or method needs to be captured before further typing occurs. Setting aside some poorly-specified cases, it's easy to come up with examples, like a method invocation (see JLS 15.12.3) where the wildcards definitively get captured.)

Nor does it account for the fact that capture variables may appear in the match type because they've flowed out of upstream expressions like method calls. (Think 'Map<?,?>.entrySet().iterator().next()'.)

If I try to match the type Function<CAP1,CAP2> with a type like UnaryOperator, should that be an inference failure? The current rules say "yes": inference variable alpha=CAP1, and alpha=CAP2, a contradiction. But I don't think that's right. If the dynamic check for UnaryOperator succeeds, that means that this must actually be a Function in which the actual types represented by CAP1 and CAP2 are the same.

My conclusion is that the inference treatment of wildcards ought to apply to capture variables, too. This is justified by the fact that while usually reasoning about capture variables works by assuming "there exists some type with these properties, don't assume anything else about it", in this case we also want to incorporate the fact that the dynamic pattern-matching check did, in fact, succeed. (However, I want to validate this thinking, because my confidence isn't 100%.) 

Here's how I think step #3 of the proposed JLS 18.5.5 should read:

-----

A type T' is derived from T, as follows:

-   If T is a parameterized type, let T_cap be the result of capture conversion (5.1.10) applied to T, and let Z1, ..., Zk (k ? 0) be the type variables produced by capture that are type arguments in T_cap. (This includes type variables produced by the capture conversion in this step, and type variables produced by capture conversion elsewhere.) Let ?1, ..., ?k (k ? 0) be inference variables, and let ? be the substitution [Z1:=?1, ..., Zk:=?k]. T' is T_cap ?.

    Additional bounds for ?1, ..., ?k are incorporated into B0 to form a bound set B1, as follows:

    - If ?i (1 ? i ? k) replaced a type variable with an upper bound U, then the bound ?i <: U ? appears in the bound set

    - If ?i (1 ? i ? k) replaced a type variable with a lower bound L, then the bound L ? <: ?i appears in the bound set

    - If no proper upper bounds otherwise exist for ?i (1 ? i ? k), the bound ?i <: Object appears in the bound set

- If T is any other class or interface type, then T' is the same as T, and B1 is the same as B0.

- If T is a type variable or an intersection type, then for each upper bound of the type variable or element of the intersection type, this step and step 4 are repeated recursively. All bounds produced in steps 3 and 4 are incorporated into a single bound set.


From james.laskey at oracle.com  Mon Mar 27 13:07:12 2023
From: james.laskey at oracle.com (Jim Laskey)
Date: Mon, 27 Mar 2023 13:07:12 +0000
Subject: StringTemplates name changes
In-Reply-To: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
References: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
Message-ID: <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>

After the string template interface name changes, i.e., TemplateProcessor becoming Processor, the rationale for the existence of SimpleProcessor and StringProcessor has lessened to the point where they should be dropped.

SimpleProcessor owed its existence to the long-winded name TemplateProcessor and that ugly second parameter, E, in Processor<R, E> (in a many of cases E will be the unchecked RuntimeException). StringProcessor existed because most template processors will produce strings.

TemplateProcessor<JSONObject, RuntimeException> JSON = st-> new JSONObject(st.interpolate());
TemplateProcessor<String, RuntimeException> INTER = StringTemplate::interpolate;

vs.

SimpleProcessor<JSONObject> JSON = st-> new JSONObject(st.interpolate());
StringProcessor INTER = StringTemplate::interpolate;

It was thought that having the friendlier interfaces would provide clarity, hide RuntimeException and simplify explanation. The reality is that most developers will define template processors using full class declarations. Furthermore, developers will learn to use RuntimeException regularly due to the abundance of template processor examples.

public class InterpolateProcessor implements Processor<String, RuntimeException> {
    @Override
    public String process(StringTemplate st) {
        return st.interpolate();
    }
}

SimpleProcessor<String> INTER = new InterpolateProcessor();


Even after SimpleProcessor and StringProcessor go away, developers can still use the functional interface shorthand.

Processor<JSONObject, RuntimeException> JSON = st-> new JSONObject(st.interpolate());
Processor<String, RuntimeException> INTER = StringTemplate::interpolate;

And, a new factory method, Processor.of, will be added for fans of var.

var JSON = Processor.of(st-> new JSONObject(st.interpolate()));
var INTER = Processor.of(StringTemplate::interpolate);

For those developers that like the notion of SimpleProcessor and StringProcessor, these interfaces can be trivially defined per project;

@FunctionalInterface
public interface SimpleProcessor<R> extends Processor<R, RuntimeException> {}

@FunctionalInterface
public interface StringProcessor extends SimpleProcessor<String> {}






On Mar 17, 2023, at 10:24 AM, Jim Laskey <james.laskey at oracle.com> wrote:

This is a  heads up about some name changes coming to the string template feature with the intent of eliminating the ?java.lang.template? package along with clarifying the processor hierarchy,

Old     New
java.lang.template.Carriers*    java.lang.runtime.Carriers*
java.lang.template.ReferencedKeyMap*    java.lang.runtime.ReferencedKeyMap*
java.lang.template.ReferenceKey*        java.lang.runtime.ReferenceKey*
java.lang.template.StringTemplateImpl*  java.lang.runtime.StringTemplateImpl*
java.lang.template.StringTemplateImplFactory*   java.lang.runtime.StringTemplateImplFactory*
java.lang.runtime.TemplateRuntime       java.lang.runtime.TemplateRuntime
java.lang.template.TemplateSupport*     java.lang.runtime.TemplateSupport
java.lang.template.StringTemplate       java.lang.StringTemplate
java.lang.template.ValidatingProcessor  java.lang.StringTemplate.Processor
java.lang.template.ProcessorLinkage     java.lang.StringTemplate.Processor.Linkage
java.lang.template.TemplateProcessor    java.lang.StringTemplate.SimpleProcessor
java.lang.template.StringProcessor      java.lang.StringTemplate.StringProcessor

(*) - package private


The new processor hierarchy will be;

interface Processor<R, E>
interface SimpleProcessor<R> extends Processor<R, RuntimeException>
interface StringProcessor extends SimpleProcessor<String>

It will take me a few days to update the JEP, CSRs, PR and JLS, so stay tuned. As always, comments are welcome.

Cheers,

? Jim


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230327/64de2693/attachment-0001.htm>

From brian.goetz at oracle.com  Mon Mar 27 14:18:50 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Mon, 27 Mar 2023 10:18:50 -0400
Subject: StringTemplates name changes
In-Reply-To: <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>
References: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
 <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>
Message-ID: <38a90674-200f-cf56-90a0-387b786a43be@oracle.com>

Nice.

So that means java.lang additions are limited to StringTemplate (with 
nested Processor), and public jl.runtime limited to 
Template{Runtime,Support}.

In the spirit of "nothing left to take away", who are the methods in 
TemplateSupport used by?? Authors of processors? There's not too much 
here -- creation, interpolation, combination.? Would these make sense as 
static helper methods on STringTemplate?? They'd be more discoverable 
there.


On 3/27/2023 9:07 AM, Jim Laskey wrote:
>
> After the string template interface name changes, i.e., 
> |TemplateProcessor|?becoming |Processor|, the rationale for the 
> existence of |SimpleProcessor |and |StringProcessor|?has lessened to 
> the point where they should be dropped.
>
> |SimpleProcessor|?owed its existence to the long-winded name 
> |TemplateProcessor|?and that ugly second parameter, |E|, in 
> |Processor<R, E>|?(in a many of cases |E|?will be the unchecked 
> |RuntimeException|). |StringProcessor|?existed because most template 
> processors will produce strings.
>
> |TemplateProcessor<JSONObject, RuntimeException> JSON = st-> new 
> JSONObject(st.interpolate()); TemplateProcessor<String, 
> RuntimeException> INTER = StringTemplate::interpolate;|
>
> vs.
>
> |SimpleProcessor<JSONObject> JSON = st-> new 
> JSONObject(st.interpolate()); StringProcessor INTER = 
> StringTemplate::interpolate;|
>
> It was thought that having the friendlier interfaces would provide 
> clarity, hide |RuntimeException|?and simplify explanation. The reality 
> is that most developers will define template processors using full 
> class declarations. Furthermore, developers will learn to use 
> |RuntimeException|?regularly due to the abundance of template 
> processor examples.
>
> |public class InterpolateProcessor implements Processor<String, 
> RuntimeException> { @Override public String process(StringTemplate st) 
> { return st.interpolate(); } } SimpleProcessor<String> INTER = new 
> InterpolateProcessor(); |
>
> Even after |SimpleProcessor|?and |StringProcessor|?go away, developers 
> can still use the functional interface shorthand.
>
> |Processor<JSONObject, RuntimeException> JSON = st-> new 
> JSONObject(st.interpolate()); Processor<String, RuntimeException> 
> INTER = StringTemplate::interpolate;|
>
> And, a new factory method, |Processor.of|, will be added for fans of 
> |var|.
>
> |var JSON = Processor.of(st-> new JSONObject(st.interpolate())); var 
> INTER = Processor.of(StringTemplate::interpolate);|
>
> For those developers that like the notion of |SimpleProcessor|?and 
> |StringProcessor|, these interfaces can be trivially defined per project;
>
> |@FunctionalInterface public interface SimpleProcessor<R> extends 
> Processor<R, RuntimeException> {} @FunctionalInterface public 
> interface StringProcessor extends SimpleProcessor<String> {}|
>
>
>
>
>
>
>> On Mar 17, 2023, at 10:24 AM, Jim Laskey <james.laskey at oracle.com> wrote:
>>
>> This is a ?heads up about some name changes coming to the string 
>> template feature with the intent of eliminating the 
>> ?java.lang.template? package along with clarifying the processor 
>> hierarchy,
>>
>> _Old_ 	_New_
>> java.lang.template.Carriers* 	java.lang.runtime.Carriers*
>> java.lang.template.ReferencedKeyMap* 
>> java.lang.runtime.ReferencedKeyMap*
>> java.lang.template.ReferenceKey* 	java.lang.runtime.ReferenceKey*
>> java.lang.template.StringTemplateImpl* 
>> java.lang.runtime.StringTemplateImpl*
>> java.lang.template.StringTemplateImplFactory* 
>> java.lang.runtime.StringTemplateImplFactory*
>> java.lang.runtime.TemplateRuntime 	java.lang.runtime.TemplateRuntime
>> java.lang.template.TemplateSupport* 	java.lang.runtime.TemplateSupport
>> java.lang.template.StringTemplate 	java.lang.StringTemplate
>> java.lang.template.ValidatingProcessor 
>> java.lang.StringTemplate.Processor
>> java.lang.template.ProcessorLinkage 
>> java.lang.StringTemplate.Processor.Linkage
>> java.lang.template.TemplateProcessor 
>> java.lang.StringTemplate.SimpleProcessor
>> java.lang.template.StringProcessor 
>> java.lang.StringTemplate.StringProcessor
>>
>>
>> (*) - package private
>>
>>
>> The new processor hierarchy will be;
>>
>> interface Processor<R, E>
>> interface SimpleProcessor<R> extends Processor<R, RuntimeException>
>> interface StringProcessor extends SimpleProcessor<String>
>>
>> It will take me a few days to update the JEP, CSRs, PR and JLS, so 
>> stay tuned. As always, comments are welcome.
>>
>> Cheers,
>>
>> ? Jim
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230327/5d53351b/attachment-0001.htm>

From forax at univ-mlv.fr  Mon Mar 27 14:38:18 2023
From: forax at univ-mlv.fr (Remi Forax)
Date: Mon, 27 Mar 2023 16:38:18 +0200 (CEST)
Subject: StringTemplates name changes
In-Reply-To: <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>
References: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
 <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>
Message-ID: <919117086.21028495.1679927898627.JavaMail.zimbra@univ-eiffel.fr>

+1 from me. 

It makes the spec less intimidating for users. 

R?mi 

> From: "Jim Laskey" <james.laskey at oracle.com>
> To: "amber-spec-experts" <amber-spec-experts at openjdk.java.net>
> Sent: Monday, March 27, 2023 3:07:12 PM
> Subject: Re: StringTemplates name changes

> After the string template interface name changes, i.e., TemplateProcessor
> becoming Processor , the rationale for the existence of SimpleProcessor and
> StringProcessor has lessened to the point where they should be dropped.

> SimpleProcessor owed its existence to the long-winded name TemplateProcessor and
> that ugly second parameter, E , in Processor<R, E> (in a many of cases E will
> be the unchecked RuntimeException ). StringProcessor existed because most
> template processors will produce strings.
> TemplateProcessor<JSONObject, RuntimeException> JSON = st-> new
> JSONObject(st.interpolate());
> TemplateProcessor<String, RuntimeException> INTER = StringTemplate::interpolate;

> vs.
> SimpleProcessor<JSONObject> JSON = st-> new JSONObject(st.interpolate());
> StringProcessor INTER = StringTemplate::interpolate;

> It was thought that having the friendlier interfaces would provide clarity, hide
> RuntimeException and simplify explanation. The reality is that most developers
> will define template processors using full class declarations. Furthermore,
> developers will learn to use RuntimeException regularly due to the abundance of
> template processor examples.
> public class InterpolateProcessor implements Processor<String, RuntimeException>
> {
>     @Override
>     public String process(StringTemplate st) {
>         return st.interpolate();
>     }
> }

> SimpleProcessor<String> INTER = new InterpolateProcessor();

> Even after SimpleProcessor and StringProcessor go away, developers can still use
> the functional interface shorthand.
> Processor<JSONObject, RuntimeException> JSON = st-> new
> JSONObject(st.interpolate());
> Processor<String, RuntimeException> INTER = StringTemplate::interpolate;

> And, a new factory method, Processor.of , will be added for fans of var .
> var JSON = Processor.of(st-> new JSONObject(st.interpolate()));
> var INTER = Processor.of(StringTemplate::interpolate);

> For those developers that like the notion of SimpleProcessor and StringProcessor
> , these interfaces can be trivially defined per project;
> @FunctionalInterface
> public interface SimpleProcessor<R> extends Processor<R, RuntimeException> {}

> @FunctionalInterface
> public interface StringProcessor extends SimpleProcessor<String> {}

>> On Mar 17, 2023, at 10:24 AM, Jim Laskey <james.laskey at oracle.com> wrote:

>> This is a heads up about some name changes coming to the string template feature
>> with the intent of eliminating the ?java.lang.template? package along with
>> clarifying the processor hierarchy,

>> 	Old 	New
>> 	java.lang.template.Carriers* 	java.lang.runtime.Carriers*
>> 	java.lang.template.ReferencedKeyMap* 	java.lang.runtime.ReferencedKeyMap*
>> 	java.lang.template.ReferenceKey* 	java.lang.runtime.ReferenceKey*
>> 	java.lang.template.StringTemplateImpl* 	java.lang.runtime.StringTemplateImpl*
>>	java.lang.template.StringTemplateImplFactory*
>> 		java.lang.runtime.StringTemplateImplFactory*
>> 	java.lang.runtime.TemplateRuntime 	java.lang.runtime.TemplateRuntime
>> 	java.lang.template.TemplateSupport* 	java.lang.runtime.TemplateSupport
>> 	java.lang.template.StringTemplate 	java.lang.StringTemplate
>> 	java.lang.template.ValidatingProcessor 	java.lang.StringTemplate.Processor
>> 	java.lang.template.ProcessorLinkage 	java.lang.StringTemplate.Processor.Linkage
>> 	java.lang.template.TemplateProcessor 	java.lang.StringTemplate.SimpleProcessor
>> 	java.lang.template.StringProcessor 	java.lang.StringTemplate.StringProcessor

>> (*) - package private

>> The new processor hierarchy will be;

>> interface Processor<R, E>
>> interface SimpleProcessor<R> extends Processor<R, RuntimeException>
>> interface StringProcessor extends SimpleProcessor<String>

>> It will take me a few days to update the JEP, CSRs, PR and JLS, so stay tuned.
>> As always, comments are welcome.

>> Cheers,

>> ? Jim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230327/90284cee/attachment.htm>

From james.laskey at oracle.com  Mon Mar 27 14:46:57 2023
From: james.laskey at oracle.com (Jim Laskey)
Date: Mon, 27 Mar 2023 14:46:57 +0000
Subject: StringTemplates name changes
In-Reply-To: <38a90674-200f-cf56-90a0-387b786a43be@oracle.com>
References: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
 <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>
 <38a90674-200f-cf56-90a0-387b786a43be@oracle.com>
Message-ID: <7C4081D8-414D-487D-ADB0-9310BEE0AD6F@oracle.com>

TemplateSupport is used internally in java.lang.runtime and is package private. Nothing to see there.

On Mar 27, 2023, at 11:18 AM, Brian Goetz <brian.goetz at oracle.com> wrote:

Nice.

So that means java.lang additions are limited to StringTemplate (with nested Processor), and public jl.runtime limited to Template{Runtime,Support}.

In the spirit of "nothing left to take away", who are the methods in TemplateSupport used by?  Authors of processors?  There's not too much here -- creation, interpolation, combination.  Would these make sense as static helper methods on STringTemplate?  They'd be more discoverable there.


On 3/27/2023 9:07 AM, Jim Laskey wrote:

After the string template interface name changes, i.e., TemplateProcessor becoming Processor, the rationale for the existence of SimpleProcessor and StringProcessor has lessened to the point where they should be dropped.

SimpleProcessor owed its existence to the long-winded name TemplateProcessor and that ugly second parameter, E, in Processor<R, E> (in a many of cases E will be the unchecked RuntimeException). StringProcessor existed because most template processors will produce strings.

TemplateProcessor<JSONObject, RuntimeException> JSON = st-> new JSONObject(st.interpolate());
TemplateProcessor<String, RuntimeException> INTER = StringTemplate::interpolate;

vs.

SimpleProcessor<JSONObject> JSON = st-> new JSONObject(st.interpolate());
StringProcessor INTER = StringTemplate::interpolate;

It was thought that having the friendlier interfaces would provide clarity, hide RuntimeException and simplify explanation. The reality is that most developers will define template processors using full class declarations. Furthermore, developers will learn to use RuntimeException regularly due to the abundance of template processor examples.

public class InterpolateProcessor implements Processor<String, RuntimeException> {
    @Override
    public String process(StringTemplate st) {
        return st.interpolate();
    }
}

SimpleProcessor<String> INTER = new InterpolateProcessor();


Even after SimpleProcessor and StringProcessor go away, developers can still use the functional interface shorthand.

Processor<JSONObject, RuntimeException> JSON = st-> new JSONObject(st.interpolate());
Processor<String, RuntimeException> INTER = StringTemplate::interpolate;

And, a new factory method, Processor.of, will be added for fans of var.

var JSON = Processor.of(st-> new JSONObject(st.interpolate()));
var INTER = Processor.of(StringTemplate::interpolate);

For those developers that like the notion of SimpleProcessor and StringProcessor, these interfaces can be trivially defined per project;

@FunctionalInterface
public interface SimpleProcessor<R> extends Processor<R, RuntimeException> {}

@FunctionalInterface
public interface StringProcessor extends SimpleProcessor<String> {}






On Mar 17, 2023, at 10:24 AM, Jim Laskey <james.laskey at oracle.com><mailto:james.laskey at oracle.com> wrote:

This is a  heads up about some name changes coming to the string template feature with the intent of eliminating the ?java.lang.template? package along with clarifying the processor hierarchy,

Old     New
java.lang.template.Carriers*    java.lang.runtime.Carriers*
java.lang.template.ReferencedKeyMap*    java.lang.runtime.ReferencedKeyMap*
java.lang.template.ReferenceKey*        java.lang.runtime.ReferenceKey*
java.lang.template.StringTemplateImpl*  java.lang.runtime.StringTemplateImpl*
java.lang.template.StringTemplateImplFactory*   java.lang.runtime.StringTemplateImplFactory*
java.lang.runtime.TemplateRuntime       java.lang.runtime.TemplateRuntime
java.lang.template.TemplateSupport*     java.lang.runtime.TemplateSupport
java.lang.template.StringTemplate       java.lang.StringTemplate
java.lang.template.ValidatingProcessor  java.lang.StringTemplate.Processor
java.lang.template.ProcessorLinkage     java.lang.StringTemplate.Processor.Linkage
java.lang.template.TemplateProcessor    java.lang.StringTemplate.SimpleProcessor
java.lang.template.StringProcessor      java.lang.StringTemplate.StringProcessor

(*) - package private


The new processor hierarchy will be;

interface Processor<R, E>
interface SimpleProcessor<R> extends Processor<R, RuntimeException>
interface StringProcessor extends SimpleProcessor<String>

It will take me a few days to update the JEP, CSRs, PR and JLS, so stay tuned. As always, comments are welcome.

Cheers,

? Jim




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230327/9a525f72/attachment-0001.htm>

From brian.goetz at oracle.com  Mon Mar 27 14:55:05 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Mon, 27 Mar 2023 10:55:05 -0400
Subject: StringTemplates name changes
In-Reply-To: <7C4081D8-414D-487D-ADB0-9310BEE0AD6F@oracle.com>
References: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
 <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>
 <38a90674-200f-cf56-90a0-387b786a43be@oracle.com>
 <7C4081D8-414D-487D-ADB0-9310BEE0AD6F@oracle.com>
Message-ID: <212f4f55-3809-5b7b-c876-ee1be1ff6958@oracle.com>

Was going by

_Old_ 	_New_
java.lang.template.Carriers* 	java.lang.runtime.Carriers*
java.lang.template.ReferencedKeyMap* 	java.lang.runtime.ReferencedKeyMap*
java.lang.template.ReferenceKey* 	java.lang.runtime.ReferenceKey*
java.lang.template.StringTemplateImpl* 
java.lang.runtime.StringTemplateImpl*
java.lang.template.StringTemplateImplFactory* 
java.lang.runtime.StringTemplateImplFactory*
java.lang.runtime.TemplateRuntime 	java.lang.runtime.TemplateRuntime
java.lang.template.TemplateSupport* 	java.lang.runtime.TemplateSupport
java.lang.template.StringTemplate 	java.lang.StringTemplate
java.lang.template.ValidatingProcessor 	java.lang.StringTemplate.Processor
java.lang.template.ProcessorLinkage 
java.lang.StringTemplate.Processor.Linkage
java.lang.template.TemplateProcessor 
java.lang.StringTemplate.SimpleProcessor
java.lang.template.StringProcessor 
java.lang.StringTemplate.StringProcessor



which didn't flag TemplateSupport as package-private.? If it gets a 
star, great.


On 3/27/2023 10:46 AM, Jim Laskey wrote:
> TemplateSupport is used internally in java.lang.runtime and is package 
> private. Nothing to see there.
>
>> On Mar 27, 2023, at 11:18 AM, Brian Goetz <brian.goetz at oracle.com> wrote:
>>
>> Nice.
>>
>> So that means java.lang additions are limited to StringTemplate (with 
>> nested Processor), and public jl.runtime limited to 
>> Template{Runtime,Support}.
>>
>> In the spirit of "nothing left to take away", who are the methods in 
>> TemplateSupport used by?? Authors of processors?? There's not too 
>> much here -- creation, interpolation, combination.? Would these make 
>> sense as static helper methods on STringTemplate?? They'd be more 
>> discoverable there.
>>
>>
>> On 3/27/2023 9:07 AM, Jim Laskey wrote:
>>>
>>> After the string template interface name changes, i.e., 
>>> |TemplateProcessor|?becoming |Processor|, the rationale for the 
>>> existence of |SimpleProcessor |and |StringProcessor|?has lessened to 
>>> the point where they should be dropped.
>>>
>>> |SimpleProcessor|?owed its existence to the long-winded name 
>>> |TemplateProcessor|?and that ugly second parameter, |E|, in 
>>> |Processor<R, E>|?(in a many of cases |E|?will be the unchecked 
>>> |RuntimeException|). |StringProcessor|?existed because most template 
>>> processors will produce strings.
>>>
>>> |TemplateProcessor<JSONObject, RuntimeException> JSON = st-> new 
>>> JSONObject(st.interpolate()); TemplateProcessor<String, 
>>> RuntimeException> INTER = StringTemplate::interpolate;|
>>>
>>> vs.
>>>
>>> |SimpleProcessor<JSONObject> JSON = st-> new 
>>> JSONObject(st.interpolate()); StringProcessor INTER = 
>>> StringTemplate::interpolate;|
>>>
>>> It was thought that having the friendlier interfaces would provide 
>>> clarity, hide |RuntimeException|?and simplify explanation. The 
>>> reality is that most developers will define template processors 
>>> using full class declarations. Furthermore, developers will learn to 
>>> use |RuntimeException|?regularly due to the abundance of template 
>>> processor examples.
>>>
>>> |public class InterpolateProcessor implements Processor<String, 
>>> RuntimeException> { @Override public String process(StringTemplate 
>>> st) { return st.interpolate(); } } SimpleProcessor<String> INTER = 
>>> new InterpolateProcessor(); |
>>>
>>> Even after |SimpleProcessor|?and |StringProcessor|?go away, 
>>> developers can still use the functional interface shorthand.
>>>
>>> |Processor<JSONObject, RuntimeException> JSON = st-> new 
>>> JSONObject(st.interpolate()); Processor<String, RuntimeException> 
>>> INTER = StringTemplate::interpolate;|
>>>
>>> And, a new factory method, |Processor.of|, will be added for fans of 
>>> |var|.
>>>
>>> |var JSON = Processor.of(st-> new JSONObject(st.interpolate())); var 
>>> INTER = Processor.of(StringTemplate::interpolate);|
>>>
>>> For those developers that like the notion of |SimpleProcessor|?and 
>>> |StringProcessor|, these interfaces can be trivially defined per 
>>> project;
>>>
>>> |@FunctionalInterface public interface SimpleProcessor<R> extends 
>>> Processor<R, RuntimeException> {} @FunctionalInterface public 
>>> interface StringProcessor extends SimpleProcessor<String> {}|
>>>
>>>
>>>
>>>
>>>
>>>
>>>> On Mar 17, 2023, at 10:24 AM, Jim Laskey <james.laskey at oracle.com> 
>>>> wrote:
>>>>
>>>> This is a ?heads up about some name changes coming to the string 
>>>> template feature with the intent of eliminating the 
>>>> ?java.lang.template? package along with clarifying the processor 
>>>> hierarchy,
>>>>
>>>> _Old_ 	_New_
>>>> java.lang.template.Carriers* 	java.lang.runtime.Carriers*
>>>> java.lang.template.ReferencedKeyMap* 
>>>> java.lang.runtime.ReferencedKeyMap*
>>>> java.lang.template.ReferenceKey* 	java.lang.runtime.ReferenceKey*
>>>> java.lang.template.StringTemplateImpl* 
>>>> java.lang.runtime.StringTemplateImpl*
>>>> java.lang.template.StringTemplateImplFactory* 
>>>> java.lang.runtime.StringTemplateImplFactory*
>>>> java.lang.runtime.TemplateRuntime 	java.lang.runtime.TemplateRuntime
>>>> java.lang.template.TemplateSupport* 	java.lang.runtime.TemplateSupport
>>>> java.lang.template.StringTemplate 	java.lang.StringTemplate
>>>> java.lang.template.ValidatingProcessor 
>>>> java.lang.StringTemplate.Processor
>>>> java.lang.template.ProcessorLinkage 
>>>> java.lang.StringTemplate.Processor.Linkage
>>>> java.lang.template.TemplateProcessor 
>>>> java.lang.StringTemplate.SimpleProcessor
>>>> java.lang.template.StringProcessor 
>>>> java.lang.StringTemplate.StringProcessor
>>>>
>>>>
>>>> (*) - package private
>>>>
>>>>
>>>> The new processor hierarchy will be;
>>>>
>>>> interface Processor<R, E>
>>>> interface SimpleProcessor<R> extends Processor<R, RuntimeException>
>>>> interface StringProcessor extends SimpleProcessor<String>
>>>>
>>>> It will take me a few days to update the JEP, CSRs, PR and JLS, so 
>>>> stay tuned. As always, comments are welcome.
>>>>
>>>> Cheers,
>>>>
>>>> ? Jim
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230327/81ab731b/attachment-0001.htm>

From brian.goetz at oracle.com  Mon Mar 27 15:19:52 2023
From: brian.goetz at oracle.com (Brian Goetz)
Date: Mon, 27 Mar 2023 11:19:52 -0400
Subject: Record pattern inference & capture
In-Reply-To: <05E51BDD-F0BA-481E-A73B-5C5A4EC49ECB@oracle.com>
References: <05E51BDD-F0BA-481E-A73B-5C5A4EC49ECB@oracle.com>
Message-ID: <8fc2522f-0090-9712-985c-dc627aa09db6@oracle.com>

Not directly related, but something to keep in mind - when we get to 
full deconstruction patterns, we will also have to do some overload 
selection here, because the type named in the pattern may have multiple 
deconstructors.

OK, on to your point.? You've got a match candidate that is something 
like Function<String,String>, and a pattern like

 ??? case UnaryOperator(...)

on it.? (I assume the use of functional interfaces is merely incidental 
here, and you're just trying to infer the type arguments of the pattern 
UnaryOperator<T> from a match target of Function<U,V>.)

Mapper<T> extends Function<T,T>, so really you are asking whether there 
is a solution for T <: CharSequence and T :> String, because if there is 
no solution, we should reject the pattern as not applicable, right?



On 3/24/2023 6:00 PM, Dan Smith wrote:
> The record pattern inference strategy introduced in 20 attempts to map a pattern's match type to a parameterization of a given generic record class (typically a subclass of the match type).
>
> One of its capabilities is to interpret a "duplicated" type argument?as illustrated in the proposed JLS 18.5.5 text, for example, the type Function<Foo,Foo> can be mapped to a type like UnaryOperator<Foo>.
>
> Another of its capabilities is an inference-based approach to wildcards, so that a match type like Function<? extends Foo, ? extends Bar> can map to something like UnaryOperator<? extends Foo & Bar>:
>
> record Mapper<T>(T in, T out) implements UnaryOperator<T> {
>      T apply(T arg) { return in.equals(arg) ? out : null; }
> }
>
> void test(Function<? super String, ? extends CharSequence> f) {
>      if (f instanceof Mapper(var in, var out)) {
>          boolean shorter = out.length() < in.length();
>      }
> }
>
> Unfortunately, this strategy doesn't account for the fact that the match type should probably be captured before doing anything with it. (JLS is a little fuzzy in this when it comes to variable references, but in general every use/read of a variable or method needs to be captured before further typing occurs. Setting aside some poorly-specified cases, it's easy to come up with examples, like a method invocation (see JLS 15.12.3) where the wildcards definitively get captured.)
>
> Nor does it account for the fact that capture variables may appear in the match type because they've flowed out of upstream expressions like method calls. (Think 'Map<?,?>.entrySet().iterator().next()'.)
>
> If I try to match the type Function<CAP1,CAP2> with a type like UnaryOperator, should that be an inference failure? The current rules say "yes": inference variable alpha=CAP1, and alpha=CAP2, a contradiction. But I don't think that's right. If the dynamic check for UnaryOperator succeeds, that means that this must actually be a Function in which the actual types represented by CAP1 and CAP2 are the same.
>
> My conclusion is that the inference treatment of wildcards ought to apply to capture variables, too. This is justified by the fact that while usually reasoning about capture variables works by assuming "there exists some type with these properties, don't assume anything else about it", in this case we also want to incorporate the fact that the dynamic pattern-matching check did, in fact, succeed. (However, I want to validate this thinking, because my confidence isn't 100%.)
>
> Here's how I think step #3 of the proposed JLS 18.5.5 should read:
>
> -----
>
> A type T' is derived from T, as follows:
>
> -   If T is a parameterized type, let T_cap be the result of capture conversion (5.1.10) applied to T, and let Z1, ..., Zk (k ? 0) be the type variables produced by capture that are type arguments in T_cap. (This includes type variables produced by the capture conversion in this step, and type variables produced by capture conversion elsewhere.) Let ?1, ..., ?k (k ? 0) be inference variables, and let ? be the substitution [Z1:=?1, ..., Zk:=?k]. T' is T_cap ?.
>
>      Additional bounds for ?1, ..., ?k are incorporated into B0 to form a bound set B1, as follows:
>
>      - If ?i (1 ? i ? k) replaced a type variable with an upper bound U, then the bound ?i <: U ? appears in the bound set
>
>      - If ?i (1 ? i ? k) replaced a type variable with a lower bound L, then the bound L ? <: ?i appears in the bound set
>
>      - If no proper upper bounds otherwise exist for ?i (1 ? i ? k), the bound ?i <: Object appears in the bound set
>
> - If T is any other class or interface type, then T' is the same as T, and B1 is the same as B0.
>
> - If T is a type variable or an intersection type, then for each upper bound of the type variable or element of the intersection type, this step and step 4 are repeated recursively. All bounds produced in steps 3 and 4 are incorporated into a single bound set.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230327/0bd282f5/attachment.htm>

From daniel.smith at oracle.com  Wed Mar 29 21:05:49 2023
From: daniel.smith at oracle.com (Dan Smith)
Date: Wed, 29 Mar 2023 21:05:49 +0000
Subject: Record pattern inference & capture
In-Reply-To: <8fc2522f-0090-9712-985c-dc627aa09db6@oracle.com>
References: <05E51BDD-F0BA-481E-A73B-5C5A4EC49ECB@oracle.com>
 <8fc2522f-0090-9712-985c-dc627aa09db6@oracle.com>
Message-ID: <37EDB60B-F222-4B6B-AA71-6B53216A7BE5@oracle.com>

> On Mar 27, 2023, at 8:19 AM, Brian Goetz <brian.goetz at oracle.com> wrote:
> 
> Not directly related, but something to keep in mind - when we get to full deconstruction patterns, we will also have to do some overload selection here, because the type named in the pattern may have multiple deconstructors.

Sure, although I'm skeptical of attempts to mirror method invocation overload resolution here. The intuitions about invocation compatibility (as used by method invocations) don't carry over very when you switch to cast compatibility (as used by patterns).

> OK, on to your point.  You've got a match candidate that is something like Function<String,String>, and a pattern like 
> 
>     case UnaryOperator(...)
> 
> on it.  (I assume the use of functional interfaces is merely incidental here, and you're just trying to infer the type arguments of the pattern UnaryOperator<T> from a match target of Function<U,V>.)

Yep, just some well-known types that have interesting properties in their use of generics.

> Mapper<T> extends Function<T,T>, so really you are asking whether there is a solution for T <: CharSequence and T :> String, because if there is no solution, we should reject the pattern as not applicable, right?

Right. And further, if there is a solution, what is the parameterization? The answer will be a wildcard with certain bounds, as determined by bounds on an inference variable.


From gavin.bierman at oracle.com  Fri Mar 31 16:44:28 2023
From: gavin.bierman at oracle.com (Gavin Bierman)
Date: Fri, 31 Mar 2023 16:44:28 +0000
Subject: StringTemplates name changes
In-Reply-To: <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>
References: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
 <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>
Message-ID: <1BAB49A9-1D27-4846-A591-49084608D4F0@oracle.com>

I?ve updated the spec change document to reflect this change: https://cr.openjdk.org/~gbierman/jep430/latest/

Thanks,
Gavin

On 27 Mar 2023, at 14:07, Jim Laskey <james.laskey at oracle.com> wrote:


After the string template interface name changes, i.e., TemplateProcessor becoming Processor, the rationale for the existence of SimpleProcessor and StringProcessor has lessened to the point where they should be dropped.

SimpleProcessor owed its existence to the long-winded name TemplateProcessor and that ugly second parameter, E, in Processor<R, E> (in a many of cases E will be the unchecked RuntimeException). StringProcessor existed because most template processors will produce strings.

TemplateProcessor<JSONObject, RuntimeException> JSON = st-> new JSONObject(st.interpolate());
TemplateProcessor<String, RuntimeException> INTER = StringTemplate::interpolate;

vs.

SimpleProcessor<JSONObject> JSON = st-> new JSONObject(st.interpolate());
StringProcessor INTER = StringTemplate::interpolate;

It was thought that having the friendlier interfaces would provide clarity, hide RuntimeException and simplify explanation. The reality is that most developers will define template processors using full class declarations. Furthermore, developers will learn to use RuntimeException regularly due to the abundance of template processor examples.

public class InterpolateProcessor implements Processor<String, RuntimeException> {
    @Override
    public String process(StringTemplate st) {
        return st.interpolate();
    }
}

SimpleProcessor<String> INTER = new InterpolateProcessor();


Even after SimpleProcessor and StringProcessor go away, developers can still use the functional interface shorthand.

Processor<JSONObject, RuntimeException> JSON = st-> new JSONObject(st.interpolate());
Processor<String, RuntimeException> INTER = StringTemplate::interpolate;

And, a new factory method, Processor.of, will be added for fans of var.

var JSON = Processor.of(st-> new JSONObject(st.interpolate()));
var INTER = Processor.of(StringTemplate::interpolate);

For those developers that like the notion of SimpleProcessor and StringProcessor, these interfaces can be trivially defined per project;

@FunctionalInterface
public interface SimpleProcessor<R> extends Processor<R, RuntimeException> {}

@FunctionalInterface
public interface StringProcessor extends SimpleProcessor<String> {}






On Mar 17, 2023, at 10:24 AM, Jim Laskey <james.laskey at oracle.com> wrote:

This is a  heads up about some name changes coming to the string template feature with the intent of eliminating the ?java.lang.template? package along with clarifying the processor hierarchy,

Old     New
java.lang.template.Carriers*    java.lang.runtime.Carriers*
java.lang.template.ReferencedKeyMap*    java.lang.runtime.ReferencedKeyMap*
java.lang.template.ReferenceKey*        java.lang.runtime.ReferenceKey*
java.lang.template.StringTemplateImpl*  java.lang.runtime.StringTemplateImpl*
java.lang.template.StringTemplateImplFactory*   java.lang.runtime.StringTemplateImplFactory*
java.lang.runtime.TemplateRuntime       java.lang.runtime.TemplateRuntime
java.lang.template.TemplateSupport*     java.lang.runtime.TemplateSupport
java.lang.template.StringTemplate       java.lang.StringTemplate
java.lang.template.ValidatingProcessor  java.lang.StringTemplate.Processor
java.lang.template.ProcessorLinkage     java.lang.StringTemplate.Processor.Linkage
java.lang.template.TemplateProcessor    java.lang.StringTemplate.SimpleProcessor
java.lang.template.StringProcessor      java.lang.StringTemplate.StringProcessor

(*) - package private


The new processor hierarchy will be;

interface Processor<R, E>
interface SimpleProcessor<R> extends Processor<R, RuntimeException>
interface StringProcessor extends SimpleProcessor<String>

It will take me a few days to update the JEP, CSRs, PR and JLS, so stay tuned. As always, comments are welcome.

Cheers,

? Jim



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-spec-observers/attachments/20230331/1dff6421/attachment-0001.htm>

From alex.buckley at oracle.com  Fri Mar 31 17:19:21 2023
From: alex.buckley at oracle.com (Alex Buckley)
Date: Fri, 31 Mar 2023 10:19:21 -0700
Subject: StringTemplates name changes
In-Reply-To: <1BAB49A9-1D27-4846-A591-49084608D4F0@oracle.com>
References: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
 <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>
 <1BAB49A9-1D27-4846-A591-49084608D4F0@oracle.com>
Message-ID: <89062213-8753-dc23-39ff-b883327a6ce4@oracle.com>

Hi Gavin,

Re: "The type of the TemplateProcessor expression must be a subtype of a 
type StringTemplate.Processor<R,E>, for some types R and E ..."

I think this clause can be improved to ensure that the {Class,Interface} 
vs. Type police don't come after us. The term 
`StringTemplate.Processor<R,E>` looks like a generic interface, not a 
parameterized interface type, but then "for some types R and E" suggests 
type arguments, which means the term denotes a parameterized interface 
type after all.

The JLS has a so-so record of handling must-be-a-parameterized-type in 
the past. Grepping ch.14 for "subtype", we find:

-----
Otherwise, if the Expression has a type that is a subtype of 
Iterable<X>, for some type X, then R is X.
...
If the type of Expression is a subtype of Iterable<X> for some type 
argument X, then I is the type java.util.Iterator<X>
-----

There's also verbiage in ch.14 about the raw type `Iterable`, which 
makes me wonder if the type of the TemplateProcessor expression can be 
the raw type `StringTemplate.Processor`.

Can we improve the presentation of the required type to prevent it 
looking like a generic interface? I pondered saying that the type of the 
expression _must implement_ <something>, but _must be a subtype_ is more 
common. Here's an idea:

-----
The type TP of the TemplateProcessor expression must be a subtype of 
StringTemplate.Processor, or a compile-time error occurs. [That opens 
the door to a raw supertype.] If TP implements the parameterized type 
StringTemplate.Processor<Result,Exc>, then the type of the template 
expression is Result. If TP implements the raw type 
StringTemplate.Processor, then the type of the template expression is 
Object.

   StringTemplate.Processor<R,E> is a generic functional interface (9.8)
   whose single abstract method takes a StringTemplate, returns R, and
   throws E.
-----

Alex

On 3/31/2023 9:44 AM, Gavin Bierman wrote:
> I?ve updated the spec change document to reflect this change: 
> https://cr.openjdk.org/~gbierman/jep430/latest/ 
> <https://cr.openjdk.org/~gbierman/jep430/latest/>
> 
> Thanks,
> Gavin
> 
>> On 27 Mar 2023, at 14:07, Jim Laskey <james.laskey at oracle.com> wrote:
>>
>> After the string template interface name changes, i.e., 
>> |TemplateProcessor|?becoming |Processor|, the rationale for the 
>> existence of |SimpleProcessor |and |StringProcessor|?has lessened to 
>> the point where they should be dropped.
>>
>> |SimpleProcessor|?owed its existence to the long-winded name 
>> |TemplateProcessor|?and that ugly second parameter, |E|, in 
>> |Processor<R, E>|?(in a many of cases |E|?will be the unchecked 
>> |RuntimeException|). |StringProcessor|?existed because most template 
>> processors will produce strings.
>>
>> |TemplateProcessor<JSONObject, RuntimeException> JSON = st-> new 
>> JSONObject(st.interpolate()); TemplateProcessor<String, 
>> RuntimeException> INTER = StringTemplate::interpolate;|
>>
>> vs.
>>
>> |SimpleProcessor<JSONObject> JSON = st-> new 
>> JSONObject(st.interpolate()); StringProcessor INTER = 
>> StringTemplate::interpolate;|
>>
>> It was thought that having the friendlier interfaces would provide 
>> clarity, hide |RuntimeException|?and simplify explanation. The reality 
>> is that most developers will define template processors using full 
>> class declarations. Furthermore, developers will learn to use 
>> |RuntimeException|?regularly due to the abundance of template 
>> processor examples.
>>
>> |public class InterpolateProcessor implements Processor<String, 
>> RuntimeException> { @Override public String process(StringTemplate st) 
>> { return st.interpolate(); } } SimpleProcessor<String> INTER = new 
>> InterpolateProcessor(); |
>>
>> Even after |SimpleProcessor|?and |StringProcessor|?go away, developers 
>> can still use the functional interface shorthand.
>>
>> |Processor<JSONObject, RuntimeException> JSON = st-> new 
>> JSONObject(st.interpolate()); Processor<String, RuntimeException> 
>> INTER = StringTemplate::interpolate;|
>>
>> And, a new factory method, |Processor.of|, will be added for fans of 
>> |var|.
>>
>> |var JSON = Processor.of(st-> new JSONObject(st.interpolate())); var 
>> INTER = Processor.of(StringTemplate::interpolate);|
>>
>> For those developers that like the notion of |SimpleProcessor|?and 
>> |StringProcessor|, these interfaces can be trivially defined per project;
>>
>> |@FunctionalInterface public interface SimpleProcessor<R> extends 
>> Processor<R, RuntimeException> {} @FunctionalInterface public 
>> interface StringProcessor extends SimpleProcessor<String> {}|
>>
>>
>>
>>
>>
>>
>>> On Mar 17, 2023, at 10:24 AM, Jim Laskey <james.laskey at oracle.com> wrote:
>>>
>>> This is a ?heads up about some name changes coming to the string 
>>> template feature with the intent of eliminating the 
>>> ?java.lang.template? package along with clarifying the processor 
>>> hierarchy,
>>>
>>> _Old_ 	_New_
>>> java.lang.template.Carriers* 	java.lang.runtime.Carriers*
>>> java.lang.template.ReferencedKeyMap* 
>>> java.lang.runtime.ReferencedKeyMap*
>>> java.lang.template.ReferenceKey* 	java.lang.runtime.ReferenceKey*
>>> java.lang.template.StringTemplateImpl* 
>>> java.lang.runtime.StringTemplateImpl*
>>> java.lang.template.StringTemplateImplFactory* 
>>> java.lang.runtime.StringTemplateImplFactory*
>>> java.lang.runtime.TemplateRuntime 	java.lang.runtime.TemplateRuntime
>>> java.lang.template.TemplateSupport* 	java.lang.runtime.TemplateSupport
>>> java.lang.template.StringTemplate 	java.lang.StringTemplate
>>> java.lang.template.ValidatingProcessor 
>>> java.lang.StringTemplate.Processor
>>> java.lang.template.ProcessorLinkage 
>>> java.lang.StringTemplate.Processor.Linkage
>>> java.lang.template.TemplateProcessor 
>>> java.lang.StringTemplate.SimpleProcessor
>>> java.lang.template.StringProcessor 
>>> java.lang.StringTemplate.StringProcessor
>>>
>>>
>>> (*) - package private
>>>
>>>
>>> The new processor hierarchy will be;
>>>
>>> interface Processor<R, E>
>>> interface SimpleProcessor<R> extends Processor<R, RuntimeException>
>>> interface StringProcessor extends SimpleProcessor<String>
>>>
>>> It will take me a few days to update the JEP, CSRs, PR and JLS, so 
>>> stay tuned. As always, comments are welcome.
>>>
>>> Cheers,
>>>
>>> ? Jim
>>>
>>
> 

From gavin.bierman at oracle.com  Fri Mar 31 21:34:44 2023
From: gavin.bierman at oracle.com (Gavin Bierman)
Date: Fri, 31 Mar 2023 21:34:44 +0000
Subject: StringTemplates name changes
In-Reply-To: <89062213-8753-dc23-39ff-b883327a6ce4@oracle.com>
References: <CC579D36-D79B-4F8C-860B-B85169658128@oracle.com>
 <15178054-61DC-49CB-B14D-174F5A88ECD2@oracle.com>
 <1BAB49A9-1D27-4846-A591-49084608D4F0@oracle.com>
 <89062213-8753-dc23-39ff-b883327a6ce4@oracle.com>
Message-ID: <47C82452-69D4-4E5A-B4C6-FC1A41DB8E72@oracle.com>

Thanks Alex - I?ve tweaked the spec accordingly.

Gavin

> On 31 Mar 2023, at 18:19, Alex Buckley <alex.buckley at oracle.com> wrote:
> 
> Hi Gavin,
> 
> Re: "The type of the TemplateProcessor expression must be a subtype of a type StringTemplate.Processor<R,E>, for some types R and E ..."
> 
> I think this clause can be improved to ensure that the {Class,Interface} vs. Type police don't come after us. The term `StringTemplate.Processor<R,E>` looks like a generic interface, not a parameterized interface type, but then "for some types R and E" suggests type arguments, which means the term denotes a parameterized interface type after all.
> 
> The JLS has a so-so record of handling must-be-a-parameterized-type in the past. Grepping ch.14 for "subtype", we find:
> 
> -----
> Otherwise, if the Expression has a type that is a subtype of Iterable<X>, for some type X, then R is X.
> ...
> If the type of Expression is a subtype of Iterable<X> for some type argument X, then I is the type java.util.Iterator<X>
> -----
> 
> There's also verbiage in ch.14 about the raw type `Iterable`, which makes me wonder if the type of the TemplateProcessor expression can be the raw type `StringTemplate.Processor`.
> 
> Can we improve the presentation of the required type to prevent it looking like a generic interface? I pondered saying that the type of the expression _must implement_ <something>, but _must be a subtype_ is more common. Here's an idea:
> 
> -----
> The type TP of the TemplateProcessor expression must be a subtype of StringTemplate.Processor, or a compile-time error occurs. [That opens the door to a raw supertype.] If TP implements the parameterized type StringTemplate.Processor<Result,Exc>, then the type of the template expression is Result. If TP implements the raw type StringTemplate.Processor, then the type of the template expression is Object.
> 
>  StringTemplate.Processor<R,E> is a generic functional interface (9.8)
>  whose single abstract method takes a StringTemplate, returns R, and
>  throws E.
> -----
> 
> Alex
> 
> On 3/31/2023 9:44 AM, Gavin Bierman wrote:
>> I?ve updated the spec change document to reflect this change: https://cr.openjdk.org/~gbierman/jep430/latest/ <https://cr.openjdk.org/~gbierman/jep430/latest/>
>> Thanks,
>> Gavin
>>> On 27 Mar 2023, at 14:07, Jim Laskey <james.laskey at oracle.com> wrote:
>>> 
>>> After the string template interface name changes, i.e., |TemplateProcessor| becoming |Processor|, the rationale for the existence of |SimpleProcessor |and |StringProcessor| has lessened to the point where they should be dropped.
>>> 
>>> |SimpleProcessor| owed its existence to the long-winded name |TemplateProcessor| and that ugly second parameter, |E|, in |Processor<R, E>| (in a many of cases |E| will be the unchecked |RuntimeException|). |StringProcessor| existed because most template processors will produce strings.
>>> 
>>> |TemplateProcessor<JSONObject, RuntimeException> JSON = st-> new JSONObject(st.interpolate()); TemplateProcessor<String, RuntimeException> INTER = StringTemplate::interpolate;|
>>> 
>>> vs.
>>> 
>>> |SimpleProcessor<JSONObject> JSON = st-> new JSONObject(st.interpolate()); StringProcessor INTER = StringTemplate::interpolate;|
>>> 
>>> It was thought that having the friendlier interfaces would provide clarity, hide |RuntimeException| and simplify explanation. The reality is that most developers will define template processors using full class declarations. Furthermore, developers will learn to use |RuntimeException| regularly due to the abundance of template processor examples.
>>> 
>>> |public class InterpolateProcessor implements Processor<String, RuntimeException> { @Override public String process(StringTemplate st) { return st.interpolate(); } } SimpleProcessor<String> INTER = new InterpolateProcessor(); |
>>> 
>>> Even after |SimpleProcessor| and |StringProcessor| go away, developers can still use the functional interface shorthand.
>>> 
>>> |Processor<JSONObject, RuntimeException> JSON = st-> new JSONObject(st.interpolate()); Processor<String, RuntimeException> INTER = StringTemplate::interpolate;|
>>> 
>>> And, a new factory method, |Processor.of|, will be added for fans of |var|.
>>> 
>>> |var JSON = Processor.of(st-> new JSONObject(st.interpolate())); var INTER = Processor.of(StringTemplate::interpolate);|
>>> 
>>> For those developers that like the notion of |SimpleProcessor| and |StringProcessor|, these interfaces can be trivially defined per project;
>>> 
>>> |@FunctionalInterface public interface SimpleProcessor<R> extends Processor<R, RuntimeException> {} @FunctionalInterface public interface StringProcessor extends SimpleProcessor<String> {}|
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>>> On Mar 17, 2023, at 10:24 AM, Jim Laskey <james.laskey at oracle.com> wrote:
>>>> 
>>>> This is a  heads up about some name changes coming to the string template feature with the intent of eliminating the ?java.lang.template? package along with clarifying the processor hierarchy,
>>>> 
>>>> _Old_ 	_New_
>>>> java.lang.template.Carriers* 	java.lang.runtime.Carriers*
>>>> java.lang.template.ReferencedKeyMap* java.lang.runtime.ReferencedKeyMap*
>>>> java.lang.template.ReferenceKey* 	java.lang.runtime.ReferenceKey*
>>>> java.lang.template.StringTemplateImpl* java.lang.runtime.StringTemplateImpl*
>>>> java.lang.template.StringTemplateImplFactory* java.lang.runtime.StringTemplateImplFactory*
>>>> java.lang.runtime.TemplateRuntime 	java.lang.runtime.TemplateRuntime
>>>> java.lang.template.TemplateSupport* 	java.lang.runtime.TemplateSupport
>>>> java.lang.template.StringTemplate 	java.lang.StringTemplate
>>>> java.lang.template.ValidatingProcessor java.lang.StringTemplate.Processor
>>>> java.lang.template.ProcessorLinkage java.lang.StringTemplate.Processor.Linkage
>>>> java.lang.template.TemplateProcessor java.lang.StringTemplate.SimpleProcessor
>>>> java.lang.template.StringProcessor java.lang.StringTemplate.StringProcessor
>>>> 
>>>> 
>>>> (*) - package private
>>>> 
>>>> 
>>>> The new processor hierarchy will be;
>>>> 
>>>> interface Processor<R, E>
>>>> interface SimpleProcessor<R> extends Processor<R, RuntimeException>
>>>> interface StringProcessor extends SimpleProcessor<String>
>>>> 
>>>> It will take me a few days to update the JEP, CSRs, PR and JLS, so stay tuned. As always, comments are welcome.
>>>> 
>>>> Cheers,
>>>> 
>>>> ? Jim
>>>> 
>>>