The introduction of Sequenced collections is not a source compatible change

[Joseph D. Darcy]

PPS Various compatibility topics were covered in a talk I gave at the 
JCP EC in April this year:

"Contributing to OpenJDK: Participating in stewardship for the long-term"
https://jcp.org/aboutJava/communityprocess/ec-public/materials/2023-04-12/Contributing_to_OpenJDK_2023_04_12.pdf

-Joe

On 5/4/2023 9:08 PM, Joseph D. Darcy wrote:
> PS And as a general policy, over the releases we've done more to 
> preserve source compatibility when possible for language changes.
>
> For example, there were many source compatibility breakages when 
> "assert" was added as a keyword back in JDK 1.4; the current keyword 
> management policies (https://openjdk.org/jeps/8223002), including 
> contextual keywords (JLS §3.9) and hyphenated-keywords, mitigate the 
> impact of analogous changes today.
>
> -Joe
>
> On 5/4/2023 7:47 PM, - wrote:
>> In addition, in the CSR of sequenced collection, it already
>> anticipated some other form of source incompatibility:
>> If a class implements List and Deque at the same time, the return type
>> of reversed() must extend both interfaces as well.
>>
>> This alone would be a greater source incompatibility than this type
>> interference already; however, both are binarily compatible:
>> Java doesn't care about generics at runtime, and for default
>> reversed() overrides, reversed ()Ljava/util/List; and reversed
>> ()Ljava/util/Deque; are distinct methods; code calling reversed via
>> List or Deque interfaces will get the reversed version of respective
>> interfaces, which are functionally correct as well.
>>
>> I personally would consider this type inference usage moot. It is same
>> as adding another method addAll(List<String>) when there is already
>> addAll(String[]): existing addAll(null) calls will break, but this is
>> not a valid argument against adding a List<String> variant of addAll.
>>
>> On Thu, May 4, 2023 at 9:38 PM Joseph D. Darcy <joe.darcy at oracle.com> 
>> wrote:
>>> A few comments on the general compatibility policy for the JDK. 
>>> Compatibility is looked after by the Compatibility and Specification 
>>> Review (CSR) process ( Compatibility & Specification Review). 
>>> Summarizing the approach,
>>>
>>> The general compatibility policy for exported APIs implemented in 
>>> the JDK is:
>>>
>>>      * Don't break binary compatibility (as defined in the Java 
>>> Language Specification) without sufficient cause.
>>>      * Avoid introducing source incompatibilities.
>>>      * Manage behavioral compatibility changes.
>>>
>>> https://wiki.openjdk.org/display/csr/Main
>>>
>>> None of binary, source, and behavioral compatibly are absolutes and 
>>> judgement is used to assess the cost/benefits of changes. For 
>>> example, strict source compatibility would preclude, say, 
>>> introducing new public types in the java.lang package since the 
>>> implicit import of types in java.lang could conflict with a 
>>> same-named type *-imported from another package.
>>>
>>> When a proposed change is estimated to be sufficiently disruptive, 
>>> we conduct a corpus experiment to evaluate the impact on the change 
>>> on many public Java libraries. Back in Project Coin in JDK 7, that 
>>> basic approach was used to help quantify various language design 
>>> choices and the infrastructure to run such experiments has been 
>>> built-out in the subsequent releases.
>>>
>>> HTH,
>>>
>>> -Joe
>>> CSR Group Lead
>>>
>>> On 5/4/2023 6:32 AM, Ethan McCue wrote:
>>>
>>> I guess this a good time to ask, ignoring the benefit part of a cost 
>>> benefit analysis, what mechanisms do we have to measure the number 
>>> of codebases relying on type inference this will break?
>>>
>>> Iirc Adoptium built/ran the unit tests of a bunch of public repos, 
>>> but it's also a bit shocking if the jtreg suite had nothing for this.
>>>
>>> On Thu, May 4, 2023, 9:27 AM Raffaello Giulietti 
>>> <raffaello.giulietti at oracle.com> wrote:
>>>> Without changing the semantics at all, you could also write
>>>>
>>>>          final List<Collection<String>> list =
>>>> Stream.<Collection<String>>of(nestedDequeue, nestedList).toList();
>>>>
>>>> to "help" type inference.
>>>>
>>>>
>>>>
>>>>
>>>> On 2023-05-03 15:12, forax at univ-mlv.fr wrote:
>>>>> Another example sent to me by a fellow French guy,
>>>>>
>>>>>       final Deque<String> nestedDequeue = new ArrayDeque<>();
>>>>>       nestedDequeue.addFirst("C");
>>>>>       nestedDequeue.addFirst("B");
>>>>>       nestedDequeue.addFirst("A");
>>>>>
>>>>>       final List<String> nestedList = new ArrayList<>();
>>>>>       nestedList.add("D");
>>>>>       nestedList.add("E");
>>>>>       nestedList.add("F");
>>>>>
>>>>>       final List<Collection<String>> list = 
>>>>> Stream.of(nestedDequeue, nestedList).toList();
>>>>>
>>>>> This one is cool because no 'var' is involved and using 
>>>>> collect(Collectors.toList()) instead of toList() solves the 
>>>>> inference problem.
>>>>>
>>>>> Rémi
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Stuart Marks" <stuart.marks at oracle.com>
>>>>>> To: "Remi Forax" <forax at univ-mlv.fr>
>>>>>> Cc: "core-libs-dev" <core-libs-dev at openjdk.java.net>
>>>>>> Sent: Tuesday, May 2, 2023 2:44:28 AM
>>>>>> Subject: Re: The introduction of Sequenced collections is not a 
>>>>>> source compatible change
>>>>>> Hi Rémi,
>>>>>>
>>>>>> Thanks for trying out the latest build!
>>>>>>
>>>>>> I'll make sure this gets mentioned in the release note for Sequenced
>>>>>> Collections.
>>>>>> We'll also raise this issue when we talk about this feature in 
>>>>>> the Quality
>>>>>> Outreach
>>>>>> program.
>>>>>>
>>>>>> s'marks
>>>>>>
>>>>>> On 4/29/23 3:46 AM, Remi Forax wrote:
>>>>>>> I've several repositories that now fails to compile with the 
>>>>>>> latest jdk21, which
>>>>>>> introduces sequence collections.
>>>>>>>
>>>>>>> The introduction of a common supertype to existing collections 
>>>>>>> is *not* a source
>>>>>>> compatible change because of type inference.
>>>>>>>
>>>>>>> Here is a simplified example:
>>>>>>>
>>>>>>>      public static void m(List<Supplier<? extends Map<String, 
>>>>>>> String>>> factories) {
>>>>>>>      }
>>>>>>>
>>>>>>>      public static void main(String[] args) {
>>>>>>> Supplier<LinkedHashMap<String,String>> supplier1 = 
>>>>>>> LinkedHashMap::new;
>>>>>>>        Supplier<SortedMap<String,String>> supplier2 = TreeMap::new;
>>>>>>>        var factories = List.of(supplier1, supplier2);
>>>>>>>        m(factories);
>>>>>>>      }
>>>>>>>
>>>>>>>
>>>>>>> This example compiles fine with Java 20 but report an error with 
>>>>>>> Java 21:
>>>>>>>      SequencedCollectionBug.java:28: error: method m in class 
>>>>>>> SequencedCollectionBug
>>>>>>>      cannot be applied to given types;
>>>>>>>        m(factories);
>>>>>>>        ^
>>>>>>>      required: List<Supplier<? extends Map<String,String>>>
>>>>>>>      found:    List<Supplier<? extends 
>>>>>>> SequencedMap<String,String>>>
>>>>>>>      reason: argument mismatch; List<Supplier<? extends 
>>>>>>> SequencedMap<String,String>>>
>>>>>>>      cannot be converted to List<Supplier<? extends 
>>>>>>> Map<String,String>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Apart from the example above, most of the failures I see are in 
>>>>>>> the unit tests
>>>>>>> provided to the students, because we are using a lot of 'var' in 
>>>>>>> them so they
>>>>>>> work whatever the name of the types chosen by the students.
>>>>>>>
>>>>>>> Discussing with a colleague, we also believe that this bug is 
>>>>>>> not limited to
>>>>>>> Java, existing Kotlin codes will also fail to compile due to 
>>>>>>> this bug.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Rémi