<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi Remi, hi all,<br>
</p>
<div class="moz-cite-prefix">I'd like to add some information from
open source projects and why I don't see the problem discussed
here is a really serious one.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Background: We tested Apache Lucene and
Apache Solr with Java 21. The compilation with Gradle went fine.
So actually there are no problems with the new superclasses. We
have extensive use of chains of stream() calls with Stream.of()
and similar apis. Use of "var" is still seldom but we use it now
when newly introduced code around streams is added to spare
verbosity. But still we got no problems. But why is this so? <br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">A good open source project should
trigger the compiler with "--release". Apache Lucene uses Java 17
on main branch and Java 11 on 9.x branch. In both cases
compilation worked due to the use of "--release". If we would
change to Java 21 as compilation target, we may need to adapt our
code.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">There are some problems with that:</div>
<div class="moz-cite-prefix">
<ul>
<li>Not all projects use "--release", some projects still use
"--source --target". The problem with that is Maven and Gradle
still not making "--release" a first class citizen. Default
configs only use "--source --target".</li>
<li>Code still on Java 8 can't use "--release", as the compiler
does not support it. The Lucene 8.x branch still open for
bugfixes has a trick: It detects the compiler and if it is
Java 8 it passes "--source 8 --target 8", while starting with
Java 9 compiler it passes "--release 8". On the other hand
code still supporting java 8 is unlikely affected by the
problem, as it cannot use "var". But still chains of
Stream.of().foo().bar() may still be affected. <br>
</li>
</ul>
</div>
<div class="moz-cite-prefix">What is a more serious
source-incompatibility issue that I would always report to OpenJDK
bug tracker: During testing Java 20 we were trapped by a compiler
change that caused a source incompatibility (which was reverted,
see <a class="moz-txt-link-freetext" href="https://bugs.openjdk.org/browse/JDK-8299416">https://bugs.openjdk.org/browse/JDK-8299416</a>). So not even
passing "--release" fixed the issue, because the compiler changed
its semantics. This is in my opinion a breaking issue because it
prevents code from compiling!</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">The changes in sequenced collections
should not be a too big problem for the community if they have
setup their projects correctly.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Uwe</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">P.S.: To be honest: I tried to pass
"--release 21" when compiling Lucene and it failed, but not for
sequenced collections reasons. It was more some tests calling
Runtime#runFinalization().<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Am 05.05.2023 um 13:14 schrieb
<a class="moz-txt-link-abbreviated" href="mailto:forax@univ-mlv.fr">forax@univ-mlv.fr</a>:<br>
</div>
<blockquote type="cite"
cite="mid:1907607213.49439971.1683285286489.JavaMail.zimbra@univ-eiffel.fr">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div style="font-family: arial, helvetica, sans-serif; font-size:
12pt; color: #000000">
<div>Hi Joe,<br>
</div>
<div>in this peculiar case, there are several reasons to be
worried compared to other potential breaking changes that has
appeared in the past (see the message of Tagir for an
example).<br data-mce-bogus="1">
</div>
<div><br data-mce-bogus="1">
</div>
<div>Unlike other changes<br data-mce-bogus="1">
</div>
<div>- this one touch the collection API, and those
interfaces/types are widely used,<br>
</div>
<div>- we know that the source compatibility changes occurs
mostly if 'var' or the "new" inference algorithm (the one from
Java 8), so this is likely that most of the issues will be
found in Java 11+ source code,</div>
<div>- this changes may also affect all typed languages based on
the JVM, not only Java. Corpus of codes in Groovy, Kotlin and
Scala also need to be checked. In case of Kotlin and Scala,
'var' is the default behavior but they have their own
collections (or type system around collections in case of
Kotlin), so knowing the real impact of this change is hard
here.<br data-mce-bogus="1">
</div>
<div><br data-mce-bogus="1">
</div>
<div>The problem of using a corpus experiment is that the corpus
may not represent the current state of the Java ecosystem, or
at least the one that may be impacted.<br data-mce-bogus="1">
</div>
</div>
</blockquote>
The problem with the corpus experiment is also that you need to be
aware that most moden open source projects use "--release" flag, so
you have to patch it away from the build system.<br>
<blockquote type="cite"
cite="mid:1907607213.49439971.1683285286489.JavaMail.zimbra@univ-eiffel.fr">
<div style="font-family: arial, helvetica, sans-serif; font-size:
12pt; color: #000000">
<div>In my case, on my own repositories (public and private), i
had only one occurrence of the issue in the main source codes
because many of those repositories are not using 'var' or even
the stream API but on the corpus of the unit tests we give to
students to check their implementations, little less than a
third of those JUnit classes had source compatibility issues
because those tests are using 'var' and different collections
heavily.<br data-mce-bogus="1">
</div>
<div><br data-mce-bogus="1">
</div>
<div>And the situation is a little worst than that because in
between now and the time people will use Java 21, a lot of
codes will be written using Java 11 and 17 and may found
incompatible later.<br data-mce-bogus="1">
</div>
<div><br data-mce-bogus="1">
</div>
<div>A source incompatibility issue is not a big deal, as said
in this thread, most of the time, explicitly fixing the type
argument instead of inferring it make the code compile again.<br
data-mce-bogus="1">
</div>
<div>So the house is not burning, but we should raise awareness
of this issue given that it may have a bigger impact than
other source incompatible changes that occur previously.<br
data-mce-bogus="1">
</div>
<div><br data-mce-bogus="1">
</div>
<div>Rémi<br data-mce-bogus="1">
</div>
<div><br>
</div>
<hr id="zwchr" data-marker="__DIVIDER__">
<div data-marker="__HEADERS__">
<blockquote style="border-left:2px solid
#1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
</b>"joe darcy" <a class="moz-txt-link-rfc2396E" href="mailto:joe.darcy@oracle.com"><joe.darcy@oracle.com></a><br>
<b>To: </b>"Ethan McCue" <a class="moz-txt-link-rfc2396E" href="mailto:ethan@mccue.dev"><ethan@mccue.dev></a>,
"Raffaello Giulietti" <a class="moz-txt-link-rfc2396E" href="mailto:raffaello.giulietti@oracle.com"><raffaello.giulietti@oracle.com></a><br>
<b>Cc: </b>"Remi Forax" <a class="moz-txt-link-rfc2396E" href="mailto:forax@univ-mlv.fr"><forax@univ-mlv.fr></a>, "Stuart
Marks" <a class="moz-txt-link-rfc2396E" href="mailto:stuart.marks@oracle.com"><stuart.marks@oracle.com></a>, "core-libs-dev"
<a class="moz-txt-link-rfc2396E" href="mailto:core-libs-dev@openjdk.java.net"><core-libs-dev@openjdk.java.net></a><br>
<b>Sent: </b>Friday, May 5, 2023 4:38:16 AM<br>
<b>Subject: </b>Re: The introduction of Sequenced
collections is not a source compatible change<br>
</blockquote>
</div>
<div data-marker="__QUOTED_TEXT__">
<blockquote style="border-left:2px solid
#1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;">
<p>A few comments on the general compatibility policy for
the JDK. Compatibility is looked after by the
Compatibility and Specification Review (CSR) process (
Compatibility & Specification Review). Summarizing the
approach,</p>
<p> </p>
<blockquote>The general compatibility policy for exported
APIs implemented in the JDK is:<br>
<br>
* Don't break binary compatibility (as defined in the
Java Language Specification) without sufficient cause.<br>
* Avoid introducing source incompatibilities.<br>
* Manage behavioral compatibility changes.</blockquote>
<p><a class="moz-txt-link-freetext"
href="https://wiki.openjdk.org/display/csr/Main"
target="_blank" moz-do-not-send="true">https://wiki.openjdk.org/display/csr/Main</a><br>
</p>
<p>None of binary, source, and behavioral compatibly are
absolutes and judgement is used to assess the
cost/benefits of changes. For example, strict source
compatibility would preclude, say, introducing new public
types in the java.lang package since the implicit import
of types in java.lang could conflict with a same-named
type *-imported from another package.</p>
<p>When a proposed change is estimated to be sufficiently
disruptive, we conduct a corpus experiment to evaluate the
impact on the change on many public Java libraries. Back
in Project Coin in JDK 7, that basic approach was used to
help quantify various language design choices and the
infrastructure to run such experiments has been built-out
in the subsequent releases.</p>
<p>HTH,<br>
</p>
<p>-Joe<br>
CSR Group Lead<br>
</p>
<div class="moz-cite-prefix">On 5/4/2023 6:32 AM, Ethan
McCue wrote:<br>
</div>
<blockquote
cite="mid:CA+NR86hdGHFN7j_eWsBVznsCojZ2GthV-1zzhvgEra0=FtQLRQ@mail.gmail.com">
<div dir="auto">I guess this a good time to ask, ignoring
the benefit part of a cost benefit analysis, what
mechanisms do we have to measure the number of codebases
relying on type inference this will break?
<div dir="auto"><br>
</div>
<div dir="auto">Iirc Adoptium built/ran the unit tests
of a bunch of public repos, but it's also a bit
shocking if the jtreg suite had nothing for this.</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, May 4, 2023,
9:27 AM Raffaello Giulietti <<a
href="mailto:raffaello.giulietti@oracle.com"
class="moz-txt-link-freetext" target="_blank"
moz-do-not-send="true">raffaello.giulietti@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Without
changing the semantics at all, you could also write<br>
<br>
final List<Collection<String>>
list = <br>
Stream.<Collection<String>>of(nestedDequeue,
nestedList).toList();<br>
<br>
to "help" type inference.<br>
<br>
<br>
<br>
<br>
On 2023-05-03 15:12, <a
href="mailto:forax@univ-mlv.fr" target="_blank"
rel="noreferrer" class="moz-txt-link-freetext"
moz-do-not-send="true">forax@univ-mlv.fr</a> wrote:<br>
> Another example sent to me by a fellow French
guy,<br>
> <br>
> final Deque<String> nestedDequeue =
new ArrayDeque<>();<br>
> nestedDequeue.addFirst("C");<br>
> nestedDequeue.addFirst("B");<br>
> nestedDequeue.addFirst("A");<br>
> <br>
> final List<String> nestedList = new
ArrayList<>();<br>
> nestedList.add("D");<br>
> nestedList.add("E");<br>
> nestedList.add("F");<br>
> <br>
> final List<Collection<String>>
list = Stream.of(nestedDequeue, nestedList).toList();<br>
> <br>
> This one is cool because no 'var' is involved and
using collect(Collectors.toList()) instead of toList()
solves the inference problem.<br>
> <br>
> Rémi<br>
> <br>
> ----- Original Message -----<br>
>> From: "Stuart Marks" <<a
href="mailto:stuart.marks@oracle.com"
target="_blank" rel="noreferrer"
class="moz-txt-link-freetext" moz-do-not-send="true">stuart.marks@oracle.com</a>><br>
>> To: "Remi Forax" <<a
href="mailto:forax@univ-mlv.fr" target="_blank"
rel="noreferrer" class="moz-txt-link-freetext"
moz-do-not-send="true">forax@univ-mlv.fr</a>><br>
>> Cc: "core-libs-dev" <<a
href="mailto:core-libs-dev@openjdk.java.net"
target="_blank" rel="noreferrer"
class="moz-txt-link-freetext" moz-do-not-send="true">core-libs-dev@openjdk.java.net</a>><br>
>> Sent: Tuesday, May 2, 2023 2:44:28 AM<br>
>> Subject: Re: The introduction of Sequenced
collections is not a source compatible change<br>
> <br>
>> Hi Rémi,<br>
>><br>
>> Thanks for trying out the latest build!<br>
>><br>
>> I'll make sure this gets mentioned in the
release note for Sequenced<br>
>> Collections.<br>
>> We'll also raise this issue when we talk
about this feature in the Quality<br>
>> Outreach<br>
>> program.<br>
>><br>
>> s'marks<br>
>><br>
>> On 4/29/23 3:46 AM, Remi Forax wrote:<br>
>>> I've several repositories that now fails
to compile with the latest jdk21, which<br>
>>> introduces sequence collections.<br>
>>><br>
>>> The introduction of a common supertype to
existing collections is *not* a source<br>
>>> compatible change because of type
inference.<br>
>>><br>
>>> Here is a simplified example:<br>
>>><br>
>>> public static void
m(List<Supplier<? extends Map<String,
String>>> factories) {<br>
>>> }<br>
>>><br>
>>> public static void main(String[]
args) {<br>
>>>
Supplier<LinkedHashMap<String,String>>
supplier1 = LinkedHashMap::new;<br>
>>>
Supplier<SortedMap<String,String>>
supplier2 = TreeMap::new;<br>
>>> var factories = List.of(supplier1,
supplier2);<br>
>>> m(factories);<br>
>>> }<br>
>>><br>
>>><br>
>>> This example compiles fine with Java 20
but report an error with Java 21:<br>
>>> SequencedCollectionBug.java:28:
error: method m in class SequencedCollectionBug<br>
>>> cannot be applied to given types;<br>
>>> m(factories);<br>
>>> ^<br>
>>> required: List<Supplier<?
extends Map<String,String>>><br>
>>> found: List<Supplier<?
extends SequencedMap<String,String>>><br>
>>> reason: argument mismatch;
List<Supplier<? extends
SequencedMap<String,String>>><br>
>>> cannot be converted to
List<Supplier<? extends
Map<String,String>>><br>
>>><br>
>>><br>
>>><br>
>>> Apart from the example above, most of the
failures I see are in the unit tests<br>
>>> provided to the students, because we are
using a lot of 'var' in them so they<br>
>>> work whatever the name of the types
chosen by the students.<br>
>>><br>
>>> Discussing with a colleague, we also
believe that this bug is not limited to<br>
>>> Java, existing Kotlin codes will also
fail to compile due to this bug.<br>
>>><br>
>>> Regards,<br>
>>> Rémi<br>
</blockquote>
</div>
</blockquote>
<br>
</blockquote>
</div>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
Uwe Schindler
<a class="moz-txt-link-abbreviated" href="mailto:uschindler@apache.org">uschindler@apache.org</a>
ASF Member, Member of PMC and Committer of Apache Lucene and Apache Solr
Bremen, Germany
<a class="moz-txt-link-freetext" href="https://lucene.apache.org/">https://lucene.apache.org/</a>
<a class="moz-txt-link-freetext" href="https://solr.apache.org/">https://solr.apache.org/</a></pre>
</body>
</html>