<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi again,</p>
<p>about Alan's comment:</p>
<p>> MR JARs are already complicated and it would greatly
complicate it further if version sections were further divided to
support classes compiled to make use of preview features.</p>
<p>I think when the Preview specification was written, maybe some
section like this would have been nice to have. The only change
that would have been needed is in the classloading logic
(searching for class file in ClassLoader):</p>
<blockquote>
<p>When the JVM/classloader searches for a class, and it figures
out that the version in a feature-version specific MR-JAR part
is a preview class (based on minor version 65535), it should
ignore it unless the JVM's feature version is exactly the one of
the MR-JAR section. Because it is treated as "does not exist",
it would go down to lower feature versions in the versions
folder until a valid one is found.</p>
</blockquote>
<p>This would have been an backwards compatible change, as it would
only affect classes marked as preview. This would have made it
simple for use, because we could have used a real MR-JAR in
Lucene. Now we have a provider class (see previous mail) and some
try/catch logic with UnsupportedClassVersionError as signal to
detect preview and/or wrong version.<br>
</p>
<p>> I'll bet Uwe/team understand that and they will need to add
support for JDK 20 anyway. It looks like things are shaping up to
have at least some shimmer in the j.l.foreign API in Java 20 so
they probably have at least some work to do.</p>
<p>Yes, see previous mail. The MR-JAR is not enough and only partly
fixes the problems, so there's more logic in our code. Please
review it in my previous mail. :-)<br>
</p>
<div class="moz-cite-prefix">Am 01.10.2022 um 10:49 schrieb Uwe
Schindler:<br>
</div>
<blockquote type="cite"
cite="mid:B404339F-4A44-46ED-BE2B-A069F20197A3@apache.org">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
Hi Alan, hi Remi,<br>
<br>
Of course the MR-JAR is only half of the story. Magically it will
work (and with java 20 EA it just logs a warning and uses the old
impl).<br>
<br>
It works like this:<br>
<br>
We compile the classes using a separate Geadle source set using
toolkit auto-provisioning. There's also some SPI like
factory/provider API on our side for IndexInputs. Class names are
also different to the old code (IndexInputs, Provider), so
actually the MR-JAR is not needed. Theoretically we could package
them into the main part of JAR file compiled against Java 11. But
this would complicate build and IDEs could stumble on this. So the
additional new classes are isolated in MR-JAR.<br>
<br>
The service provider like code tries to instantiate the
MemoryMappingIndexInputProvider using a Lookup and method handle
(no reflection as the class is PKG protected, so with lookup works
without accessible hacks). On Java before 19 it will fail to load
the class (not found due to MR-JAR). On Java 19 or later it will
find the class and tries to load it. As it is marked as "using
preview", without preview enabled it will throw classformat error.
When preview is enabled, it succeeds to load (voila). If java 20
is used, it will also find the class file, but it always gets some
classformat error. We will in Java 20 add another source set (this
time with same provider class name). The same finally in Java 21.
<br>
<br>
On all loading errors of our provider class it falls back to old
impl. We have tests for all this including java 20 through CI.
When java 20 comes out we add another source set,...<br>
<br>
The SPI like code is in linked pull request, or here: <a
href="https://github.com/apache/lucene/blob/branch_9x/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java#L329"
moz-do-not-send="true" class="moz-txt-link-freetext">https://github.com/apache/lucene/blob/branch_9x/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java#L329</a><br>
<br>
Btw: It is now released, happy testing: <a
href="https://lucene.apache.org/core/corenews.html#apache-lucenetm-940-available"
moz-do-not-send="true" class="moz-txt-link-freetext">https://lucene.apache.org/core/corenews.html#apache-lucenetm-940-available</a><br>
<br>
Uwe<br>
<br>
<div class="gmail_quote">Am 30. September 2022 11:11:56 MESZ
schrieb Alan Bateman <a class="moz-txt-link-rfc2396E" href="mailto:Alan.Bateman@oracle.com"><Alan.Bateman@oracle.com></a>:
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
<pre dir="auto" class="k9mail">On 30/09/2022 08:40, Remi Forax wrote:
<blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">Hi Uwe,
i wonder if you will not be in trouble when Java 20 will be out with your multi-releases jar,
because the VM 10 will try to run the bytecodes under the 19 folder and because it is compiled with --enable-preview --source 19, it will not work.
(Alan in CC, to check i'm not saying something stupid.)
</blockquote>That's right. JDK 19 with --enable-preview will work. JDK 20 EA builds with --enable-preview will fail because the class file version will be 63.65535 rather than 64.65535. MR JARs are already complicated and it would greatly complicate it further if version sections were further divided to support classes compiled to make use of preview features. I'll bet Uwe/team understand that and they will need to add support for JDK 20 anyway. It looks like things are shaping up to have at least some shimmer in the j.l.foreign API in Java 20 so they probably have at least some work to do.
-Alan
</pre>
</blockquote>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
Uwe Schindler
<a class="moz-txt-link-abbreviated" href="mailto:uschindler@apache.org">uschindler@apache.org</a>
ASF Member, Member of PMC and Committer of Apache Lucene and Apache Solr
Bremen, Germany
<a class="moz-txt-link-freetext" href="https://lucene.apache.org/">https://lucene.apache.org/</a>
<a class="moz-txt-link-freetext" href="https://solr.apache.org/">https://solr.apache.org/</a></pre>
</body>
</html>