JEP 442: Foreign Function & Memory API => why is it again preview API?

Tue May 16 17:22:50 UTC 2023

Hi Alan,

Am 16.05.2023 um 18:07 schrieb Alan Bateman:
> On 16/05/2023 16:38, Uwe Schindler wrote:
>> :
>>
>> Is it really needed to have a next round of preview API? Why not have 
>> Java 21 with Foreign API finalized in the same way like virtual 
>> threads? To me it is strange that an API like virtual threads was 
>> thrown to end-users (it caused problems at many places including 
>> slowdowns we have seen when preview was enabled in Java 19 and 20) in 
>> very short time (I never noticed a incubation period). 
>
> Would it be possible to expand on "it caused problems at many places"? 
> I'm wondering if there are issues that never made it to JBS.

It was mainly the slowdowns when preview features were enabled. We were 
testing at the beginning with our nightly Lucene performance tests while 
preview was enabled at same time (reason was because at that time we had 
no way to enable MemorySegment for our users without them passing 
--enable-preview). So the combination caused a major slowdown especially 
when writing index files, but also some queries went down.

At first we thought it may be the MemorySegment API, but this changed 
soon later. The cool idea came after reading the preview API spec 
several times and thinking about how to get rid of the preview flag in 
our class files. Basically it works like the "--release" flag in JDK and 
the related "ct.sym" file, just simpified a bit.

Since Lucene 9.5 we now use MemorySegment preview API without requiring 
the user to enable it. At first (Lucene 9.5, Java 19 support only) we 
stipped the preview flag and later (Lucene 9.6, Java 19 and Java 20) we 
changed to another approach (see below). Users can still opt-out by a 
system property, but behind the scenes we changed our Gradle compilation 
logic to compile with our main compiler JDK (17 in main, 11 in Lucene 
9.x) against stubs of the new APIs etracted from Java 19, Java 20 and 
now also Java 21 (without those stubs, developers compiling Lucene would 
have needed to install Java 11/17 in addition to 19, 20, 21 for the 
MR-JAR parts which is not manageable, especially as Gradle doe not allow 
EA builds auto-provisioned). Those stubs extracted from JDK simply have 
the preview flag removed and are injected into compiler with 
patch-module. We carefully do all pre-checks against runtime version 
before enabling the API and loading the classes from the MR-JAR. But it 
allows our downstream users to use the API as if it would be officially 
supported.

When we compiled against our stubs and then no longer had to pass 
"--enable-preview", the nightly benchmarks got *much* faster; faster 
than with the problematic ByteBuffer API (which was the win we were all 
hoping of; in addition to safe unmapping of mmapped files. The speedup 
has to do with the fact that we can now memory map huge files in one go 
and are not limited to 2 Gigabytes adding a lot of branches in Lucene's 
code that cannot be optimized away).

So for Apache Lucene (+ Solr / Elasticserach) the MemorySegment API is a 
huge win. We have now already 2 releases with it (supporting Java 19 and 
Java 20) and it helps.

> As regards "slowdowns" with preview features enabled, I'm guessing you 
> mean JDK-8300002 [1]. Can you confirm that these issues you observed 
> do not exist with the JDK 21 EA builds?

Yes that's likely the reason. I had the feeling it was caused by the 
additional manual scheduling on waiting for IO in various 
InputStream/OutputStream/Files/... classes. This code in those classes 
is another thing why I hate the virtual threads so much: It makes some 
IO code in the JDK unreadable.

Another problem was/is memory exhaustion by people using ThreadLocals. 
We already removed all ThreadLocals from Lucene, so it is no longer an 
issue, but to me this all sounds like a release of something which is 
not ready for primetime.

We have not yet tested with Java 21 and "--enable-preview" flag enabled 
(as we no longer need it), but I will contact Mike McCandless to do it.

P.S.: Another reason why many core Lucene/SOlr/Elasticsearch people are 
unhappy about Virtual Threads is the statement by one of its maintainers 
that they are the reason for deprecation and removal of Security Manager 
without any useful replacement. So please excuse that I start this 
discussion again, it has to be said.

>> In contrast the foreign memory API is very stable already since Java 
>> 19 and the changes were minimal in 21. It exists now since Java 14, 
>> where it first appeared as incubation. I know theres still work on 
>> the foreign linker, but why aren't the stable classes like 
>> MemorySegment, ValueLayout not public now?
>
> Are you on the panama-dev list? Maurizio sent status in December 2022 
> [2]. JEP 442, proposing a 3rd preview, has been on the technical 
> roadmap since March [3]. So it shouldn't be a big surprise. The memory 
> APIs has been very difficult to get right, hats off to Maurizio and 
> team for the persistence.

Yes, I am working with Maurizio. I wrote this email to the main list in 
the hope you would change your mid and at least release the "stable" 
parts of the API. I got Maurizio's answer in the other mail.

It looks like we will create new stub JARs for the next JDK releases to 
compile against and need to add hard checks on used Java versions. 
That's fine with our compilation trick above, but it is a lot of work to 
maintain .java files for many Java versions so I had the hope that this 
ends and we can make our MR-JAR in a way that "starting with Java 21 
only a single set of classes is needed for all later versions".

If you have any questions about how the above compilation tricks work, I 
can give a close insight to anybody interested. The code is public 
available, the first PR adding Java 20 support is here: 
https://github.com/apache/lucene/pull/12188/files

The code for generating the stub files is also there.

Java 21 support for Lucene is here (it just adds new stubs): 
https://github.com/apache/lucene/pull/12294/files

Uwe

P.S.: Don't hurt me for programming around the preview limitations! It 
is all according to spec, it explicitely says you can use preview APIs 
with runtime checks and with own class files or reflection.

> -Alan
>
> [1] https://bugs.openjdk.org/browse/JDK-8300002
> [2] 
> https://mail.openjdk.org/pipermail/panama-dev/2022-December/018182.html
> [3] https://mail.openjdk.org/pipermail/jdk-dev/2023-March/007473.html

-- 
Uwe Schindler
uschindler at apache.org
ASF Member, Member of PMC and Committer of Apache Lucene and Apache Solr
Bremen, Germany
https://lucene.apache.org/
https://solr.apache.org/