Great news: Lucene 9.4 (optionally) uses mapped MemorySegments when JDK 19 is detected

[Uwe Schindler]

Hi Alan,

> There's nothing to force all class files in a MR JAR versioned section 
> to have the same class file version. A versioned section may contain 
> resources that aren't class files. It's important that a MR JAR 
> provide a consistent view at run-time and that means that APIs such as 
> ClassLoader.getResource need to be consistent with class loading. It 
> gets further complicated once there URLs to resources in a MR JAR come 
> into existence.  So on the surface, having a class loader attempt to 
> load a class from versions/19 and fallback to older versioned sections 
> on CFE might look simple, it fraught with issues. I've no doubt that 
> it would be feasible to extend the MR JAR format to support version 
> entries for class files compiled with preview features but it would 
> add more complications and risks the curse of dimensionality.

I agree with all your points and I was also thinking about resources, 
but I came to a slightly different decission. At moment there's nothing 
that ensures the resources are loaded from same folder or are consistent 
with the classes at runtime. So in my opinion, I would have reduced the 
whole MR-JAR classloading to "go through the feature version folders 
backwards starting from current Java version and pick the first class 
found that is no preview or "has correct major version and is marked as 
preview".

Of course one can discuss about this for long time and there are 
arguments pro and con. This is just my personal opinion. There's also 
nothing we can change anymore, so we should live what's there.

I may write a blog post about how we did it in Lucene. Our approach is a 
combination of MR-JAR (but that's only to "hide" the preview classes 
from IDEs of older version) and a service provider lookup at runtime.

> As a general point, preview features were envisaged for developers 
> using a preview feature directly. In the discussion here we can think 
> of Lucene as a library that is published with class files using 
> preview features. You don't want to force all users to run with JDK 19 
> and preview features enabled so you've got code to handle all that. 
> It's great that you got this working, it's just not clear to me that 
> the platform should go out of its way to make this easier.

Thanks for this feedback. We see it exactly as you described it. Lucene 
is a library and developers out there should be able to test new JDK 
preview features with it. After reading the Preview Spec I was able to 
implement this (the notes at end of the preview spec about reflection 
were very useful to get it running).

For Lucene, a downstream developer does not need to compile his own 
classes with --enable-preview (never gets in touch with JDK classes 
marked as preview), but a downstream developer or a user of 
Elasticsearch and/or Solr can test our new implementation. This allows 
users and developers to test the new implementation. To me this is 
already a success of preview APIs. With the previous incubation this was 
not "easy doable".

Uwe

-- 
Uwe Schindler
uschindler at apache.org
ASF Member, Member of PMC and Committer of Apache Lucene and Apache Solr
Bremen, Germany
https://lucene.apache.org/
https://solr.apache.org/