Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch

[Uwe Schindler]

Thanks Alan,

I will look forward to see a solution for this! THANKS! A static method somewhere in jdk.internal to trigger the unmapping of a MappedByteBuffer would be fine. This is easy to change in our code, we just need a MethodHandle to a function with the following signature: static void unmap(MappedByteBuffer)

Our current code just generates the MethodHandle with exactly that signature and functionality by some guardWithTest, filterArguments,... and using the Cleaner as Java 8 internal class or as Runnable in Java 9: https://goo.gl/TfQWl6

The second thing was how to make a JEP proposal for solving the underlying problem in Java 10? As said before, on last FOSDEM we had some ideas how to make Hotspot able to unmap without the risk that the JVM SIGSEGV/SIGBUS or exposes private data. This would need some Hotspot changes ("volatile only during safepoints"), the idea was proposed by Andrew Haley.

Uwe

-----
Uwe Schindler
uschindler at apache.org 
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/

> -----Original Message-----
> From: Alan Bateman [mailto:Alan.Bateman at oracle.com]
> Sent: Saturday, December 10, 2016 6:14 AM
> To: Uwe Schindler <uschindler at apache.org>; jigsaw-dev at openjdk.java.net;
> Core-Libs-Dev <core-libs-dev at openjdk.java.net>
> Subject: Re: Java 9 build 148 causes trouble in Apache
> Lucene/Solr/Elasticsearch
> 
> On 09/12/2016 22:32, Uwe Schindler wrote:
> 
> > Hi,
> >
> > I updated our Jenkins server for the JDK 9 preview testing to use build 148.
> Previously we had build 140 and build 147, which both worked without any
> issues. But after the update the following stuff goes wrong:
> >
> > (1) Unmapping of direct buffers no longer works, although this API was
> marked as critical because there is no replacement up to now, so code can
> unmap memory mapped files, which is one of the most important things
> Apache Lucene needs to use to access huge random access files while
> reading the index. Without memory mapping, the slowdown for Lucene
> users will be huge
> sun.misc.Cleaner was indeed on the original list of APIs for JEP 260 to
> identify as a "critical internal API". It turned out not to be useful
> because it would have required some way to get the Cleaner in the first
> place. That lead to the "new" hack that is reading the private "cleaner"
> field from DBB and treating it as a Runnable. That hack now breaks
> because setAccessible has changed in jdk-9+148 to align with the JSR 376
> proposal tracked as #AwkwardStrongEncapsulation.
> 
> No need to panic though, there is an update to JEP 260 coming soon for
> this specific need. Details TDB but it will probably be a method in
> jdk.unsupported module. It does mean that libraries using the old (or
> "new") hacks will need to change. I hope it will be seen as a reasonable
> compromise for this generally awkward issue.
> 
> -Alan