Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch

[Uwe Schindler]

Hi Andrew,

Thanks for taking care! Your proposal is still the "volatile only during safepoint" idea? I remember our discussions last FOSDEM and this is why I brought it up in this mail thread (see my original mail referring to your name).

I was aware, that Java 10 would be a good place for the "correct fix that no longer relies on GC". For Java 9, we should go the current route and add the Unsafe method to forcefully unmap. The new unmap method is now needed, because the recent Jigsaw enhancements make it completely impossible to do setAccessible on JDK modules by default so the internal Cleaner is unreachable. Lucene is happy to get a "dirty workaround" for Java 9 and we can then wait for Java 10 where we get a public close() method on MappedByteBuffer / DirectBuffer.

Uwe

-----
Uwe Schindler
uschindler at apache.org 
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> -----Original Message-----
> From: jigsaw-dev [mailto:jigsaw-dev-bounces at openjdk.java.net] On Behalf
> Of Andrew Haley
> Sent: Monday, December 12, 2016 3:00 PM
> To: jigsaw-dev at openjdk.java.net
> Subject: Re: Java 9 build 148 causes trouble in Apache
> Lucene/Solr/Elasticsearch
> 
> On 11/12/16 18:33, Peter Levart wrote:
> > In order for deallocation to be effective and, above all, safe, user has
> > to know the whole story of a buffer (s)he intends to deallocate and the
> > story of all possible derived buffers. I don't believe one can create a
> > safe program by choosing to deallocate a direct buffer for which (s)he
> > does not know where it came from, because then (s)he also doesn't know
> > what other buffers might still be using the same piece of memory.
> 
> This is JDK-4724038.  It is possible to unmap a MappedByteBuffer
> safely with little effect on efficiency: my basic idea is to add an
> indirection which all callers (even those using derived buffers) have
> to use to access the memory, and then removing most uses of this
> indirection with a little compiler magic.  A few will still remain, but
> these will be the minimum required to guarantee security and won't
> affect the speed of most operations.
> 
> I have a plan to implement this, but it requires changes to the Java
> Memory Model and to the compilers.  I didn't get it done in time for
> JDK 9 because of being distracted by a ton of other things, but I hope
> I'll get it done by JDK 10.
> 
> Andrew.