Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch

[Uwe Schindler]

Hi Jochen,

thank you for the information! Is there any plan about a release? I also found no JIRA issue about this issue to link it against our JIRA: https://issues.apache.org/jira/browse/LUCENE-7596

The problem makes our build system unusable, so it would be very important to have a fix quite soon! As our Ant/Ivy-based build relies on Maven Central, it would be good to have the bugfix release available there, which requires a release. I think the same applies for Gradle users (Elasticsearch).

As a temporary workaround we might be able to use the Apache Snapshot repository, but this is not allowed if we do a release of Lucene.

Uwe

-----
Uwe Schindler
uschindler at apache.org 
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/

> -----Original Message-----
> From: Jochen Theodorou [mailto:blackdrag at gmx.org]
> Sent: Saturday, December 10, 2016 9:23 AM
> To: Uwe Schindler <uschindler at apache.org>; jigsaw-dev at openjdk.java.net;
> Core-Libs-Dev <core-libs-dev at openjdk.java.net>
> Subject: Re: Java 9 build 148 causes trouble in Apache
> Lucene/Solr/Elasticsearch
> 
> On 09.12.2016 23:32, Uwe Schindler wrote:
> > Hi,
> >
> > I updated our Jenkins server for the JDK 9 preview testing to use build 148.
> Previously we had build 140 and build 147, which both worked without any
> issues. But after the update the following stuff goes wrong:
> >
> > (1) Unmapping of direct buffers no longer works, although this API was
> marked as critical because there is no replacement up to now, so code can
> unmap memory mapped files, which is one of the most important things
> Apache Lucene needs to use to access huge random access files while
> reading the index. Without memory mapping, the slowdown for Lucene
> users will be huge
> >
> > This is caused by the recent Jigsaw changes, published in build 148.
> Unfortunately we did not test the Jigsaw builds, so we would have noticed
> that earlier. Basically the following peace of code fails now (with or without
> doPrivileged and with/without security manager):
> >
> >        final Class<?> directBufferClass =
> Class.forName("java.nio.DirectByteBuffer");
> >
> >        final Method m = directBufferClass.getMethod("cleaner");
> >        m.setAccessible(true);
> >        MethodHandle directBufferCleanerMethod = lookup.unreflect(m);
> >        Class<?> cleanerClass =
> directBufferCleanerMethod.type().returnType();
> >        // build method handle for unmapping, full code is here:
> https://goo.gl/TfQWl6
> 
> I guess that is the effect of #AwkwardStrongEncapsulation. I would
> advise doing regular checks against the jigsaw builds to know about such
> problems in the future earlier... but seeing your code break without an
> obvious good solution sure is stressful. I feel with you.
> 
> [...]
> > (2) A second thing  we noticed is that Groovy no longer works and dies with
> strange error messages.
> 
> That is because versions including Groovy 2.4.7 are using
> setAccessible(AccessibleObject[] array, true), and the array will also
> include private methods or fields. This worked till
> #AwkwardStrongEncapsulation because will then a class was either
> exported and its method can all be made accessible or not. For example
> on GAE or earlier versions of the module system. Now an exported class
> may break this, since its private methods can no longer be made
> accessible using setAccessible.
> 
> A fix for this is already committed, we are only waiting for release of
> Groovy 2.4.8. Of course even with the fix Groovy code can possibly
> break... for example if you did the direct buffer access in Groovy.
> 
> Btw, do not hesitate to ask about such problems on groovy-user, please.
> 
> bye Jochen