Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch

[Jochen Theodorou]

On 09.12.2016 23:32, Uwe Schindler wrote:
> Hi,
>
> I updated our Jenkins server for the JDK 9 preview testing to use build 148. Previously we had build 140 and build 147, which both worked without any issues. But after the update the following stuff goes wrong:
>
> (1) Unmapping of direct buffers no longer works, although this API was marked as critical because there is no replacement up to now, so code can unmap memory mapped files, which is one of the most important things Apache Lucene needs to use to access huge random access files while reading the index. Without memory mapping, the slowdown for Lucene users will be huge
>
> This is caused by the recent Jigsaw changes, published in build 148. Unfortunately we did not test the Jigsaw builds, so we would have noticed that earlier. Basically the following peace of code fails now (with or without doPrivileged and with/without security manager):
>
>        final Class<?> directBufferClass = Class.forName("java.nio.DirectByteBuffer");
>
>        final Method m = directBufferClass.getMethod("cleaner");
>        m.setAccessible(true);
>        MethodHandle directBufferCleanerMethod = lookup.unreflect(m);
>        Class<?> cleanerClass = directBufferCleanerMethod.type().returnType();
>        // build method handle for unmapping, full code is here: https://goo.gl/TfQWl6

I guess that is the effect of #AwkwardStrongEncapsulation. I would 
advise doing regular checks against the jigsaw builds to know about such 
problems in the future earlier... but seeing your code break without an 
obvious good solution sure is stressful. I feel with you.

[...]
> (2) A second thing  we noticed is that Groovy no longer works and dies with strange error messages.

That is because versions including Groovy 2.4.7 are using
setAccessible(AccessibleObject[] array, true), and the array will also 
include private methods or fields. This worked till 
#AwkwardStrongEncapsulation because will then a class was either 
exported and its method can all be made accessible or not. For example 
on GAE or earlier versions of the module system. Now an exported class 
may break this, since its private methods can no longer be made 
accessible using setAccessible.

A fix for this is already committed, we are only waiting for release of 
Groovy 2.4.8. Of course even with the fix Groovy code can possibly 
break... for example if you did the direct buffer access in Groovy.

Btw, do not hesitate to ask about such problems on groovy-user, please.

bye Jochen