Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch

[Jochen Theodorou]

Hi,

I strongly hope Paul and Cedric will be able to start the release 
process next week, if not we will have to do it the old way I think.

what would help us a lot would be you testing the GROOVY_2_4_X branch 
with your build system to see if it really does solve your problem. Even 
if it is only locally on your computer

bye Jochen

On 16.12.2016 10:58, Uwe Schindler wrote:
> Hi Jochen,
>
> thank you for the information! Is there any plan about a release? I also found no JIRA issue about this issue to link it against our JIRA: https://issues.apache.org/jira/browse/LUCENE-7596
>
> The problem makes our build system unusable, so it would be very important to have a fix quite soon! As our Ant/Ivy-based build relies on Maven Central, it would be good to have the bugfix release available there, which requires a release. I think the same applies for Gradle users (Elasticsearch).
>
> As a temporary workaround we might be able to use the Apache Snapshot repository, but this is not allowed if we do a release of Lucene.
>
> Uwe
>
> -----
> Uwe Schindler
> uschindler at apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
>> -----Original Message-----
>> From: Jochen Theodorou [mailto:blackdrag at gmx.org]
>> Sent: Saturday, December 10, 2016 9:23 AM
>> To: Uwe Schindler <uschindler at apache.org>; jigsaw-dev at openjdk.java.net;
>> Core-Libs-Dev <core-libs-dev at openjdk.java.net>
>> Subject: Re: Java 9 build 148 causes trouble in Apache
>> Lucene/Solr/Elasticsearch
>>
>> On 09.12.2016 23:32, Uwe Schindler wrote:
>>> Hi,
>>>
>>> I updated our Jenkins server for the JDK 9 preview testing to use build 148.
>> Previously we had build 140 and build 147, which both worked without any
>> issues. But after the update the following stuff goes wrong:
>>>
>>> (1) Unmapping of direct buffers no longer works, although this API was
>> marked as critical because there is no replacement up to now, so code can
>> unmap memory mapped files, which is one of the most important things
>> Apache Lucene needs to use to access huge random access files while
>> reading the index. Without memory mapping, the slowdown for Lucene
>> users will be huge
>>>
>>> This is caused by the recent Jigsaw changes, published in build 148.
>> Unfortunately we did not test the Jigsaw builds, so we would have noticed
>> that earlier. Basically the following peace of code fails now (with or without
>> doPrivileged and with/without security manager):
>>>
>>>        final Class<?> directBufferClass =
>> Class.forName("java.nio.DirectByteBuffer");
>>>
>>>        final Method m = directBufferClass.getMethod("cleaner");
>>>        m.setAccessible(true);
>>>        MethodHandle directBufferCleanerMethod = lookup.unreflect(m);
>>>        Class<?> cleanerClass =
>> directBufferCleanerMethod.type().returnType();
>>>        // build method handle for unmapping, full code is here:
>> https://goo.gl/TfQWl6
>>
>> I guess that is the effect of #AwkwardStrongEncapsulation. I would
>> advise doing regular checks against the jigsaw builds to know about such
>> problems in the future earlier... but seeing your code break without an
>> obvious good solution sure is stressful. I feel with you.
>>
>> [...]
>>> (2) A second thing  we noticed is that Groovy no longer works and dies with
>> strange error messages.
>>
>> That is because versions including Groovy 2.4.7 are using
>> setAccessible(AccessibleObject[] array, true), and the array will also
>> include private methods or fields. This worked till
>> #AwkwardStrongEncapsulation because will then a class was either
>> exported and its method can all be made accessible or not. For example
>> on GAE or earlier versions of the module system. Now an exported class
>> may break this, since its private methods can no longer be made
>> accessible using setAccessible.
>>
>> A fix for this is already committed, we are only waiting for release of
>> Groovy 2.4.8. Of course even with the fix Groovy code can possibly
>> break... for example if you did the direct buffer access in Groovy.
>>
>> Btw, do not hesitate to ask about such problems on groovy-user, please.
>>
>> bye Jochen
>