Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch

[Uwe Schindler]

Hi Chris,

looks good to me. I already created a patch / branch of Apache Lucene that uses your proposed API and removes the Runnable hack. You can see it and check it out on Github:

<https://github.com/apache/lucene-solr/compare/LUCENE-6989-v2>

For your info the issue description and description is:
<https://issues.apache.org/jira/browse/LUCENE-6989?focusedCommentId=15748848&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15748848>

If you would like to test it against your patch, check out this branch, run "ant ivy-bootstrap" and then cd into the lucene/core dirto execute:
$ ant test -Dtestcase=TestMmapDirectory

Those tests should run without any error message (disabled test because unmapping not supported):

[junit4] Started J0 PID(3464 at localhost).
   [junit4] Suite: org.apache.lucene.store.TestMmapDirectory
   [junit4] OK      0.07s | TestMmapDirectory.testIllegalEOF
   [junit4] OK      0.01s | TestMmapDirectory.testShort
   [junit4] OK      0.06s | TestMmapDirectory.testRandomInt
   [junit4] OK      0.02s | TestMmapDirectory.testFsyncDoesntCreateNewFiles
   [junit4] OK      0.01s | TestMmapDirectory.testSetOfStrings
   [junit4] OK      0.01s | TestMmapDirectory.testLargeWrites
   [junit4] OK      0.01s | TestMmapDirectory.testMapOfStrings
   [junit4] OK      0.01s | TestMmapDirectory.testSliceOfSlice
   [junit4] OK      0.01s | TestMmapDirectory.testByte
   [junit4] OK      0.01s | TestMmapDirectory.testVInt
   [junit4] OK      0.01s | TestMmapDirectory.testChecksum
   [junit4] OK      0.02s | TestMmapDirectory.testZLong
   [junit4] OK      0.06s | TestMmapDirectory.testRandomShort
   [junit4] OK      0.01s | TestMmapDirectory.testRename
   [junit4] OK      0.10s | TestMmapDirectory.testCreateTempOutput
   [junit4] OK      0.01s | TestMmapDirectory.testZInt
   [junit4] OK      0.01s | TestMmapDirectory.testIndexOutputToString
   [junit4] OK      0.01s | TestMmapDirectory.testDetectClose
   [junit4] OK      0.03s | TestMmapDirectory.testListAllIsSorted
   [junit4] OK      0.01s | TestMmapDirectory.testDoubleCloseDirectory
   [junit4] OK      0.00s | TestMmapDirectory.testDirectoryFilter
   [junit4] OK      0.11s | TestMmapDirectory.testPendingDeletions
   [junit4] OK      0.04s | TestMmapDirectory.testCopyFromDestination
   [junit4] OK      0.01s | TestMmapDirectory.testDeleteFile
   [junit4] OK      0.04s | TestMmapDirectory.testCopyBytesWithThreads
   [junit4] OK      0.03s | TestMmapDirectory.testRandomByte
   [junit4] IGNORED 0.00s | TestMmapDirectory.testAceWithThreads
   [junit4]    > Cause: Annotated @Ignore(This test is for JVM testing purposes. There are no guarantees that it may not fail with SIGSEGV!)
   [junit4] OK      0.02s | TestMmapDirectory.testNoDir
   [junit4] OK      0.01s | TestMmapDirectory.testSeekBeyondEndOfFile
   [junit4] OK      0.13s | TestMmapDirectory.testRandomLong
   [junit4] OK      0.01s | TestMmapDirectory.testDoubleCloseInput
   [junit4] OK      0.01s | TestMmapDirectory.testSliceOutOfBounds
   [junit4] OK      2.75s | TestMmapDirectory.testThreadSafety
   [junit4] OK      0.01s | TestMmapDirectory.testSeekToEndOfFile
   [junit4] OK      0.00s | TestMmapDirectory.testLong
   [junit4] OK      0.01s | TestMmapDirectory.testCopyFrom
   [junit4] OK      0.00s | TestMmapDirectory.testString
   [junit4] OK      0.01s | TestMmapDirectory.testSeekToEOFThenBack
   [junit4] OK      0.00s | TestMmapDirectory.testInt
   [junit4] OK      0.01s | TestMmapDirectory.testSeekPastEOF
   [junit4] OK      0.02s | TestMmapDirectory.testCopyBytes
   [junit4] OK      0.01s | TestMmapDirectory.testVLong
   [junit4] OK      0.01s | TestMmapDirectory.testDoubleCloseOutput
   [junit4] Completed [1/1] in 5.14s, 43 tests, 1 skipped

With Java 9 build 148 it currently looks like this:

   [junit4] Suite: org.apache.lucene.store.TestMmapDirectory
   [junit4] IGNOR/A 0.04s | TestMmapDirectory.testShort
   [junit4]    > Assumption #1: test requires a jre that supports unmapping: Unmapping is not supported on this platform, because internal Java APIs are not compatible to this Lucene version: java.lang.reflect.InaccessibleObjectException: Unable to make public jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: module java.base does not "opens java.nio" to unnamed module @45c7ad7f

Don't care about the Groovy error printed later, this one is known, but does not affect testing.

Uwe

-----
Uwe Schindler
uschindler at apache.org 
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/

> -----Original Message-----
> From: Chris Hegarty [mailto:chris.hegarty at oracle.com]
> Sent: Wednesday, December 14, 2016 12:58 PM
> To: Peter Levart <peter.levart at gmail.com>; Core-Libs-Dev <core-libs-
> dev at openjdk.java.net>; jigsaw-dev <jigsaw-dev at openjdk.java.net>; Uwe
> Schindler <uschindler at apache.org>
> Subject: Re: Java 9 build 148 causes trouble in Apache
> Lucene/Solr/Elasticsearch
> 
> Webrev updated in-place.
>    http://cr.openjdk.java.net/~chegar/Unsafe_invokeCleaner/
> 
> -Chris.
> 
> On 13/12/16 21:18, Peter Levart wrote:
> > I think this is OK.
> >
> > Just a couple of nits in test:
> >
> > 1. You create a static Path bob = Paths.get("bob") field, but then you
> > don't use it in:
> >
> >   56         try (FileChannel fc = FileChannel.open(Paths.get("bob"),
> > CREATE, WRITE)) {
> >
> > 2. badBuffers could include a duplicate and a slice of a direct buffer
> > allocated with ByteBuffer.allocateDirect()
> >
> > 3. The comment in the test is referencing the old method name:
> >
> >   26  * @summary Basic test for Unsafe::deallocate
> >
> >
> > Regards, Peter
> >
> > On 12/13/2016 08:47 PM, Chris Hegarty wrote:
> >> Taking into account the feedback so far, and changing the method name (
> since
> >> it is an attractive nuisance ), here is where I think we ended up.
> >>
> >> http://cr.openjdk.java.net/~chegar/Unsafe_invokeCleaner/
> >>
> >> If this is agreeable, I’ll file an issue in JIRA to track the code changes, and
> >> update JEP 260.
> >>
> >> -Chris.
> >