JDK 9 Build 111 seems to miss some locale data, Lucene tests fail with Farsi and Thai language

Uwe Schindler uschindler at apache.org
Sat Mar 26 11:56:59 UTC 2016


Hi,

after also testing the separate "Jigsaw" build on jdk9.java.net I see the same problems. So both builds 111 are wrong.

To me it looks like the Unicode data files are missing some information - which could again be a packaging bug. As said before, build 110 does not have this problem, so it seems to be a side-effect of Jigsaw merging.

The following stuff does not work:

(1) Thai's locale does not have working dictionary-based BreakIterator available. The following "check" in Lucene for this fails, because it cannot detect a boundary correctly:

  /** 
   * True if the JRE supports a working dictionary-based breakiterator for Thai.
   * If this is false, this tokenizer will not work at all!
   */
  public static final boolean DBBI_AVAILABLE;
  private static final BreakIterator proto = BreakIterator.getWordInstance(new Locale("th"));
  static {
    // check that we have a working dictionary-based break iterator for thai
    proto.setText("ภาษาไทย");
    DBBI_AVAILABLE = proto.isBoundary(4);
  }

After this static initializer, DBBI_AVAILABLE is false. This makes some tests to be ignored, but 2 fail because of this (which might be an oversight on our side). But nevertheless, this is a bug in build 111.

(2) The collator for Arabic (Farsi) language fails to work correctly. This also looks like missing data.

Collator collator = Collator.getInstance(new Locale("ar"));

This one fails some tests that have worked before (in any previous Java 9 build and any previous Java version 1.4+).

I have to revert to build 110, because we cannot make our tests pass on the Jenkins server. I hope this can be fixed with the next build!

Uwe

-----
Uwe Schindler
uschindler at apache.org 
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/


> -----Original Message-----
> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On
> Behalf Of Uwe Schindler
> Sent: Saturday, March 26, 2016 11:49 AM
> To: 'Alan Bateman' <Alan.Bateman at oracle.com>
> Cc: 'Rory O'Donnell' <rory.odonnell at oracle.com>; 'Core-Libs-Dev' <core-libs-
> dev at openjdk.java.net>
> Subject: RE: JDK 9 Build 111 still has ManagementFactory problems, but only
> on Linux - not on Windows
> 
> Hi Alan,
> 
> It is ok from the management factory perspective - sorry for trouble. But the
> first test showed that during language analysis something seems missing,
> which did not happen with the b110 one.
> 
> Currently we see the following tests fail: Everything Thai and Farsi language
> related.
> 
> It looks like the BreakIterators (at least the Thai one) are missing. I have to
> dig into this. I will also compare with the Jigsaw-build 111, if this is different
> (what is the difference between both, I only see size is largely different and
> Windows has a ZIP file?). Those test failures did not happen with neither
> build 110 of Jigsaw and without.
> 
> Here are 2 failures:
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/253/consoleFull
> http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/16339/consoleFull
> 
> Uwe
> 
> -----
> Uwe Schindler
> uschindler at apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
> 
> 
> > -----Original Message-----
> > From: Alan Bateman [mailto:Alan.Bateman at oracle.com]
> > Sent: Saturday, March 26, 2016 11:27 AM
> > To: Uwe Schindler <uschindler at apache.org>
> > Cc: 'Rory O'Donnell' <rory.odonnell at oracle.com>; 'Core-Libs-Dev' <core-
> libs-
> > dev at openjdk.java.net>
> > Subject: Re: JDK 9 Build 111 still has ManagementFactory problems, but
> only
> > on Linux - not on Windows
> >
> >
> >
> > On 26/03/2016 09:59, Uwe Schindler wrote:
> > > Sorry,
> > >
> > > false alarm! I missed to change the build number when changing EA build
> > paths on the Jenkins server. There was still "b110" in it. :(
> > > Sorry for noise, all fine.
> > >
> > >
> > No problem and thanks for confirming that jdk-9+111 is okay.
> >
> > -Alan




More information about the core-libs-dev mailing list