JDK 9 Build 111 seems to miss some locale data, Lucene tests fail with Farsi and Thai language

[Uwe Schindler]

Thanks Alan.

 

I can confirm that Jenkins now runs Lucene/Solr tests fine. We had many runs last night, all succeeded – great!

 

About the new issue: Thanks for opening.

 

One addition: This not only affects BreakIterators, also the Arabian Collator as found in the other failing test! It reproduces in same way:

 

Collator collator = Collator.getInstance(new Locale(“ar”));

 

the returned one is the one from root locale (or something similar), not the Arabian one. We found the bug by “testing” the sort order which failed – because its different than default ROOT sorting. But you can also look at class names like I did for the BreakIterators.

 

I wish you happy Easter!

 

Uwe

 

-----

Uwe Schindler

uschindler at apache.org 

ASF Member, Apache Lucene PMC / Committer

Bremen, Germany

http://lucene.apache.org/

 

From: Alan Bateman [mailto:Alan.Bateman at oracle.com] 
Sent: Sunday, March 27, 2016 11:50 AM
To: Uwe Schindler <uschindler at apache.org>
Cc: 'Rory O'Donnell' <rory.odonnell at oracle.com>; 'Core-Libs-Dev' <core-libs-dev at openjdk.java.net>; 'Robert Muir' <rcmuir at gmail.com>; dev at lucene.apache.org
Subject: Re: JDK 9 Build 111 seems to miss some locale data, Lucene tests fail with Farsi and Thai language

 

On 26/03/2016 23:16, Uwe Schindler wrote:



: 

 

Now all analysis tests pass, also the small test program posted previously:

 

$ java -Djava.security.manager Test

class sun.util.locale.provider.DictionaryBasedBreakIterator

 

FYI: Lucene runs all tests with a security manager to enforce some restrictions (so tests can't escape their working dir, no useless permissions are required that may conflict,...). Lucene is designed to work with lowest permissions (except the memory mapping unmapper).

 

I will patch the Jenkins Server's JDK-9 b111 dirs the same way, so we can run tests.

 

I have the following question: Why don’t we see an exception when loading the locale data? Shouln’t Java fail in some way and print a stack trace? It is just silent!

 

Thanks for the confirming the workaround. I've created JDK-8152817 to track this issue and I hope we can get this fixed in jdk9/dev quickly.

The reason you aren't seeing the exception is that an internal class sun.util.Resources.loadBundleFromProviders is catching (and recording) the security exception. The cause may surface as the cause on a MissingResourceException but not in this case. This area has a lot of history, I'm sure Naoto can say more about this.

-Alan