RFC: Make test failed because of the locale LANG
Jing Tian
jingtian at loongson.cn
Fri Mar 29 02:20:29 UTC 2019
Hi,
First of all, I want to say sorry . Because of the problems with my mail
client, this typography looks bad.I found the typography so bad on the
webpage today. T_T
These suggestions I just want to show some ideas. We can't make changes
in each test case, because it is too much, and the changes I made are
not good.
The following is the fail test cases because of the locale(exclude some
X11 fail and sanity fail, because some of fails are in the list because
of them):
java/lang/System/LoggerFinder/internal/BootstrapLogger/BootstrapLoggerTest.java
java/util/logging/LocalizedLevelName.java
java/util/logging/SimpleFormatterFormat.java
sun/util/logging/SourceClassName.java
tools/pack200/DeprecatePack200.java
java/security/KeyStore/PKCS12/KeytoolReaderP12Test.java
java/time/tck/java/time/format/TCKDateTimeFormatterBuilder.java
javax/security/auth/login/JAASConfigSyntaxCheck/JAASConfigSyntaxTest.java
javax/security/auth/login/LoginContext/DynamicConfigurationTest.java
jdk/security/logging/TestSecurityPropertyModificationLog.java
jdk/security/logging/TestTLSHandshakeLog.java
jdk/security/logging/TestX509CertificateLog.java
jdk/security/logging/TestX509ValidationLog.java
tools/jar/DeprecateOptionN.java
tools/jar/compat/CLICompatibility.java
tools/jar/mmrjar/Basic.java
tools/jar/modularJar/Basic.java
tools/jar/multiRelease/ApiValidatorTest.java
tools/jar/multiRelease/Basic.java
tools/jar/multiRelease/Basic1.java
tools/jlink/JLinkNegativeTest.java
tools/jlink/JLinkOptionsTest.java
tools/jlink/JLinkPluginsTest.java
tools/jlink/JLinkTest.java
tools/jlink/bindservices/SuggestProviders.java
tools/jlink/plugins/LegalFilePluginTest.java
tools/jmod/JmodNegativeTest.java
tools/jmod/JmodTest.java
tools/launcher/FXLauncherTest.java
tools/launcher/InfoStreams.java
tools/launcher/SourceMode.java
tools/launcher/TestSpecialArgs.java
javax/xml/jaxp/functional/org/xml/sax/ptests/EHFatalTest.java
test/hotspot/jtreg/compiler/c2/Test8062950.java
test/hotspot/jtreg/runtime/classFileParserBug/TestEmptyBootstrapMethodsAttr.java
The following is the langtools fail test cases.(1450/3922)
jdk/javadoc/doclet/AccessAsciiArt/AccessAsciiArt.java
jdk/javadoc/doclet/AccessFrameTitle/AccessFrameTitle.java
jdk/javadoc/doclet/AccessSkipNav/AccessSkipNav.java
jdk/javadoc/doclet/AuthorDD/AuthorDD.java
jdk/javadoc/doclet/MetaTag/MetaTag.java
jdk/javadoc/doclet/WindowTitles/WindowTitles.java
jdk/javadoc/doclet/testAbstractMethod/TestAbstractMethod.java
jdk/javadoc/doclet/testAnchorNames/TestAnchorNames.java
jdk/javadoc/doclet/testAnnotationTypes/TestAnnotationTypes.java
jdk/javadoc/doclet/testAuthor/TestAuthor.java
jdk/javadoc/doclet/testBadPackageFileInJar/TestBadPackageFileInJar.java
jdk/javadoc/doclet/testCharsetDocencodingOptions/TestCharsetDocencodingOptions.java
jdk/javadoc/doclet/testClassCrossReferences/TestClassCrossReferences.java
jdk/javadoc/doclet/testClassDocCatalog/TestClassDocCatalog.java
..........
I don't know how to solve this problem is a better solution.
在 2019/3/29 9:23, naoto.sato at oracle.com 写道:
> On 3/28/19 4:54 PM, David Holmes wrote:
>> On 29/03/2019 1:59 am, Naoto Sato wrote:
>>> Hi,
>>>
>>> I don't think there is any *official* rule for the regression tests
>>> to succeed in any locale, but if the test case is locale sensitive
>>> such as in this case, I'd suggest it should correctly specify the
>>> locale beforehand, or quit gracefully in general.
>>>
>>> For this specific case, I'd suggest not set the environment variable
>>> (as it won't work with Windows), using runtime system properties
>>> (user.language/user.country) would help.
>>
>> Which specific case are you referring to? :)
>
> I was referring to the test case below, Test8062950.java, where it
> issues "env.put("LC_ALL", "en_US.UTF-8")". Instead I would recommend
> "-Duser.language=en -Duser.country=US" runtime system property.
>
>>
>> Given there are well over a thousand failures in total I think we
>> need to set a reasonable initial locale as part of the test setup.
>> Many of the hotspot tests check actual and expected output and we, in
>> general, have no idea what kinds of output may be subject to locale
>> specific > changes.
>
> Agree. Specifying a reasonable default locale would not be a bad idea.
>
> Naoto
>
>>
>> Cheers,
>> David
>>
>>> Naoto
>>>
>>> On 3/27/19 10:47 PM, David Holmes wrote:
>>>> Hi Jing,
>>>>
>>>> On 28/03/2019 3:22 pm, Jing Tian wrote:
>>>>> Hi,
>>>>>
>>>>> When I am doing the 'make test'.If the local LANG is set as
>>>>> 'zh_CN.UTF-8', Test cases will have a lot of error messages.
>>>>>
>>>>> ==============================
>>>>> Test summary
>>>>> ==============================
>>>>> TEST TOTAL PASS FAIL ERROR
>>>>> >> jtreg:test/hotspot/jtreg:tier1 1373 1371 2
>>>>> 0 <<
>>>>> >> jtreg:test/jdk:tier1 1867 1860 7
>>>>> 0 <<
>>>>> >> jtreg:test/langtools:tier1 3922 2470
>>>>> 1450 2 <<
>>>>> jtreg:test/nashorn:tier1 0 0 0 0
>>>>> jtreg:test/jaxp:tier1 0 0 0 0
>>>>> >> jtreg:test/jdk:tier2 3334 3305
>>>>> 29 0 <<
>>>>> >> jtreg:test/langtools:tier2 11 9 2 0 <<
>>>>> jtreg:test/nashorn:tier2 35 35 0 0
>>>>> >> jtreg:test/jaxp:tier2 438 437 1
>>>>> 0 <<
>>>>> >> jtreg:test/jdk:tier3 1104 1068
>>>>> 36 0 <<
>>>>> jtreg:test/langtools:tier3 0 0 0 0
>>>>> jtreg:test/nashorn:tier3 0 0 0 0
>>>>> jtreg:test/jaxp:tier3 0 0 0 0
>>>>> ==============================
>>>>
>>>> Given most of these are not hotspot issues and given the number of
>>>> failures, I think this is something that would need to be discussed
>>>> much more broadly. So I've cc'd core-libs-dev and compiler-dev. I
>>>> recall there was a very recent discussion regarding some tests
>>>> failing for the same reason, but don't recall the outcome.
>>>>
>>>> David
>>>> -----
>>>>
>>>>> On the same machine,when i set LANG=en_US.UTF-8.
>>>>>
>>>>> ==============================
>>>>> Test summary
>>>>> ==============================
>>>>> TEST TOTAL PASS FAIL ERROR
>>>>> >> jtreg:test/hotspot/jtreg:tier1 1388 1386 2
>>>>> 0 <<
>>>>> >> jtreg:test/jdk:tier1 1867 1843
>>>>> 19 5 <<
>>>>> jtreg:test/langtools:tier1 3920 3920 0 0
>>>>> jtreg:test/nashorn:tier1 0 0 0 0
>>>>> jtreg:test/jaxp:tier1 0 0 0 0
>>>>> >> jtreg:test/jdk:tier2 3328 3290
>>>>> 31 7 <<
>>>>> jtreg:test/langtools:tier2 11 11 0 0
>>>>> jtreg:test/nashorn:tier2 35 35 0 0
>>>>> jtreg:test/jaxp:tier2 438 438 0 0
>>>>> >> jtreg:test/jdk:tier3 1104 1067
>>>>> 37 0 <<
>>>>> jtreg:test/langtools:tier3 0 0 0 0
>>>>> jtreg:test/nashorn:tier3 0 0 0 0
>>>>> jtreg:test/jaxp:tier3 0 0 0 0
>>>>>
>>>>>
>>>>> By comparison we can find, lots of(1000+) langtools tests will get
>>>>> fail, and other(about 30+, exclude some X11 and sanity that
>>>>>
>>>>> result problem) test cases will also fail because of local LANG.
>>>>>
>>>>>
>>>>> such as in the test/hotspot/jtreg/compiler/c2/Test8062950.java,
>>>>>
>>>>> shouldContain("Error: Could not find or load main class " +
>>>>> CLASSNAME)
>>>>>
>>>>>
>>>>> When in the zh_CN.UTF-8 environment, because some of the output
>>>>> information is changed to Chinese by some properties file,
>>>>>
>>>>> the English cannot be matched, which will result in an fail.
>>>>>
>>>>> When I set LANG=en_US/LC_ALL=C, this test will pass.
>>>>>
>>>>>
>>>>> I think there are some possible solutions.
>>>>>
>>>>>
>>>>> 1.we set LC_ALL=C/LANG=EN_us in the Runtests.gmk, but this
>>>>> modification is more violent because he will affect all test cases.
>>>>>
>>>>>
>>>>> 2.We modify each individual test，E.g
>>>>> test/hotspot/jtreg/compiler/c2/Test8062950.java
>>>>>
>>>>> diff -r 0421d49b6217 test/hotspot/jtreg/compiler/c2/Test8062950.java
>>>>> package compiler.c2;
>>>>> import jdk.test.lib.process.ProcessTools;
>>>>> +import java.util.Map;
>>>>> +import jdk.test.lib.process.OutputAnalyzer;
>>>>>
>>>>> public class Test8062950 {
>>>>> private static final String CLASSNAME = "DoesNotExist";
>>>>> public static void main(String[] args) throws Exception {
>>>>> - ProcessTools.executeTestJvm("-Xcomp",
>>>>> - "-XX:-TieredCompilation",
>>>>> - "-XX:-UseOptoBiasInlining",
>>>>> - CLASSNAME)
>>>>> - .shouldHaveExitValue(1)
>>>>> - .shouldContain("Error: Could not find or load
>>>>> main class " + CLASSNAME)
>>>>> - .shouldNotContain("A fatal error has been
>>>>> detected")
>>>>> - .shouldNotContain("Internal Error")
>>>>> - .shouldNotContain("HotSpot Virtual Machine
>>>>> Error");
>>>>> + final ProcessBuilder pb =
>>>>> ProcessTools.createJavaProcessBuilder(true,
>>>>> + "-Xcomp",
>>>>> + "-XX:-TieredCompilation",
>>>>> + "-XX:-UseOptoBiasInlining",
>>>>> + CLASSNAME);
>>>>> + final Map<String, String> env = pb.environment();
>>>>> + env.put("LC_ALL", "en_US.UTF-8");
>>>>> + OutputAnalyzer output = new OutputAnalyzer(pb.start());
>>>>> + output.shouldHaveExitValue(1);
>>>>> + output.shouldContain("Error: Could not find or load main
>>>>> class " + CLASSNAME);
>>>>> + output.shouldNotContain("A fatal error has been detected");
>>>>> + output.shouldNotContain("Internal Error");
>>>>> + output.shouldNotContain("HotSpot Virtual Machine Error");
>>>>> }
>>>>> }
>>>>>
>>>>> But I don't know if this will change the test program too much,
>>>>> because the related problems are a lot in langtools tests.
>>>>>
>>>>>
>>>>> 3.And i find that there is a function can judge the locale
>>>>>
>>>>> if (!isEnglishLocale()) { // only english version
>>>>> return;
>>>>> }
>>>>>
>>>>> But in this case, I think in many test cases, we may not be able
>>>>> to return so directly, because some test cases may have other test
>>>>> purposes.
>>>>>
>>>>>
>>>>> So I don't know if you have any ideas or some suggestions to solve
>>>>> the problem that the output information and English do not match
>>>>> in this
>>>>>
>>>>> non-English environment.
>>>>>
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Jing Tian
>>>>>
>>>>>
>>>>>
>>>>>
More information about the compiler-dev
mailing list