RFC: Make test failed because of the locale LANG
naoto.sato at oracle.com
naoto.sato at oracle.com
Fri Mar 29 01:23:29 UTC 2019
On 3/28/19 4:54 PM, David Holmes wrote:
> On 29/03/2019 1:59 am, Naoto Sato wrote:
>> Hi,
>>
>> I don't think there is any *official* rule for the regression tests to
>> succeed in any locale, but if the test case is locale sensitive such
>> as in this case, I'd suggest it should correctly specify the locale
>> beforehand, or quit gracefully in general.
>>
>> For this specific case, I'd suggest not set the environment variable
>> (as it won't work with Windows), using runtime system properties
>> (user.language/user.country) would help.
>
> Which specific case are you referring to? :)
I was referring to the test case below, Test8062950.java, where it
issues "env.put("LC_ALL", "en_US.UTF-8")". Instead I would recommend
"-Duser.language=en -Duser.country=US" runtime system property.
>
> Given there are well over a thousand failures in total I think we need
> to set a reasonable initial locale as part of the test setup. Many of
> the hotspot tests check actual and expected output and we, in general,
> have no idea what kinds of output may be subject to locale specific > changes.
Agree. Specifying a reasonable default locale would not be a bad idea.
Naoto
>
> Cheers,
> David
>
>> Naoto
>>
>> On 3/27/19 10:47 PM, David Holmes wrote:
>>> Hi Jing,
>>>
>>> On 28/03/2019 3:22 pm, Jing Tian wrote:
>>>> Hi,
>>>>
>>>> When I am doing the 'make test'.If the local LANG is set as
>>>> 'zh_CN.UTF-8', Test cases will have a lot of error messages.
>>>>
>>>> ==============================
>>>> Test summary
>>>> ==============================
>>>> TEST TOTAL PASS
>>>> FAIL ERROR
>>>> >> jtreg:test/hotspot/jtreg:tier1 1373 1371 2 0 <<
>>>> >> jtreg:test/jdk:tier1 1867 1860 7 0 <<
>>>> >> jtreg:test/langtools:tier1 3922 2470
>>>> 1450 2 <<
>>>> jtreg:test/nashorn:tier1 0 0 0 0
>>>> jtreg:test/jaxp:tier1 0 0 0 0
>>>> >> jtreg:test/jdk:tier2 3334 3305 29
>>>> 0 <<
>>>> >> jtreg:test/langtools:tier2 11 9 2 0 <<
>>>> jtreg:test/nashorn:tier2 35 35 0 0
>>>> >> jtreg:test/jaxp:tier2 438 437 1 0 <<
>>>> >> jtreg:test/jdk:tier3 1104 1068 36
>>>> 0 <<
>>>> jtreg:test/langtools:tier3 0 0 0 0
>>>> jtreg:test/nashorn:tier3 0 0 0 0
>>>> jtreg:test/jaxp:tier3 0 0 0 0
>>>> ==============================
>>>
>>> Given most of these are not hotspot issues and given the number of
>>> failures, I think this is something that would need to be discussed
>>> much more broadly. So I've cc'd core-libs-dev and compiler-dev. I
>>> recall there was a very recent discussion regarding some tests
>>> failing for the same reason, but don't recall the outcome.
>>>
>>> David
>>> -----
>>>
>>>> On the same machine,when i set LANG=en_US.UTF-8.
>>>>
>>>> ==============================
>>>> Test summary
>>>> ==============================
>>>> TEST TOTAL PASS
>>>> FAIL ERROR
>>>> >> jtreg:test/hotspot/jtreg:tier1 1388 1386 2 0 <<
>>>> >> jtreg:test/jdk:tier1 1867 1843 19
>>>> 5 <<
>>>> jtreg:test/langtools:tier1 3920 3920
>>>> 0 0
>>>> jtreg:test/nashorn:tier1 0 0 0 0
>>>> jtreg:test/jaxp:tier1 0 0 0 0
>>>> >> jtreg:test/jdk:tier2 3328 3290 31
>>>> 7 <<
>>>> jtreg:test/langtools:tier2 11 11 0 0
>>>> jtreg:test/nashorn:tier2 35 35 0 0
>>>> jtreg:test/jaxp:tier2 438 438 0 0
>>>> >> jtreg:test/jdk:tier3 1104 1067 37
>>>> 0 <<
>>>> jtreg:test/langtools:tier3 0 0 0 0
>>>> jtreg:test/nashorn:tier3 0 0 0 0
>>>> jtreg:test/jaxp:tier3 0 0 0 0
>>>>
>>>>
>>>> By comparison we can find, lots of(1000+) langtools tests will get
>>>> fail, and other(about 30+, exclude some X11 and sanity that
>>>>
>>>> result problem) test cases will also fail because of local LANG.
>>>>
>>>>
>>>> such as in the test/hotspot/jtreg/compiler/c2/Test8062950.java,
>>>>
>>>> shouldContain("Error: Could not find or load main class " + CLASSNAME)
>>>>
>>>>
>>>> When in the zh_CN.UTF-8 environment, because some of the output
>>>> information is changed to Chinese by some properties file,
>>>>
>>>> the English cannot be matched, which will result in an fail.
>>>>
>>>> When I set LANG=en_US/LC_ALL=C, this test will pass.
>>>>
>>>>
>>>> I think there are some possible solutions.
>>>>
>>>>
>>>> 1.we set LC_ALL=C/LANG=EN_us in the Runtests.gmk, but this
>>>> modification is more violent because he will affect all test cases.
>>>>
>>>>
>>>> 2.We modify each individual test，E.g
>>>> test/hotspot/jtreg/compiler/c2/Test8062950.java
>>>>
>>>> diff -r 0421d49b6217 test/hotspot/jtreg/compiler/c2/Test8062950.java
>>>> package compiler.c2;
>>>> import jdk.test.lib.process.ProcessTools;
>>>> +import java.util.Map;
>>>> +import jdk.test.lib.process.OutputAnalyzer;
>>>>
>>>> public class Test8062950 {
>>>> private static final String CLASSNAME = "DoesNotExist";
>>>> public static void main(String[] args) throws Exception {
>>>> - ProcessTools.executeTestJvm("-Xcomp",
>>>> - "-XX:-TieredCompilation",
>>>> - "-XX:-UseOptoBiasInlining",
>>>> - CLASSNAME)
>>>> - .shouldHaveExitValue(1)
>>>> - .shouldContain("Error: Could not find or load
>>>> main class " + CLASSNAME)
>>>> - .shouldNotContain("A fatal error has been
>>>> detected")
>>>> - .shouldNotContain("Internal Error")
>>>> - .shouldNotContain("HotSpot Virtual Machine
>>>> Error");
>>>> + final ProcessBuilder pb =
>>>> ProcessTools.createJavaProcessBuilder(true,
>>>> + "-Xcomp",
>>>> + "-XX:-TieredCompilation",
>>>> + "-XX:-UseOptoBiasInlining",
>>>> + CLASSNAME);
>>>> + final Map<String, String> env = pb.environment();
>>>> + env.put("LC_ALL", "en_US.UTF-8");
>>>> + OutputAnalyzer output = new OutputAnalyzer(pb.start());
>>>> + output.shouldHaveExitValue(1);
>>>> + output.shouldContain("Error: Could not find or load main
>>>> class " + CLASSNAME);
>>>> + output.shouldNotContain("A fatal error has been detected");
>>>> + output.shouldNotContain("Internal Error");
>>>> + output.shouldNotContain("HotSpot Virtual Machine Error");
>>>> }
>>>> }
>>>>
>>>> But I don't know if this will change the test program too much,
>>>> because the related problems are a lot in langtools tests.
>>>>
>>>>
>>>> 3.And i find that there is a function can judge the locale
>>>>
>>>> if (!isEnglishLocale()) { // only english version
>>>> return;
>>>> }
>>>>
>>>> But in this case, I think in many test cases, we may not be able to
>>>> return so directly, because some test cases may have other test
>>>> purposes.
>>>>
>>>>
>>>> So I don't know if you have any ideas or some suggestions to solve
>>>> the problem that the output information and English do not match in
>>>> this
>>>>
>>>> non-English environment.
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Jing Tian
>>>>
>>>>
>>>>
>>>>
More information about the core-libs-dev
mailing list