RFC: Make test failed because of the locale LANG

[David Holmes]

Hi Jing,

On 28/03/2019 3:22 pm, Jing Tian wrote:
> Hi,
> 
> When I am doing the 'make test'.If the local LANG is set as 
> 'zh_CN.UTF-8', Test cases will have a lot of error messages.
> 
> ==============================
> Test summary
> ==============================
>     TEST                                              TOTAL  PASS FAIL 
> ERROR
>  >> jtreg:test/hotspot/jtreg:tier1                     1373 1371 2     0 <<
>  >> jtreg:test/jdk:tier1                               1867 1860 7     0 <<
>  >> jtreg:test/langtools:tier1                         3922 2470 
> 1450     2 <<
>     jtreg:test/nashorn:tier1                              0 0     0     0
>     jtreg:test/jaxp:tier1                                 0 0     0     0
>  >> jtreg:test/jdk:tier2                               3334 3305 29     
> 0 <<
>  >> jtreg:test/langtools:tier2 11     9     2     0 <<
>     jtreg:test/nashorn:tier2                             35 35     0     0
>  >> jtreg:test/jaxp:tier2                               438 437 1     0 <<
>  >> jtreg:test/jdk:tier3                               1104 1068 36     
> 0 <<
>     jtreg:test/langtools:tier3                            0 0     0     0
>     jtreg:test/nashorn:tier3                              0 0     0     0
>     jtreg:test/jaxp:tier3                                 0 0     0     0
> ==============================

Given most of these are not hotspot issues and given the number of 
failures, I think this is something that would need to be discussed much 
more broadly. So I've cc'd core-libs-dev and compiler-dev. I recall 
there was a very recent discussion regarding some tests failing for the 
same reason, but don't recall the outcome.

David
-----

> On the same machine,when i set LANG=en_US.UTF-8.
> 
> ==============================
> Test summary
> ==============================
>     TEST                                              TOTAL  PASS FAIL 
> ERROR
>  >> jtreg:test/hotspot/jtreg:tier1                     1388 1386 2     0 <<
>  >> jtreg:test/jdk:tier1                               1867 1843 19     
> 5 <<
>     jtreg:test/langtools:tier1                         3920 3920 0     0
>     jtreg:test/nashorn:tier1                              0 0     0     0
>     jtreg:test/jaxp:tier1                                 0 0     0     0
>  >> jtreg:test/jdk:tier2                               3328 3290 31     
> 7 <<
>     jtreg:test/langtools:tier2                           11 11     0     0
>     jtreg:test/nashorn:tier2                             35 35     0     0
>     jtreg:test/jaxp:tier2                               438 438     0     0
>  >> jtreg:test/jdk:tier3                               1104 1067 37     
> 0 <<
>     jtreg:test/langtools:tier3                            0 0     0     0
>     jtreg:test/nashorn:tier3                              0 0     0     0
>     jtreg:test/jaxp:tier3                                 0 0     0     0
> 
> 
> By comparison we can find, lots of(1000+) langtools tests will get fail, 
> and other(about 30+, exclude some X11 and sanity that
> 
> result problem) test cases will also fail because of local LANG.
> 
> 
> such as in the test/hotspot/jtreg/compiler/c2/Test8062950.java,
> 
> shouldContain("Error: Could not find or load main class " + CLASSNAME)
> 
> 
> When in the zh_CN.UTF-8 environment, because some of the output 
> information is changed to Chinese by some properties file,
> 
> the English cannot be matched, which will result in an fail.
> 
> When I set  LANG=en_US/LC_ALL=C, this test will pass.
> 
> 
> I think there are some possible solutions.
> 
> 
> 1.we set LC_ALL=C/LANG=EN_us in the Runtests.gmk, but this modification 
> is more violent because he will affect all test cases.
> 
> 
> 2.We modify each individual test，E.g 
> test/hotspot/jtreg/compiler/c2/Test8062950.java
> 
> diff -r 0421d49b6217 test/hotspot/jtreg/compiler/c2/Test8062950.java
>   package compiler.c2;
>   import jdk.test.lib.process.ProcessTools;
> +import java.util.Map;
> +import jdk.test.lib.process.OutputAnalyzer;
> 
>   public class Test8062950 {
>       private static final String CLASSNAME = "DoesNotExist";
>       public static void main(String[] args) throws Exception {
> - ProcessTools.executeTestJvm("-Xcomp",
> - "-XX:-TieredCompilation",
> - "-XX:-UseOptoBiasInlining",
> - CLASSNAME)
> - .shouldHaveExitValue(1)
> -                    .shouldContain("Error: Could not find or load main 
> class " + CLASSNAME)
> -                    .shouldNotContain("A fatal error has been detected")
> -                    .shouldNotContain("Internal Error")
> -                    .shouldNotContain("HotSpot Virtual Machine Error");
> +        final ProcessBuilder pb = 
> ProcessTools.createJavaProcessBuilder(true,
> + "-Xcomp",
> + "-XX:-TieredCompilation",
> + "-XX:-UseOptoBiasInlining",
> + CLASSNAME);
> +        final Map<String, String> env = pb.environment();
> +        env.put("LC_ALL", "en_US.UTF-8");
> +        OutputAnalyzer output = new OutputAnalyzer(pb.start());
> + output.shouldHaveExitValue(1);
> +        output.shouldContain("Error: Could not find or load main class 
> " + CLASSNAME);
> +        output.shouldNotContain("A fatal error has been detected");
> +        output.shouldNotContain("Internal Error");
> +        output.shouldNotContain("HotSpot Virtual Machine Error");
> }
>   }
> 
> But I don't know if this will change the test program too much, because 
> the related problems are a lot in langtools tests.
> 
> 
> 3.And i find that there is a function can judge the locale
> 
>   if (!isEnglishLocale()) { // only english version
> return;
>   }
> 
> But in this case, I think in many test cases, we may not be able to 
> return so directly, because some test cases may have other test purposes.
> 
> 
> So I don't know if you have any ideas or some suggestions to solve the 
> problem that the output information and English do not match in this
> 
> non-English environment.
> 
> 
> Cheers,
> 
> Jing Tian
> 
> 
> 
>