RFC: Make test failed because of the locale LANG

[Jing Tian]

Hi,

When I am doing the 'make test'.If the local LANG is set as 
'zh_CN.UTF-8', Test cases will have a lot of error messages.

==============================
Test summary
==============================
    TEST                                              TOTAL  PASS FAIL 
ERROR
 >> jtreg:test/hotspot/jtreg:tier1                     1373 1371     
2     0 <<
 >> jtreg:test/jdk:tier1                               1867 1860     
7     0 <<
 >> jtreg:test/langtools:tier1                         3922 2470  
1450     2 <<
    jtreg:test/nashorn:tier1                              0 0     0     0
    jtreg:test/jaxp:tier1                                 0 0     0     0
 >> jtreg:test/jdk:tier2                               3334 3305    
29     0 <<
 >> jtreg:test/langtools:tier2 11     9     2     0 <<
    jtreg:test/nashorn:tier2                             35 35     0     0
 >> jtreg:test/jaxp:tier2                               438 437     
1     0 <<
 >> jtreg:test/jdk:tier3                               1104 1068    
36     0 <<
    jtreg:test/langtools:tier3                            0 0     0     0
    jtreg:test/nashorn:tier3                              0 0     0     0
    jtreg:test/jaxp:tier3                                 0 0     0     0
==============================

On the same machine,when i set LANG=en_US.UTF-8.

==============================
Test summary
==============================
    TEST                                              TOTAL  PASS FAIL 
ERROR
 >> jtreg:test/hotspot/jtreg:tier1                     1388 1386     
2     0 <<
 >> jtreg:test/jdk:tier1                               1867 1843    
19     5 <<
    jtreg:test/langtools:tier1                         3920 3920     
0     0
    jtreg:test/nashorn:tier1                              0 0     0     0
    jtreg:test/jaxp:tier1                                 0 0     0     0
 >> jtreg:test/jdk:tier2                               3328 3290    
31     7 <<
    jtreg:test/langtools:tier2                           11 11     0     0
    jtreg:test/nashorn:tier2                             35 35     0     0
    jtreg:test/jaxp:tier2                               438 438     0     0
 >> jtreg:test/jdk:tier3                               1104 1067    
37     0 <<
    jtreg:test/langtools:tier3                            0 0     0     0
    jtreg:test/nashorn:tier3                              0 0     0     0
    jtreg:test/jaxp:tier3                                 0 0     0     0


By comparison we can find, lots of(1000+) langtools tests will get fail, 
and other(about 30+, exclude some X11 and sanity that

result problem) test cases will also fail because of local LANG.


such as in the test/hotspot/jtreg/compiler/c2/Test8062950.java,

shouldContain("Error: Could not find or load main class " + CLASSNAME)


When in the zh_CN.UTF-8 environment, because some of the output 
information is changed to Chinese by some properties file,

the English cannot be matched, which will result in an fail.

When I set  LANG=en_US/LC_ALL=C, this test will pass.


I think there are some possible solutions.


1.we set LC_ALL=C/LANG=EN_us in the Runtests.gmk, but this modification 
is more violent because he will affect all test cases.


2.We modify each individual test，E.g 
test/hotspot/jtreg/compiler/c2/Test8062950.java

diff -r 0421d49b6217 test/hotspot/jtreg/compiler/c2/Test8062950.java
  package compiler.c2;
  import jdk.test.lib.process.ProcessTools;
+import java.util.Map;
+import jdk.test.lib.process.OutputAnalyzer;

  public class Test8062950 {
      private static final String CLASSNAME = "DoesNotExist";
      public static void main(String[] args) throws Exception {
- ProcessTools.executeTestJvm("-Xcomp",
- "-XX:-TieredCompilation",
- "-XX:-UseOptoBiasInlining",
- CLASSNAME)
- .shouldHaveExitValue(1)
-                    .shouldContain("Error: Could not find or load main 
class " + CLASSNAME)
-                    .shouldNotContain("A fatal error has been detected")
-                    .shouldNotContain("Internal Error")
-                    .shouldNotContain("HotSpot Virtual Machine Error");
+        final ProcessBuilder pb = 
ProcessTools.createJavaProcessBuilder(true,
+ "-Xcomp",
+ "-XX:-TieredCompilation",
+ "-XX:-UseOptoBiasInlining",
+ CLASSNAME);
+        final Map<String, String> env = pb.environment();
+        env.put("LC_ALL", "en_US.UTF-8");
+        OutputAnalyzer output = new OutputAnalyzer(pb.start());
+ output.shouldHaveExitValue(1);
+        output.shouldContain("Error: Could not find or load main class 
" + CLASSNAME);
+        output.shouldNotContain("A fatal error has been detected");
+        output.shouldNotContain("Internal Error");
+        output.shouldNotContain("HotSpot Virtual Machine Error");
}
  }

But I don't know if this will change the test program too much, because 
the related problems are a lot in langtools tests.


3.And i find that there is a function can judge the locale

  if (!isEnglishLocale()) { // only english version
return;
  }

But in this case, I think in many test cases, we may not be able to 
return so directly, because some test cases may have other test purposes.


So I don't know if you have any ideas or some suggestions to solve the 
problem that the output information and English do not match in this

non-English environment.


Cheers,

Jing Tian