RFR: 8333427: langtools/tools/javac/newlines/NewLineTest.java is failing on Japanese Windows

[KIRIYAMA Takuya]

On Fri, 14 Jun 2024 07:08:42 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> I fixed to get Charset from native.encoding instead of Charset.defaultCharset() when reading a file to which the output of javac run in the test was redirected.
>> The modified code is based on the alternatives given in JEP400.
>> I verified that the test passed on Windows in both Japanese and English locales with this fix.
>
> test/langtools/tools/javac/newlines/NewLineTest.java line 61:
> 
>> 59:                 .run(Task.Expect.FAIL);
>> 60: 
>> 61:         String encoding = System.getProperty("native.encoding");
> 
> Hello @tkiriyama, I don't have experience in javac area or this test, but I had a brief look at this change. What I understand is that `javac` will print to `STDOUT` using a `PrinterWriter` for `System.out`, which as per the documentation https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html#out will use `stdout.encoding` value for the character encoding. The default value of `stdout.encoding` standard system property https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html#stdout.encoding is currently not mentioned but I checked with people familiar with this area and it defaults to platform specific values. It is not guaranteed that `stdout.encoding` will be the same value as `native.encoding`. So the proposed change here isn't guaranteed to (always) work.
> 
> I think to make this deterministic, you might have to update the `javac` launch command (a few lines above) to pass `-J-Dstdout.encoding=UTF-8` and then when reading from the file to which the content was redirected, use `StandardCharsets.UTF_8`.
> Can you give that change a try on your setup and see if it solves the issue for you?

Hello @jaikiran, thank you for your advice.

> What I understand is that javac will print to STDOUT using a PrinterWriter for System.out, which as per the documentation https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html#out will use stdout.encoding value for the character encoding. 

I completely agree with you.

> The default value of stdout.encoding standard system property https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/System.html#stdout.encoding is currently not mentioned but I checked with people familiar with this area and it defaults to platform specific values. It is not guaranteed that stdout.encoding will be the same value as native.encoding.

I think that in the current implementation,System.out use native.encoding for the character encoding  if stdout.encoding is not specified. The value of native.encoding is derived from the host environment and user settings. It cannot be changed.

> I think to make this deterministic, you might have to update the javac launch command (a few lines above) to pass -J-Dstdout.encoding=UTF-8 and then when reading from the file to which the content was redirected, use StandardCharsets.UTF_8.
> Can you give that change a try on your setup and see if it solves the issue for you?


        new JavacTask(tb, Task.Mode.EXEC)
                .redirect(Task.OutputKind.STDOUT, javacOutput.getPath())
                .options("-J-Dline.separator='@'", "-J-Dstdout.encoding=UTF-8")
                .run(Task.Expect.FAIL);

        List<String> lines = Files.readAllLines(javacOutput.toPath(), StandardCharsets.UTF_8);
        if (lines.size() != 1) {
            throw new AssertionError("The compiler output should have one line only");
        }

I verified that this test code passed. If the test using only UTF-8 is sufficient, this seems to be  the best fix. 
I am concerned that it would be better to test with the default encoding in a test runnning environment if possible. Therefore, I use the generic alternatives exposed in [JEP 400](https://openjdk.org/jeps/400).
Should I ignore this concern and specify UTF-8 for the test?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19519#discussion_r1642230032