Cannot configure on Windows in Chinese Environment

Mon Nov 6 13:00:46 UTC 2023

Let me expand a bit on what Erik says, and also somewhat contradict him. :-)

There is an open bug for documenting that en-US locale is needed to 
build the JDK on Windows: https://bugs.openjdk.org/browse/JDK-8264425

Unfortunately I have never gotten around to actually write this down in 
the docs. I'll try to prioritize it, since it's a simple fix and can 
help others in your position.

To expand on what Erik says: Yes, we have no principal opinion 
discriminating against non en-US locale support, and in general we 
accept patches that help users build in different scenarios. For 
non-Windows platforms, all user locales are supported, since we can set 
LC_ALL=C in the build and run with an international locale.

Unfortunately, this or any other method of temporarily changing the 
locale is not supported on Windows. :-( There have been several attempts 
over the year to overcome this problem, none of which has been 
successful. (You can search the archives of this mailing list for 
examples.) Therefore the conclusion, after the last such effort, was 
that we can only ever support building on en-US on Windows.

The problem on supporting the JDK on a different locale is that there 
are so many small things that can go wrong. Just to give an example on 
the top of my mind: a few weeks ago, the code that set the en-US locale 
to jtreg testing went AWOL. This caused some jtreg test runs to fail on 
a Swiss (iirc) locale, since that used a quote (´) as thousands 
separator, which caused parse errors.

Getting stuck on the version parsing of the compiler is just the very 
first steps on a road filled with pain and suffering.

So, to contradict with what Erik says: No, I don't think we should 
accept a patch that changes version determination for cl.exe from string 
parsing to compiling macros.

There are several reasons: compiling code in configure is always tricky. 
This would be done before we have even determined what compiler we have 
or what version it is. We used to have a binary "fixpath" tool that was 
compiled early on, it was a constant source of trouble, and have now 
been removed due to those problems.

This would also add complexity to the configure script, since a trivial 
method (read and parse the output of running with --version or similar) 
that is shared by all compilers, now need to be replaced with a 
different method for cl.exe only.

If this was guaranteed to be the only things needed to make the JDK 
build and test on non-en-US locales, then I could probably consider it. 
But it is highly unlikely to be. And even if the build passes without 
error, I would be pretty wary about assuming that the build is actually 
identical to one built on the "official" locale.

I encourage you to get in touch with Microsoft and request them to add a 
solution similar to LC_ALL, so processes can run in a different locale 
than the default user locale. I realize a single voice does not convince 
them, but if the message is repeated over and over from all kinds of 
developers, it might have some effect.

/Magnus

On 2023-11-04 13:12, 吴 国璋 wrote:
>
> If making the build work on different locales is accepted, then we can 
> further discuss on this topic.
>
> I would like to implement this with C macros instead of parsing the 
> output, because the MSVC reference includes the C predefined macros, 
> but does not include the output format.
>
> In fact, Visual Studio supports 14 language packs, and I only know how 
> cl.exe presents itself in English and Chinese, maybe also French. 
> Maybe the sentence structure is also different in Korean or Japanese, 
> I am not sure. With C macros the implementation will be less impacted 
> by locale and more stable.
>
> *From: *erik.joelsson at oracle.com
> *Sent: *2023年11月3日21:04
> *To: *吴国璋 <mailto:zcxsythenew at outlook.com>; David Holmes 
> <mailto:david.holmes at oracle.com>; build-dev at openjdk.org
> *Subject: *Re: Cannot configure on Windows in Chinese Environment
>
> On 11/2/23 22:18, 吴 国璋wrote:
>
>     If OpenJDK requires en-us environment, then nothing needs to be
>     changed. Please ignore this thread.
>
> I should clarify this a bit. We aren't against making the build work 
> on different locales, but most of us are unable to verify that it 
> keeps working on anything by US-English. If you are willing to put in 
> the work to make it work in a Chinese environment, and the set of 
> changes required seem reasonable, we would accept that contribution. 
> Just be prepared to maintain that support over time, as it's quite 
> likely that future changes may break it.
>
>     >>
>
>     >> 1. Does JDK welcome localized Visual Studio?
>     >> I read the file `make/autoconf/toolchains.m4` and found the
>     following comment:
>     >> >   elif test  "x$TOOLCHAIN_TYPE" = xmicrosoft; then
>     >> >     # There is no specific version flag, but all output
>     starts with a version string.
>     >> >     # First line typically looks something like:
>     >> >     # Microsoft (R) 32-bit C/C++ Optimizing Compiler Version
>     16.00.40219.01 for 80x86
>     >> >     # but the compiler name may vary depending on locale.
>     >> >     COMPILER_VERSION_OUTPUT=`$COMPILER 2>&1 1>/dev/null |
>     $HEAD -n 1 | $TR -d '\r'`
>     >>
>     >> Therefore, it can be inferred that JDK knows that there are
>     different localizations of Visual Studio and is ready for them.
>     JDK thinks that maybe there will be different names of "Optimizing
>     Compiler" or others. However, in some languages, the whole
>     structure  of the sentence is completely different, not just the
>     names.
>
> So far we have only received contributions for different western type 
> languages, so the current parsing logic has only been adapted for that.
>
>     >> 2. How can the problem be solved?
>     >> One solution is to change the way to parse the output of
>     `cl.exe`. For example, JDK treat the last word separated by a
>     blank as the target CPU, which is "x64" in English environment but
>     "版" in Chinese environment. (`make/autoconf/toolchain.m4`, Lines
>     983  to 997.) We may use `grep` command to search for "x64"
>     directly, and
>     > then the issue can be solved.
>     >> However, this solution is not good enough, because it is also
>     based on parsing the output, which is intended to be read by
>     human, not by scripts. (What if "x64" is changed into "64-bit" or
>     "64 位" in a future version?)
>
> If the output changes, we adapt the regex. We need explicit changes to 
> support a new major version of Visual Studio anyway, so it would be 
> done as part of those changes. The likelihood of it changing in a 
> minor update is basically non existent.
>
>     >> 3. What is the best solution?
>     >> According to MSVC reference, a solid way to get the MSVC
>     version and the target CPU is via predefined macros.
>     >> To get the MSVC version, we can use `_MSC_VER`. When Visual
>     Studio 2019 is used, the macro evaluates between 1920 and 1929.
>     When Visual Studio 2022 is used, the macro evaluates above 1930.
>     >> To get the target CPU, we can use `_M_X64`, `_M_IX86`, `_M_ARM`
>     and `_M_ARM64`. For example, if the target CPU is x64, `_M_X64`
>     will be evaluated to 100, and the other three macros are undefined.
>
> Using the C preprocessor may work, but as Robbin pointed out, we would 
> prefer if you used one of the Autoconf macros for generating the input 
> files and running it if you were to go that route. However, I think I 
> would prefer if you could just adapt the current logic for parsing the 
> compiler version output.
>
> /Erik
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/build-dev/attachments/20231106/102c3ab0/attachment-0001.htm>