Cannot configure on Windows in Chinese Environment
Magnus Ihse Bursie
magnus.ihse.bursie at oracle.com
Tue Nov 14 14:41:25 UTC 2023
On 2023-11-06 15:56, 吴国璋 wrote:
> Thanks Magnus for your opionion. Just one thing to point out: reaching
> out to Microsoft is not needed. Here is the reason:
>
> If Visual Studio is installed with both English and another language
> pack, cl.exe will present itself in English while running configure.
>
> If Visual Studio is installed without English language pack, cl.exe
> will present itself in an installed language.
>
> Therefore now the issue is that the documentation needs to be improved.
That is great news! While I'm still slightly worried that there can be
other issues lurking around the corner with a non-en-US locale, if the
Microsoft toolchain is in English it would at least take care of the
majority of the problems. There are two other broad categories of tools
we're running in the build, java tools and "unix" (cygwin) tools. The
former are already using the proper locale using -Duser.language=en
-Duser.country=US, and the latter using LC_ALL=C. Apart from this, the
compiler toolchain is the major source of issues.
However, I'd like to have some more investigation into how cl.exe picks
the language.
Is it a hard-coded behavior that it always picks English if it is
installed? If so, it does not really make sense to have more than one
language pack installed. Or is there some way to tell cl.exe which
language to pick? Perhaps we send those signals without realizing it,
like if it happens to look at LC_ALL. If that is the case, we need to
know about this, so we don't inadvertently break it.
I have never installed a VS language pack. Is it installed like other VS
components?
/Magnus
>
> Before I sent the original email, I tried to find out whether JDK
> welcomes other languages, and I found in the code that JDK did pay
> some effort for supporting them, so I assumed that an en-us
> environment is not required. Since it is not the case, then the
> documentation needs improvement.
>
> In details, the documentation needs the following modification:
>
> - It needs to advice developers to install Visual Studio English
> language pack, if they speak another language as their mother tongue;
> - Switching the Windows system to English is not required (I tried it
> several times, just installing the language pack can solve)
>
> 2023年11月6日 21:00，Magnus Ihse Bursie
> <magnus.ihse.bursie at oracle.com>写道：
>
> Let me expand a bit on what Erik says, and also somewhat
> contradict him. :-)
>
> There is an open bug for documenting that en-US locale is needed
> to build the JDK on Windows:
> https://bugs.openjdk.org/browse/JDK-8264425
>
> Unfortunately I have never gotten around to actually write this
> down in the docs. I'll try to prioritize it, since it's a simple
> fix and can help others in your position.
>
> To expand on what Erik says: Yes, we have no principal opinion
> discriminating against non en-US locale support, and in general we
> accept patches that help users build in different scenarios. For
> non-Windows platforms, all user locales are supported, since we
> can set LC_ALL=C in the build and run with an international locale.
>
> Unfortunately, this or any other method of temporarily changing
> the locale is not supported on Windows. :-( There have been
> several attempts over the year to overcome this problem, none of
> which has been successful. (You can search the archives of this
> mailing list for examples.) Therefore the conclusion, after the
> last such effort, was that we can only ever support building on
> en-US on Windows.
>
> The problem on supporting the JDK on a different locale is that
> there are so many small things that can go wrong. Just to give an
> example on the top of my mind: a few weeks ago, the code that set
> the en-US locale to jtreg testing went AWOL. This caused some
> jtreg test runs to fail on a Swiss (iirc) locale, since that used
> a quote (´) as thousands separator, which caused parse errors.
>
> Getting stuck on the version parsing of the compiler is just the
> very first steps on a road filled with pain and suffering.
>
> So, to contradict with what Erik says: No, I don't think we should
> accept a patch that changes version determination for cl.exe from
> string parsing to compiling macros.
>
> There are several reasons: compiling code in configure is always
> tricky. This would be done before we have even determined what
> compiler we have or what version it is. We used to have a binary
> "fixpath" tool that was compiled early on, it was a constant
> source of trouble, and have now been removed due to those problems.
>
> This would also add complexity to the configure script, since a
> trivial method (read and parse the output of running with
> --version or similar) that is shared by all compilers, now need to
> be replaced with a different method for cl.exe only.
>
> If this was guaranteed to be the only things needed to make the
> JDK build and test on non-en-US locales, then I could probably
> consider it. But it is highly unlikely to be. And even if the
> build passes without error, I would be pretty wary about assuming
> that the build is actually identical to one built on the
> "official" locale.
>
> I encourage you to get in touch with Microsoft and request them to
> add a solution similar to LC_ALL, so processes can run in a
> different locale than the default user locale. I realize a single
> voice does not convince them, but if the message is repeated over
> and over from all kinds of developers, it might have some effect.
>
> /Magnus
>
>
>
>
> On 2023-11-04 13:12, 吴 国璋 wrote:
>
> If making the build work on different locales is accepted,
> then we can further discuss on this topic.
>
> I would like to implement this with C macros instead of
> parsing the output, because the MSVC reference includes the C
> predefined macros, but does not include the output format.
>
> In fact, Visual Studio supports 14 language packs, and I only
> know how cl.exe presents itself in English and Chinese, maybe
> also French. Maybe the sentence structure is also different in
> Korean or Japanese, I am not sure. With C macros the
> implementation will be less impacted by locale and more stable.
>
> *From: *erik.joelsson at oracle.com
> *Sent: *2023年11月3日 21:04
> *To: *吴 国璋 <mailto:zcxsythenew at outlook.com>; David Holmes
> <mailto:david.holmes at oracle.com>; build-dev at openjdk.org
> *Subject: *Re: Cannot configure on Windows in Chinese Environment
>
> On 11/2/23 22:18, 吴 国璋 wrote:
>
> If OpenJDK requires en-us environment, then nothing needs
> to be changed. Please ignore this thread.
>
> I should clarify this a bit. We aren't against making the
> build work on different locales, but most of us are unable to
> verify that it keeps working on anything by US-English. If you
> are willing to put in the work to make it work in a Chinese
> environment, and the set of changes required seem reasonable,
> we would accept that contribution. Just be prepared to
> maintain that support over time, as it's quite likely that
> future changes may break it.
>
> >>
>
> >> 1. Does JDK welcome localized Visual Studio?
> >> I read the file `make/autoconf/toolchains.m4` and found
> the following comment:
> >> > elif test "x$TOOLCHAIN_TYPE" = xmicrosoft; then
> >> > # There is no specific version flag, but all
> output starts with a version string.
> >> > # First line typically looks something like:
> >> > # Microsoft (R) 32-bit C/C++ Optimizing Compiler
> Version 16.00.40219.01 for 80x86
> >> > # but the compiler name may vary depending on locale.
> >> > COMPILER_VERSION_OUTPUT=`$COMPILER 2>&1 1>/dev/null |
> $HEAD -n 1 | $TR -d '\r'`
> >>
> >> Therefore, it can be inferred that JDK knows that there
> are different localizations of Visual Studio and is ready
> for them. JDK thinks that maybe there will be different
> names of "Optimizing Compiler" or others. However, in some
> languages, the whole structure of the sentence is
> completely different, not just the names.
>
> So far we have only received contributions for different
> western type languages, so the current parsing logic has only
> been adapted for that.
>
> >> 2. How can the problem be solved?
> >> One solution is to change the way to parse the output
> of `cl.exe`. For example, JDK treat the last word
> separated by a blank as the target CPU, which is "x64" in
> English environment but "版" in Chinese environment.
> (`make/autoconf/toolchain.m4`, Lines 983 to 997.) We may
> use `grep` command to search for "x64" directly, and
> > then the issue can be solved.
> >> However, this solution is not good enough, because it
> is also based on parsing the output, which is intended to
> be read by human, not by scripts. (What if "x64" is
> changed into "64-bit" or "64 位" in a future version?)
>
> If the output changes, we adapt the regex. We need explicit
> changes to support a new major version of Visual Studio
> anyway, so it would be done as part of those changes. The
> likelihood of it changing in a minor update is basically non
> existent.
>
> >> 3. What is the best solution?
> >> According to MSVC reference, a solid way to get the
> MSVC version and the target CPU is via predefined macros.
> >> To get the MSVC version, we can use `_MSC_VER`. When
> Visual Studio 2019 is used, the macro evaluates between
> 1920 and 1929. When Visual Studio 2022 is used, the macro
> evaluates above 1930.
> >> To get the target CPU, we can use `_M_X64`, `_M_IX86`,
> `_M_ARM` and `_M_ARM64`. For example, if the target CPU is
> x64, `_M_X64` will be evaluated to 100, and the other
> three macros are undefined.
>
> Using the C preprocessor may work, but as Robbin pointed out,
> we would prefer if you used one of the Autoconf macros for
> generating the input files and running it if you were to go
> that route. However, I think I would prefer if you could just
> adapt the current logic for parsing the compiler version output.
>
> /Erik
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/build-dev/attachments/20231114/505736f2/attachment-0001.htm>
More information about the build-dev
mailing list