configure error on WSL 2
Yasumasa Suenaga
suenaga at oss.nttdata.com
Tue Jun 2 13:34:00 UTC 2020
Hi Magnus,
I tried to build OpenJDK on WSL2 with your ihse-wsl-branch branch, but unfortunately I saw same errors.
Some of them have been gone with following patch:
```
diff --git a/make/autoconf/toolchain.m4 b/make/autoconf/toolchain.m4
--- a/make/autoconf/toolchain.m4
+++ b/make/autoconf/toolchain.m4
@@ -427,7 +427,7 @@
# First line typically looks something like:
# Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.40219.01 for 80x86
# but the compiler name may vary depending on locale.
- COMPILER_VERSION_OUTPUT=`"$COMPILER" 2>&1 | $GREP -v 'ERROR.*UtilTranslatePathList' | $HEAD -n 1 | $TR -d '\r'`
+ COMPILER_VERSION_OUTPUT=`"$COMPILER" 2>&1 1>/dev/null | $HEAD -n 1 | $TR -d '\r'`
# Check that this is likely to be Microsoft CL.EXE.
$ECHO "$COMPILER_VERSION_OUTPUT" | $GREP "Microsoft" > /dev/null
if test $? -ne 0; then
@@ -959,7 +959,7 @@
# Check for extra potential brokenness.
if test "x$TOOLCHAIN_TYPE" = xmicrosoft; then
# On Windows, double-check that we got the right compiler.
- CC_VERSION_OUTPUT=`$CC 2>&1 | $GREP -v 'ERROR.*UtilTranslatePathList' | $HEAD -n 1 | $TR -d '\r'`
+ CC_VERSION_OUTPUT=`$CC 2>&1 1>/dev/null | $HEAD -n 1 | $TR -d '\r'`
COMPILER_CPU_TEST=`$ECHO $CC_VERSION_OUTPUT | $SED -n "s/^.* \(.*\)$/\1/p"`
if test "x$OPENJDK_TARGET_CPU" = "xx86"; then
if test "x$COMPILER_CPU_TEST" != "x80x86" -a "x$COMPILER_CPU_TEST" != "xx86"; then
diff --git a/make/src/native/fixpath.c b/make/src/native/fixpath.c
--- a/make/src/native/fixpath.c
+++ b/make/src/native/fixpath.c
@@ -32,7 +32,7 @@
void report_error(char const * msg)
{
- LPVOID lpMsgBuf;
+ LPTSTR lpMsgBuf;
DWORD dw = GetLastError();
FormatMessage(
```
I still see the error about `wslpath -m`.
According to WSL release note [1], GitHub issue #4078 [2] has been fixed in Build 18945 (it is successor of November 2019 update).
I guess this issue would be occur on Windows 10 20H1 even if the build is happen on WSL1.
I strace'ed `wslpath -m`, it attempt to open the file at first. I guess it relates to the #4078 fix.
Thanks,
Yasumasa
[1] https://docs.microsoft.com/windows/wsl/release-notes
[2] https://github.com/microsoft/WSL/issues/4078
On 2020/06/02 18:58, Magnus Ihse Bursie wrote:
> On 2020-06-01 15:36, Yasumasa Suenaga wrote:
>> Hi Magnus,
>>
>> On 2020/06/01 20:06, Magnus Ihse Bursie wrote:
>>> On 2020-05-31 14:39, Yasumasa Suenaga wrote:
>>>> Hi all,
>>>>
>>>> I tried to build jdk/jdk on WSL 2 with VS2019, but I saw three errors as below.
>>>>
>>>>
>>>> 1) Could not detect VC (cl.exe) version. So configure reports that it could not find valid Microsoft C++ compiler.
>>>>
>>>> 2) Could not generate fixpath.exe because $CONFIGURESUPPORT_OUTPUTDIR/bin did not exist. `wslpath -m` in UTIL_REWRITE_AS_WINDOWS_MIXED_PATH - it does not seem to work with unavailable path.
>>>>
>>>> 3) LPVOID value is passed to FormatMessage() in fixpath.c . FormatMessage() accepts value for lpBuffer as LPTSTR.
>>>>
>>>>
>>>> I think it is worth to fix. Someone already has worked for this?
>>> I have been experimenting with building on WSL2. I've noticed the same issues as you have, but solved them somewhat differently. For instance, you cannot grep for "Optimizing Compiler" since this phrase is localized. We've had, and fixed, that bug before.
>>
>> I run on cl.exe on Japanese locale, but it is not localized.
>> (Is it because Japanese...? but the usage is localized.)
>>
>> ```
>> $ /mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe
>> Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28805 for x64
>> Copyright (C) Microsoft Corporation. All rights reserved.
>>
>> 使い方: cl [ オプション... ] ファイル名... [ /link リンク オプション... ]
>> ```
> I can't say anything about the Japanese locale, but I know for a fact that we've had localization issues. I think it was the French or Portuguese locale that had a "Compileur Optimisée" or something like that.
>>
>>
>>> Incidentally, that fix is actually a workaround for what looks like a worrying issue in WSL2: race conditions on piping. Check this output:
>>>
>>> magnusi at winihse:/mnt/c/localdata/hg/sandbox-ALT$ /mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe 2>&1 | cat
>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
>>> usage: cl [ option... ] filename... [ /link linkoption... ]
>>> Copyright (C) Microsoft Corporation. All rights reserved.
>>>
>>> magnusi at winihse:/mnt/c/localdata/hg/sandbox-ALT$ /mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe 2>&1 | cat
>>> usage: cl [ option... ] filename... [ /link linkoption... ]
>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
>>> Copyright (C) Microsoft Corporation. All rights reserved.
>>>
>>> magnusi at winihse:/mnt/c/localdata/hg/sandbox-ALT$ /mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe
>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
>>> Copyright (C) Microsoft Corporation. All rights reserved.
>>>
>>> usage: cl [ option... ] filename... [ /link linkoption... ]
>>> magnusi at winihse:/mnt/c/localdata/hg/sandbox-ALT$ /mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe
>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
>>> Copyright (C) Microsoft Corporation. All rights reserved.
>>>
>>> usage: cl [ option... ] filename... [ /link linkoption... ]
>>>
>>> Notice how the order of lines are random when piping through cat. Worrying, indeed. I wonder what more will break...
>>
>> AFAICS the usage is in stdout, the banner is in stderr.
>>
>> ```
>> $ /mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe 2>&1 1>/dev/null
>> Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28805 for x64
>> Copyright (C) Microsoft Corporation. All rights reserved.
>>
>> $ /mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe 2>/dev/null
>> 使い方: cl [ オプション... ] ファイル名... [ /link リンク オプション... ]
>> ```
>>
>> So we can catch up version string from cl.exe as below:
>>
>> ```
>> $ /mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe 2>&1 1>/dev/null | head -n 1
>> Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28805 for x64
>> ```
> Good thinking. That is probably more stable than grepping for "microsoft".
>>
>>
>>> Also, wslpath suddenyl not working with non-existing paths looks like a pure bug to me. :( But the correct workaround is not to go around and touch arbitrary files in the file system. For one thing, you might not have write permission there. I think the proper way (short of awaiting a bug fix from Microsoft) is to iteratively remove the last part of the path until wslpath succeeds, and then add back on what we removed.
>>>
>>> Or we need to check how we are using wslpath. If it is used to verify if a file is present, we can use the failure from wslpath as a sign that the file is missing, instead of trying first to convert it to unix path, and then check if it is present.
>>
>> In my understand, wslpath for FIXPATH_BIN_W is to convert to the path for Windows (for cl.exe).
>> So I think it is nature even if it does not exist.
>> In addition, if we do not have write permission, it should be failed.
> We need to have support for general paths, regardless if they exist or if we have write permission. It might be to convert a given path to a configure option or whatever. So we need to be able to handle these cases too.
>>
>>
>>> ---
>>>
>>> However, the main problem when working with WSL2 is that it is dead slow, for building OpenJDK. :-( The method that was used in WSL1, to have the source code and build directory on a Windows share, is not working at all in WSL2. Access of NTFS drives has horrendous performance. :( On the flip side, WSL2 adds access to the Linux drive for Windows apps. So it might be possible to do the reverse -- make sure we store source and build directory on a Linux drive, and let the windows apps access it.
>>>
>>> I have started testing this but have not had time to finish my testing. A lot of the Windows integration code in autoconf do not work when you do this.
>>
>> Yeah, I haven't yet succeeded to build OpenJDK on WSL2 because I/O is very slow... but I believe Microsoft will be improve I/O performance some day :)
>>
>> Your suggestion to access to WSL partition from Windows sounds good to me, but I think configure script should be detect it runs on whether WSL 1 or 2.
>> However I couldn't find how can we detect it smartly.
> Yes, configure should check what version of WSL we are running on. And yes, it's not trivial to detect. Idiotic move by Microsoft to not make this information accessible. :-( But I've found ways to do this.
>
> I'm implementing this, and other fixes, in the jdk/sandbox repo, in the branch ihse-wsl-branch. You are free to check it out and test how it works for you. (In fact, your assistance would be appreciated!) Please note that this is a work in progress, and it can/will break from time to time. :-) I make changes in the branch and then try to verify them in different environments.
>
> /Magnus
>
>>
>>
>> Thanks,
>>
>> Yasumasa
>>
>>
>>> /Magnus
>>>
>>>> If not, I will file it to JBS, and will send review request.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Yasumasa
>>>
>
More information about the build-dev
mailing list