configure error on WSL 2

[Magnus Ihse Bursie]

On 2020-06-01 15:36, Yasumasa Suenaga wrote:
> Hi Magnus,
>
> On 2020/06/01 20:06, Magnus Ihse Bursie wrote:
>> On 2020-05-31 14:39, Yasumasa Suenaga wrote:
>>> Hi all,
>>>
>>> I tried to build jdk/jdk on WSL 2 with VS2019, but I saw three 
>>> errors as below.
>>>
>>>
>>> 1) Could not detect VC (cl.exe) version. So configure reports that 
>>> it could not find valid Microsoft C++ compiler.
>>>
>>> 2) Could not generate fixpath.exe because 
>>> $CONFIGURESUPPORT_OUTPUTDIR/bin did not exist. `wslpath -m` in 
>>> UTIL_REWRITE_AS_WINDOWS_MIXED_PATH - it does not seem to work with 
>>> unavailable path.
>>>
>>> 3) LPVOID value is passed to FormatMessage() in fixpath.c . 
>>> FormatMessage() accepts value for lpBuffer as LPTSTR.
>>>
>>>
>>> I think it is worth to fix. Someone already has worked for this?
>> I have been experimenting with building on WSL2. I've noticed the 
>> same issues as you have, but solved them somewhat differently. For 
>> instance, you cannot grep for "Optimizing Compiler" since this phrase 
>> is localized. We've had, and fixed, that bug before.
>
> I run on cl.exe on Japanese locale, but it is not localized.
> (Is it because Japanese...? but the usage is localized.)
>
> ```
> $ 
> /mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe
> Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28805 for x64
> Copyright (C) Microsoft Corporation.  All rights reserved.
>
> 使い方: cl [ オプション... ] ファイル名... [ /link リンク オプション... ]
> ```
I can't say anything about the Japanese locale, but I know for a fact 
that we've had localization issues. I think it was the French or 
Portuguese locale that had a "Compileur Optimisée" or something like that.
>
>
>> Incidentally, that fix is actually a workaround for what looks like a 
>> worrying issue in WSL2: race conditions on piping. Check this output:
>>
>> magnusi at winihse:/mnt/c/localdata/hg/sandbox-ALT$ 
>> /mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe 
>> 2>&1 | cat
>> Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
>> usage: cl [ option... ] filename... [ /link linkoption... ]
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>
>> magnusi at winihse:/mnt/c/localdata/hg/sandbox-ALT$ 
>> /mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe 
>> 2>&1 | cat
>> usage: cl [ option... ] filename... [ /link linkoption... ]
>> Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>
>> magnusi at winihse:/mnt/c/localdata/hg/sandbox-ALT$ 
>> /mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe
>> Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>
>> usage: cl [ option... ] filename... [ /link linkoption... ]
>> magnusi at winihse:/mnt/c/localdata/hg/sandbox-ALT$ 
>> /mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe
>> Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>
>> usage: cl [ option... ] filename... [ /link linkoption... ]
>>
>> Notice how the order of lines are random when piping through cat. 
>> Worrying, indeed. I wonder what more will break...
>
> AFAICS the usage is in stdout, the banner is in stderr.
>
> ```
> $ 
> /mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe 
> 2>&1 1>/dev/null
> Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28805 for x64
> Copyright (C) Microsoft Corporation.  All rights reserved.
>
> $ 
> /mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe 
> 2>/dev/null
> 使い方: cl [ オプション... ] ファイル名... [ /link リンク オプション... ]
> ```
>
> So we can catch up version string from cl.exe as below:
>
> ```
> $ 
> /mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe 
> 2>&1 1>/dev/null | head -n 1
> Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28805 for x64
> ```
Good thinking. That is probably more stable than grepping for "microsoft".
>
>
>> Also, wslpath suddenyl not working with non-existing paths looks like 
>> a pure bug to me. :( But the correct workaround is not to go around 
>> and touch arbitrary files in the file system. For one thing, you 
>> might not have write permission there. I think the proper way (short 
>> of awaiting a bug fix from Microsoft) is to iteratively remove the 
>> last part of the path until wslpath succeeds, and then add back on 
>> what we removed.
>>
>> Or we need to check how we are using wslpath. If it is used to verify 
>> if a file is present, we can use the failure from wslpath as a sign 
>> that the file is missing, instead of trying first to convert it to 
>> unix path, and then check if it is present.
>
> In my understand, wslpath for FIXPATH_BIN_W is to convert to the path 
> for Windows (for cl.exe).
> So I think it is nature even if it does not exist.
> In addition, if we do not have write permission, it should be failed.
We need to have support for general paths, regardless if they exist or 
if we have write permission. It might be to convert a given path to a 
configure option or whatever. So we need to be able to handle these 
cases too.
>
>
>> ---
>>
>> However, the main problem when working with WSL2 is that it is dead 
>> slow, for building OpenJDK. :-( The method that was used in WSL1, to 
>> have the source code and build directory on a Windows share, is not 
>> working at all in WSL2. Access of NTFS drives has horrendous 
>> performance. :( On the flip side, WSL2 adds access to the Linux drive 
>> for Windows apps. So it might be possible to do the reverse -- make 
>> sure we store source and build directory on a Linux drive, and let 
>> the windows apps access it.
>>
>> I have started testing this but have not had time to finish my 
>> testing. A lot of the Windows integration code in autoconf do not 
>> work  when you do this.
>
> Yeah, I haven't yet succeeded to build OpenJDK on WSL2 because I/O is 
> very slow... but I believe Microsoft will be improve I/O performance 
> some day :)
>
> Your suggestion to access to WSL partition from Windows sounds good to 
> me, but I think configure script should be detect it runs on whether 
> WSL 1 or 2.
> However I couldn't find how can we detect it smartly.
Yes, configure should check what version of WSL we are running on. And 
yes, it's not trivial to detect. Idiotic move by Microsoft to not make 
this information accessible. :-( But I've found ways to do this.

I'm implementing this, and other fixes, in the jdk/sandbox repo, in the 
branch ihse-wsl-branch. You are free to check it out and test how it 
works for you. (In fact, your assistance would be appreciated!) Please 
note that this is a work in progress, and it can/will break from time to 
time. :-) I make changes in the branch and then try to verify them in 
different environments.

/Magnus

>
>
> Thanks,
>
> Yasumasa
>
>
>> /Magnus
>>
>>> If not, I will file it to JBS, and will send review request.
>>>
>>>
>>> Thanks,
>>>
>>> Yasumasa
>>