[PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows

Sat Dec 15 01:41:39 UTC 2018

After having configured my WSL to mount using case=off, I was able to 
successfully build images using the latest patch as applied in the sandbox.

/Erik

On 2018-12-14 17:23, Erik Joelsson wrote:
> Hello again,
>
> I took the liberty of creating a bug for this and also a sandbox 
> branch where I've applied your latest patch. If you clone that you can 
> send further patches based on a known state in the sandbox. This will 
> make it easier to see what you are actually doing in each update, as 
> well as give us better references when discussing them. It also gives 
> me the ability to directly change things so we can keep Cygwin/msys 
> working.
>
> https://bugs.openjdk.java.net/browse/JDK-8215445
>
> http://hg.openjdk.java.net/jdk/sandbox/shortlog/12615de8335e
>
> /Erik
>
> On 2018-12-14 16:47, Erik Joelsson wrote:
>> Hello,
>>
>> You beat me to it. I just found the rc.exe problem was that 
>> FIXPATH_PATH in spec.gmk.in was quoted. Make just propagates quotes 
>> verbatim, so then fixpath.c would create a path variable like;
>>
>> $PATH;"$FIXPATH_PATH"
>>
>> Which is why link.exe could not find rc.exe.
>>
>> /Erik
>>
>> On 2018-12-14 16:32, Andrew Luo wrote:
>>> Ok, here's my latest patch (I didn't add your case sensitivity fix 
>>> yet, but will do next patch).  I believe this should get you past 
>>> the rc.exe issues.
>>>
>>> Thanks,
>>>
>>> -Andrew
>>>
>>> -----Original Message-----
>>> From: Erik Joelsson <erik.joelsson at oracle.com>
>>> Sent: Friday, December 14, 2018 4:15 PM
>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>
>>> Cc: Andrew Luo <andrewluotechnologies at outlook.com>; 
>>> build-dev at openjdk.java.net
>>> Subject: Re: [PATCH] Support for building using WSL (Windows 
>>> Subsystem for Linux) on Windows
>>>
>>>
>>> On 2018-12-14 16:06, Magnus Ihse Bursie wrote:
>>>>> 14 dec. 2018 kl. 23:42 skrev Erik Joelsson 
>>>>> <erik.joelsson at oracle.com>:
>>>>>
>>>>> I found the reason it's not failing make. It returns "1" and 
>>>>> NativeCompilation.gmk currently ignores 1 explicitly for Visual 
>>>>> Studio. I added that back in 2014 in 
>>>>> https://bugs.openjdk.java.net/browse/JDK-8065576, but I can't 
>>>>> figure out why. Nothing mentioned in either comment or review.
>>>> Sounds like it's ripe for removal then. :) I wonder what kind of 
>>>> issue you might have run into that caused a returned 1 to happen 
>>>> and yet we didn't want to consider it a failure...
>>> If I'm to guess, I think it's one of the commands we pipe the output 
>>> to when the output is zero. This would explain why it was added 
>>> together with pipefail.
>>>
>>> /Erik
>>>
>>>> /Magnus
>>>>
>>>>> /Erik
>>>>>
>>>>>> On 2018-12-14 13:59, Magnus Ihse Bursie wrote:
>>>>>>
>>>>>>
>>>>>>> On 2018-12-14 22:15, Erik Joelsson wrote:
>>>>>>> I get the same error for pch and it still continues, but this 
>>>>>>> time I let it run until it eventually fails for real when it 
>>>>>>> can't link. Perhaps it's simply cl.exe that isn't returning non 
>>>>>>> zero for this error? When the linker fails, make fails, so 
>>>>>>> propagation doesn't seem broken.
>>>>>> That does also seem really weird, considering that it claims it 
>>>>>> to be a "fatal error". Can you repeat the command at the command 
>>>>>> line and get the failure again, and then check the return value? 
>>>>>> Can you rewrite the command line and run it from the devenv 
>>>>>> prompt? That is, is there any indication that the pch file itself 
>>>>>> is messed up, or can it be used if running the compilation that 
>>>>>> should use it from an "ok" prompt?
>>>>>>
>>>>>> /Magnus
>>>>>>> /Erik
>>>>>>>
>>>>>>>> On 2018-12-14 12:55, Andrew Luo wrote:
>>>>>>>> Hmm, I get the rc.exe error as well, but now it is much later 
>>>>>>>> down the line... Still investigating...
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> -Andrew
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Andrew Luo
>>>>>>>> Sent: Friday, December 14, 2018 12:34 PM
>>>>>>>> To: 'Andrew Luo' <andrewluotechnologies at outlook.com>; Magnus Ihse
>>>>>>>> Bursie <magnus.ihse.bursie at oracle.com>; Erik Joelsson
>>>>>>>> <erik.joelsson at oracle.com>
>>>>>>>> Cc: build-dev at openjdk.java.net
>>>>>>>> Subject: RE: [PATCH] Support for building using WSL (Windows
>>>>>>>> Subsystem for Linux) on Windows
>>>>>>>>
>>>>>>>> Try this updated patch with some fixes...
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> -Andrew
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: build-dev <build-dev-bounces at openjdk.java.net> On Behalf Of
>>>>>>>> Andrew Luo
>>>>>>>> Sent: Friday, December 14, 2018 12:01 PM
>>>>>>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; Erik
>>>>>>>> Joelsson <erik.joelsson at oracle.com>
>>>>>>>> Cc: build-dev at openjdk.java.net
>>>>>>>> Subject: RE: [PATCH] Support for building using WSL (Windows
>>>>>>>> Subsystem for Linux) on Windows
>>>>>>>>
>>>>>>>> I think I have a fix for it.  Give me a minute (or a few hours 
>>>>>>>> depending on if it works).
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> -Andrew
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>
>>>>>>>> Sent: Friday, December 14, 2018 11:42 AM
>>>>>>>> To: Erik Joelsson <erik.joelsson at oracle.com>
>>>>>>>> Cc: Andrew Luo <andrewluotechnologies at outlook.com>;
>>>>>>>> build-dev at openjdk.java.net
>>>>>>>> Subject: Re: [PATCH] Support for building using WSL (Windows
>>>>>>>> Subsystem for Linux) on Windows
>>>>>>>>
>>>>>>>>
>>>>>>>>> 14 dec. 2018 kl. 20:31 skrev Erik Joelsson 
>>>>>>>>> <erik.joelsson at oracle.com>:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 2018-12-14 11:05, Magnus Ihse Bursie wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> On 2018-12-14 19:41, Erik Joelsson wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On 2018-12-14 19:23, Erik Joelsson wrote:
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I took your patch for a spin, and configure passes, but I 
>>>>>>>>>>>>> get the same build error I got with my patch:
>>>>>>>>>>>>>
>>>>>>>>>>>>> fatal error C1083: Cannot open compiler intermediate file:
>>>>>>>>>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\
>>>>>>>>>>>>> varia
>>>>>>>>>>>>> nt-server\libjvm\objs\build_libjvm.pch': No such file or
>>>>>>>>>>>>> directory
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is repeated for every C++ file in Hotspot. I see two 
>>>>>>>>>>>>> issues here. First of all, I need to figure out why the 
>>>>>>>>>>>>> compiler will not find the file, which is clearly there. 
>>>>>>>>>>>>> Second, why isn't this failure picked up by make? 
>>>>>>>>>>>>> Somewhere the return value of cl.exe is disappearing.
>>>>>>>>>>>> Can you build without errors if you disable PCH?
>>>>>>>>>> Could you? That is, is it only the PCH that is problematic?
>>>>>>>>> Trying that now.
>>>>>>>>>>>> Also, a wild guess: can it be related to file permissions? 
>>>>>>>>>>>> Can you read the file properly from both WSL and Windows?
>>>>>>>>>>> It is readable, but it could be something with case. The 
>>>>>>>>>>> file is actually called BUILD_LIBJVM.pch, but that is also 
>>>>>>>>>>> how it's given to the compiler command line. Here is the 
>>>>>>>>>>> output from DEBUG_FIXPATH:
>>>>>>>>>> Weird. What if you, after a failed build, rename it to 
>>>>>>>>>> build_libjvm.pch?
>>>>>>>>> Doing that causes a new error:
>>>>>>>>>
>>>>>>>>> d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. 
>>>>>>>>>
>>>>>>>>> cpp : fatal error C1382: the PCH file
>>>>>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\vari
>>>>>>>>> ant-s erver\libjvm\objs\build_libjvm.pch' has been rebuilt since
>>>>>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\vari
>>>>>>>>> ant-s erver\libjvm\objs\accessBarrierSupport.obj' was generated.
>>>>>>>>> Please rebuild this object
>>>>>>>>>
>>>>>>>>> But I think even more important is that make is not getting 
>>>>>>>>> the error. The build just continues until interrupted.
>>>>>>>> Agree, that's bad.
>>>>>>>>
>>>>>>>> Does fixpath_debug print exit code? If so, what does it say? If 
>>>>>>>> not, we should add that instrumentation.
>>>>>>>>
>>>>>>>> /Magnus
>>>>>>>>
>>>>>>>>>>> Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line
>>>>>>>>>>>> -wsl\build\windows-x86_64-server-release\configure-support\bin
>>>>>>>>>>>> \fixp
>>>>>>>>>>> ath.exe -w
>>>>>>>>>> This starts out quite odd..? -wsl\build\...?
>>>>>>>>> I agree, didn't look into that part.
>>>>>>>>>>> /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270
>>>>>>>>>>> /bin/
>>>>>>>>>>> Hostx86/x64/cl.exe
>>>>>>>>>> Also, FWIW, this seems not to have been properly case 
>>>>>>>>>> treated. Which version of the patch are you using?
>>>>>>>>> The last one posted by Andrew: "diff15.txt".
>>>>>>>>>
>>>>>>>>> /Erik
>>>>>>>>>
>>>>>>>>>> /Magnus
>>>>>>>>>>> -showIncludes
>>>>>>>>>>> -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hots
>>>>>>>>>>> pot/v ariant-server/libjvm/objs/BUILD_LIBJVM.pch
>>>>>>>>>>> -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
>>>>>>>>>>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN
>>>>>>>>>>> -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3
>>>>>>>>>>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86
>>>>>>>>>>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86
>>>>>>>>>>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP
>>>>>>>>>>> -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2
>>>>>>>>>>> -DINCLUDE_ZGC=0
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotsp
>>>>>>>>>>> ot/va
>>>>>>>>>>> riant-server/gensrc/adfiles
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotsp
>>>>>>>>>>> ot/va
>>>>>>>>>>> riant-server/gensrc
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/suppo
>>>>>>>>>>> rt/mo
>>>>>>>>>>> dules_include/java.base
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/suppo
>>>>>>>>>>> rt/mo
>>>>>>>>>>> dules_include/java.base/win32
>>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage
>>>>>>>>>>> -Z7
>>>>>>>>>>> -d2Zi+ -wd4800 -WX
>>>>>>>>>>> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.2
>>>>>>>>>>> 70/at
>>>>>>>>>>> lmfc/include
>>>>>>>>>>> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.2
>>>>>>>>>>> 70/in clude
>>>>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt
>>>>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared
>>>>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um
>>>>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt
>>>>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2
>>>>>>>>>>> -Oy- "-DTHIS_FILE=\"\"" -c
>>>>>>>>>>> -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hots
>>>>>>>>>>> pot/v ariant-server/libjvm/objs/ad_x86_expand.obj
>>>>>>>>>>> /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot
>>>>>>>>>>> /vari ant-server/gensrc/adfiles/ad_x86_expand.cpp<
>>>>>>>>>>> fixpath using wsl mode, with path list:
>>>>>>>>>>> fixpath converted line
>>>>>>>>>>>> c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bi
>>>>>>>>>>>> n/Hos
>>>>>>>>>>> tx86/x64/cl.exe -showIncludes
>>>>>>>>>>> -Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/
>>>>>>>>>>> varia nt-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp
>>>>>>>>>>> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
>>>>>>>>>>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN
>>>>>>>>>>> -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3
>>>>>>>>>>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86
>>>>>>>>>>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86
>>>>>>>>>>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP
>>>>>>>>>>> -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2
>>>>>>>>>>> -DINCLUDE_ZGC=0
>>>>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v
>>>>>>>>>>> arian t-server/gensrc/adfiles
>>>>>>>>>>> -Id:/erik/jdk-wsl/closed/src/hotspot/share
>>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/share
>>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/os/windows
>>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/cpu/x86
>>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86
>>>>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v
>>>>>>>>>>> arian t-server/gensrc
>>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/share/precompiled
>>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/share/include
>>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/os/windows/include
>>>>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/support/m
>>>>>>>>>>> odule
>>>>>>>>>>> s_include/java.base
>>>>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/support/m
>>>>>>>>>>> odule
>>>>>>>>>>> s_include/java.base/win32
>>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7
>>>>>>>>>>> -d2Zi+ -wd4800 -WX
>>>>>>>>>>> -Ic:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/a
>>>>>>>>>>> tlmfc
>>>>>>>>>>> /include
>>>>>>>>>>> -Ic:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/i
>>>>>>>>>>> nclud e -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt
>>>>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared
>>>>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um
>>>>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt
>>>>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy-
>>>>>>>>>>> "-DTHIS_FILE=\"\"" -c
>>>>>>>>>>> -Fod:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/
>>>>>>>>>>> varia nt-server/libjvm/objs/ad_x86_expand.obj
>>>>>>>>>>> d:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/var
>>>>>>>>>>> iant- server/gensrc/adfiles/ad_x86_expand.cpp<
>>>>>>>>>>>
>>>>>>>>>>> An interesting note is that make is rebuilding the pch file 
>>>>>>>>>>> on every invocation so it too has trouble finding the file.
>>>>>>>>>>>
>>>>>>>>>>> /Erik