[PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows

Sat Dec 15 01:23:42 UTC 2018

Hello again,

I took the liberty of creating a bug for this and also a sandbox branch 
where I've applied your latest patch. If you clone that you can send 
further patches based on a known state in the sandbox. This will make it 
easier to see what you are actually doing in each update, as well as 
give us better references when discussing them. It also gives me the 
ability to directly change things so we can keep Cygwin/msys working.

https://bugs.openjdk.java.net/browse/JDK-8215445

http://hg.openjdk.java.net/jdk/sandbox/shortlog/12615de8335e

/Erik

On 2018-12-14 16:47, Erik Joelsson wrote:
> Hello,
>
> You beat me to it. I just found the rc.exe problem was that 
> FIXPATH_PATH in spec.gmk.in was quoted. Make just propagates quotes 
> verbatim, so then fixpath.c would create a path variable like;
>
> $PATH;"$FIXPATH_PATH"
>
> Which is why link.exe could not find rc.exe.
>
> /Erik
>
> On 2018-12-14 16:32, Andrew Luo wrote:
>> Ok, here's my latest patch (I didn't add your case sensitivity fix 
>> yet, but will do next patch).  I believe this should get you past the 
>> rc.exe issues.
>>
>> Thanks,
>>
>> -Andrew
>>
>> -----Original Message-----
>> From: Erik Joelsson <erik.joelsson at oracle.com>
>> Sent: Friday, December 14, 2018 4:15 PM
>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>
>> Cc: Andrew Luo <andrewluotechnologies at outlook.com>; 
>> build-dev at openjdk.java.net
>> Subject: Re: [PATCH] Support for building using WSL (Windows 
>> Subsystem for Linux) on Windows
>>
>>
>> On 2018-12-14 16:06, Magnus Ihse Bursie wrote:
>>>> 14 dec. 2018 kl. 23:42 skrev Erik Joelsson <erik.joelsson at oracle.com>:
>>>>
>>>> I found the reason it's not failing make. It returns "1" and 
>>>> NativeCompilation.gmk currently ignores 1 explicitly for Visual 
>>>> Studio. I added that back in 2014 in 
>>>> https://bugs.openjdk.java.net/browse/JDK-8065576, but I can't 
>>>> figure out why. Nothing mentioned in either comment or review.
>>> Sounds like it's ripe for removal then. :) I wonder what kind of 
>>> issue you might have run into that caused a returned 1 to happen and 
>>> yet we didn't want to consider it a failure...
>> If I'm to guess, I think it's one of the commands we pipe the output 
>> to when the output is zero. This would explain why it was added 
>> together with pipefail.
>>
>> /Erik
>>
>>> /Magnus
>>>
>>>> /Erik
>>>>
>>>>> On 2018-12-14 13:59, Magnus Ihse Bursie wrote:
>>>>>
>>>>>
>>>>>> On 2018-12-14 22:15, Erik Joelsson wrote:
>>>>>> I get the same error for pch and it still continues, but this 
>>>>>> time I let it run until it eventually fails for real when it 
>>>>>> can't link. Perhaps it's simply cl.exe that isn't returning non 
>>>>>> zero for this error? When the linker fails, make fails, so 
>>>>>> propagation doesn't seem broken.
>>>>> That does also seem really weird, considering that it claims it to 
>>>>> be a "fatal error". Can you repeat the command at the command line 
>>>>> and get the failure again, and then check the return value? Can 
>>>>> you rewrite the command line and run it from the devenv prompt? 
>>>>> That is, is there any indication that the pch file itself is 
>>>>> messed up, or can it be used if running the compilation that 
>>>>> should use it from an "ok" prompt?
>>>>>
>>>>> /Magnus
>>>>>> /Erik
>>>>>>
>>>>>>> On 2018-12-14 12:55, Andrew Luo wrote:
>>>>>>> Hmm, I get the rc.exe error as well, but now it is much later 
>>>>>>> down the line... Still investigating...
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> -Andrew
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Andrew Luo
>>>>>>> Sent: Friday, December 14, 2018 12:34 PM
>>>>>>> To: 'Andrew Luo' <andrewluotechnologies at outlook.com>; Magnus Ihse
>>>>>>> Bursie <magnus.ihse.bursie at oracle.com>; Erik Joelsson
>>>>>>> <erik.joelsson at oracle.com>
>>>>>>> Cc: build-dev at openjdk.java.net
>>>>>>> Subject: RE: [PATCH] Support for building using WSL (Windows
>>>>>>> Subsystem for Linux) on Windows
>>>>>>>
>>>>>>> Try this updated patch with some fixes...
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> -Andrew
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: build-dev <build-dev-bounces at openjdk.java.net> On Behalf Of
>>>>>>> Andrew Luo
>>>>>>> Sent: Friday, December 14, 2018 12:01 PM
>>>>>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; Erik
>>>>>>> Joelsson <erik.joelsson at oracle.com>
>>>>>>> Cc: build-dev at openjdk.java.net
>>>>>>> Subject: RE: [PATCH] Support for building using WSL (Windows
>>>>>>> Subsystem for Linux) on Windows
>>>>>>>
>>>>>>> I think I have a fix for it.  Give me a minute (or a few hours 
>>>>>>> depending on if it works).
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> -Andrew
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>
>>>>>>> Sent: Friday, December 14, 2018 11:42 AM
>>>>>>> To: Erik Joelsson <erik.joelsson at oracle.com>
>>>>>>> Cc: Andrew Luo <andrewluotechnologies at outlook.com>;
>>>>>>> build-dev at openjdk.java.net
>>>>>>> Subject: Re: [PATCH] Support for building using WSL (Windows
>>>>>>> Subsystem for Linux) on Windows
>>>>>>>
>>>>>>>
>>>>>>>> 14 dec. 2018 kl. 20:31 skrev Erik Joelsson 
>>>>>>>> <erik.joelsson at oracle.com>:
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 2018-12-14 11:05, Magnus Ihse Bursie wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> On 2018-12-14 19:41, Erik Joelsson wrote:
>>>>>>>>>>>
>>>>>>>>>>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On 2018-12-14 19:23, Erik Joelsson wrote:
>>>>>>>>>>>> Hello,
>>>>>>>>>>>>
>>>>>>>>>>>> I took your patch for a spin, and configure passes, but I 
>>>>>>>>>>>> get the same build error I got with my patch:
>>>>>>>>>>>>
>>>>>>>>>>>> fatal error C1083: Cannot open compiler intermediate file:
>>>>>>>>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\
>>>>>>>>>>>> varia
>>>>>>>>>>>> nt-server\libjvm\objs\build_libjvm.pch': No such file or
>>>>>>>>>>>> directory
>>>>>>>>>>>>
>>>>>>>>>>>> This is repeated for every C++ file in Hotspot. I see two 
>>>>>>>>>>>> issues here. First of all, I need to figure out why the 
>>>>>>>>>>>> compiler will not find the file, which is clearly there. 
>>>>>>>>>>>> Second, why isn't this failure picked up by make? Somewhere 
>>>>>>>>>>>> the return value of cl.exe is disappearing.
>>>>>>>>>>> Can you build without errors if you disable PCH?
>>>>>>>>> Could you? That is, is it only the PCH that is problematic?
>>>>>>>> Trying that now.
>>>>>>>>>>> Also, a wild guess: can it be related to file permissions? 
>>>>>>>>>>> Can you read the file properly from both WSL and Windows?
>>>>>>>>>> It is readable, but it could be something with case. The file 
>>>>>>>>>> is actually called BUILD_LIBJVM.pch, but that is also how 
>>>>>>>>>> it's given to the compiler command line. Here is the output 
>>>>>>>>>> from DEBUG_FIXPATH:
>>>>>>>>> Weird. What if you, after a failed build, rename it to 
>>>>>>>>> build_libjvm.pch?
>>>>>>>> Doing that causes a new error:
>>>>>>>>
>>>>>>>> d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. 
>>>>>>>>
>>>>>>>> cpp : fatal error C1382: the PCH file
>>>>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\vari
>>>>>>>> ant-s erver\libjvm\objs\build_libjvm.pch' has been rebuilt since
>>>>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\vari
>>>>>>>> ant-s erver\libjvm\objs\accessBarrierSupport.obj' was generated.
>>>>>>>> Please rebuild this object
>>>>>>>>
>>>>>>>> But I think even more important is that make is not getting the 
>>>>>>>> error. The build just continues until interrupted.
>>>>>>> Agree, that's bad.
>>>>>>>
>>>>>>> Does fixpath_debug print exit code? If so, what does it say? If 
>>>>>>> not, we should add that instrumentation.
>>>>>>>
>>>>>>> /Magnus
>>>>>>>
>>>>>>>>>> Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line
>>>>>>>>>>> -wsl\build\windows-x86_64-server-release\configure-support\bin
>>>>>>>>>>> \fixp
>>>>>>>>>> ath.exe -w
>>>>>>>>> This starts out quite odd..? -wsl\build\...?
>>>>>>>> I agree, didn't look into that part.
>>>>>>>>>> /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270
>>>>>>>>>> /bin/
>>>>>>>>>> Hostx86/x64/cl.exe
>>>>>>>>> Also, FWIW, this seems not to have been properly case treated. 
>>>>>>>>> Which version of the patch are you using?
>>>>>>>> The last one posted by Andrew: "diff15.txt".
>>>>>>>>
>>>>>>>> /Erik
>>>>>>>>
>>>>>>>>> /Magnus
>>>>>>>>>> -showIncludes
>>>>>>>>>> -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hots
>>>>>>>>>> pot/v ariant-server/libjvm/objs/BUILD_LIBJVM.pch
>>>>>>>>>> -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
>>>>>>>>>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN
>>>>>>>>>> -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3
>>>>>>>>>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86
>>>>>>>>>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86
>>>>>>>>>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP
>>>>>>>>>> -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2
>>>>>>>>>> -DINCLUDE_ZGC=0
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotsp
>>>>>>>>>> ot/va
>>>>>>>>>> riant-server/gensrc/adfiles
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotsp
>>>>>>>>>> ot/va
>>>>>>>>>> riant-server/gensrc
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/suppo
>>>>>>>>>> rt/mo
>>>>>>>>>> dules_include/java.base
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/suppo
>>>>>>>>>> rt/mo
>>>>>>>>>> dules_include/java.base/win32
>>>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage
>>>>>>>>>> -Z7
>>>>>>>>>> -d2Zi+ -wd4800 -WX
>>>>>>>>>> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.2
>>>>>>>>>> 70/at
>>>>>>>>>> lmfc/include
>>>>>>>>>> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.2
>>>>>>>>>> 70/in clude
>>>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt
>>>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared
>>>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um
>>>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt
>>>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2
>>>>>>>>>> -Oy- "-DTHIS_FILE=\"\"" -c
>>>>>>>>>> -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hots
>>>>>>>>>> pot/v ariant-server/libjvm/objs/ad_x86_expand.obj
>>>>>>>>>> /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot
>>>>>>>>>> /vari ant-server/gensrc/adfiles/ad_x86_expand.cpp<
>>>>>>>>>> fixpath using wsl mode, with path list:
>>>>>>>>>> fixpath converted line
>>>>>>>>>>> c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bi
>>>>>>>>>>> n/Hos
>>>>>>>>>> tx86/x64/cl.exe -showIncludes
>>>>>>>>>> -Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/
>>>>>>>>>> varia nt-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp
>>>>>>>>>> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
>>>>>>>>>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN
>>>>>>>>>> -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3
>>>>>>>>>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86
>>>>>>>>>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86
>>>>>>>>>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP
>>>>>>>>>> -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2
>>>>>>>>>> -DINCLUDE_ZGC=0
>>>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v
>>>>>>>>>> arian t-server/gensrc/adfiles
>>>>>>>>>> -Id:/erik/jdk-wsl/closed/src/hotspot/share
>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/share
>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/os/windows
>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/cpu/x86
>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86
>>>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v
>>>>>>>>>> arian t-server/gensrc
>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/share/precompiled
>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/share/include
>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/os/windows/include
>>>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/support/m
>>>>>>>>>> odule
>>>>>>>>>> s_include/java.base
>>>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/support/m
>>>>>>>>>> odule
>>>>>>>>>> s_include/java.base/win32
>>>>>>>>>> -Id:/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7
>>>>>>>>>> -d2Zi+ -wd4800 -WX
>>>>>>>>>> -Ic:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/a
>>>>>>>>>> tlmfc
>>>>>>>>>> /include
>>>>>>>>>> -Ic:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/i
>>>>>>>>>> nclud e -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt
>>>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared
>>>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um
>>>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt
>>>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy-
>>>>>>>>>> "-DTHIS_FILE=\"\"" -c
>>>>>>>>>> -Fod:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/
>>>>>>>>>> varia nt-server/libjvm/objs/ad_x86_expand.obj
>>>>>>>>>> d:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/var
>>>>>>>>>> iant- server/gensrc/adfiles/ad_x86_expand.cpp<
>>>>>>>>>>
>>>>>>>>>> An interesting note is that make is rebuilding the pch file 
>>>>>>>>>> on every invocation so it too has trouble finding the file.
>>>>>>>>>>
>>>>>>>>>> /Erik