[PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows

Sat Dec 15 00:14:40 UTC 2018

On 2018-12-14 16:06, Magnus Ihse Bursie wrote:
>> 14 dec. 2018 kl. 23:42 skrev Erik Joelsson <erik.joelsson at oracle.com>:
>>
>> I found the reason it's not failing make. It returns "1" and NativeCompilation.gmk currently ignores 1 explicitly for Visual Studio. I added that back in 2014 in https://bugs.openjdk.java.net/browse/JDK-8065576, but I can't figure out why. Nothing mentioned in either comment or review.
> Sounds like it's ripe for removal then. :) I wonder what kind of issue you might have run into that caused a returned 1 to happen and yet we didn't want to consider it a failure...

If I'm to guess, I think it's one of the commands we pipe the output to 
when the output is zero. This would explain why it was added together 
with pipefail.

/Erik

> /Magnus
>
>> /Erik
>>
>>> On 2018-12-14 13:59, Magnus Ihse Bursie wrote:
>>>
>>>
>>>> On 2018-12-14 22:15, Erik Joelsson wrote:
>>>> I get the same error for pch and it still continues, but this time I let it run until it eventually fails for real when it can't link. Perhaps it's simply cl.exe that isn't returning non zero for this error? When the linker fails, make fails, so propagation doesn't seem broken.
>>> That does also seem really weird, considering that it claims it to be a "fatal error". Can you repeat the command at the command line and get the failure again, and then check the return value? Can you rewrite the command line and run it from the devenv prompt? That is, is there any indication that the pch file itself is messed up, or can it be used if running the compilation that should use it from an "ok" prompt?
>>>
>>> /Magnus
>>>> /Erik
>>>>
>>>>> On 2018-12-14 12:55, Andrew Luo wrote:
>>>>> Hmm, I get the rc.exe error as well, but now it is much later down the line... Still investigating...
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Andrew
>>>>>
>>>>> -----Original Message-----
>>>>> From: Andrew Luo
>>>>> Sent: Friday, December 14, 2018 12:34 PM
>>>>> To: 'Andrew Luo' <andrewluotechnologies at outlook.com>; Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; Erik Joelsson <erik.joelsson at oracle.com>
>>>>> Cc: build-dev at openjdk.java.net
>>>>> Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
>>>>>
>>>>> Try this updated patch with some fixes...
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Andrew
>>>>>
>>>>> -----Original Message-----
>>>>> From: build-dev <build-dev-bounces at openjdk.java.net> On Behalf Of Andrew Luo
>>>>> Sent: Friday, December 14, 2018 12:01 PM
>>>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; Erik Joelsson <erik.joelsson at oracle.com>
>>>>> Cc: build-dev at openjdk.java.net
>>>>> Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
>>>>>
>>>>> I think I have a fix for it.  Give me a minute (or a few hours depending on if it works).
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Andrew
>>>>>
>>>>> -----Original Message-----
>>>>> From: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>
>>>>> Sent: Friday, December 14, 2018 11:42 AM
>>>>> To: Erik Joelsson <erik.joelsson at oracle.com>
>>>>> Cc: Andrew Luo <andrewluotechnologies at outlook.com>; build-dev at openjdk.java.net
>>>>> Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
>>>>>
>>>>>
>>>>>> 14 dec. 2018 kl. 20:31 skrev Erik Joelsson <erik.joelsson at oracle.com>:
>>>>>>
>>>>>>
>>>>>>> On 2018-12-14 11:05, Magnus Ihse Bursie wrote:
>>>>>>>
>>>>>>>
>>>>>>>>> On 2018-12-14 19:41, Erik Joelsson wrote:
>>>>>>>>>
>>>>>>>>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 2018-12-14 19:23, Erik Joelsson wrote:
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I took your patch for a spin, and configure passes, but I get the same build error I got with my patch:
>>>>>>>>>>
>>>>>>>>>> fatal error C1083: Cannot open compiler intermediate file:
>>>>>>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\varia
>>>>>>>>>> nt-server\libjvm\objs\build_libjvm.pch': No such file or directory
>>>>>>>>>>
>>>>>>>>>> This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing.
>>>>>>>>> Can you build without errors if you disable PCH?
>>>>>>> Could you? That is, is it only the PCH that is problematic?
>>>>>> Trying that now.
>>>>>>>>> Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows?
>>>>>>>> It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH:
>>>>>>> Weird. What if you, after a failed build, rename it to build_libjvm.pch?
>>>>>> Doing that causes a new error:
>>>>>>
>>>>>> d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport.
>>>>>> cpp : fatal error C1382: the PCH file
>>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s
>>>>>> erver\libjvm\objs\build_libjvm.pch' has been rebuilt since
>>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s
>>>>>> erver\libjvm\objs\accessBarrierSupport.obj' was generated. Please
>>>>>> rebuild this object
>>>>>>
>>>>>> But I think even more important is that make is not getting the error. The build just continues until interrupted.
>>>>> Agree, that's bad.
>>>>>
>>>>> Does fixpath_debug print exit code? If so, what does it say? If not, we should add that instrumentation.
>>>>>
>>>>> /Magnus
>>>>>
>>>>>>>> Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line
>>>>>>>>> -wsl\build\windows-x86_64-server-release\configure-support\bin\fixp
>>>>>>>> ath.exe -w
>>>>>>> This starts out quite odd..? -wsl\build\...?
>>>>>> I agree, didn't look into that part.
>>>>>>>> /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/
>>>>>>>> Hostx86/x64/cl.exe
>>>>>>> Also, FWIW, this seems not to have been properly case treated. Which version of the patch are you using?
>>>>>> The last one posted by Andrew: "diff15.txt".
>>>>>>
>>>>>> /Erik
>>>>>>
>>>>>>> /Magnus
>>>>>>>> -showIncludes
>>>>>>>> -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v
>>>>>>>> ariant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp
>>>>>>>> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
>>>>>>>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo
>>>>>>>> -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3
>>>>>>>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86
>>>>>>>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86
>>>>>>>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64
>>>>>>>> "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2
>>>>>>>> -DINCLUDE_ZGC=0
>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/va
>>>>>>>> riant-server/gensrc/adfiles
>>>>>>>> -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share
>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share
>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows
>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86
>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86
>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/va
>>>>>>>> riant-server/gensrc
>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled
>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include
>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include
>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/mo
>>>>>>>> dules_include/java.base
>>>>>>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/mo
>>>>>>>> dules_include/java.base/win32
>>>>>>>> -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7
>>>>>>>> -d2Zi+ -wd4800 -WX
>>>>>>>> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/at
>>>>>>>> lmfc/include
>>>>>>>> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/in
>>>>>>>> clude -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt
>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared
>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um
>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt
>>>>>>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy-
>>>>>>>> "-DTHIS_FILE=\"\"" -c
>>>>>>>> -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v
>>>>>>>> ariant-server/libjvm/objs/ad_x86_expand.obj
>>>>>>>> /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/vari
>>>>>>>> ant-server/gensrc/adfiles/ad_x86_expand.cpp<
>>>>>>>> fixpath using wsl mode, with path list:
>>>>>>>> fixpath converted line
>>>>>>>>> c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hos
>>>>>>>> tx86/x64/cl.exe -showIncludes
>>>>>>>> -Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/varia
>>>>>>>> nt-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp
>>>>>>>> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
>>>>>>>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo
>>>>>>>> -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3
>>>>>>>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86
>>>>>>>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86
>>>>>>>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64
>>>>>>>> "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2
>>>>>>>> -DINCLUDE_ZGC=0
>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/varian
>>>>>>>> t-server/gensrc/adfiles -Id:/erik/jdk-wsl/closed/src/hotspot/share
>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/share
>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/os/windows
>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/cpu/x86
>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86
>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/varian
>>>>>>>> t-server/gensrc -Id:/erik/jdk-wsl/open/src/hotspot/share/precompiled
>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/share/include
>>>>>>>> -Id:/erik/jdk-wsl/open/src/hotspot/os/windows/include
>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/support/module
>>>>>>>> s_include/java.base
>>>>>>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/support/module
>>>>>>>> s_include/java.base/win32
>>>>>>>> -Id:/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7
>>>>>>>> -d2Zi+ -wd4800 -WX
>>>>>>>> -Ic:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc
>>>>>>>> /include
>>>>>>>> -Ic:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/includ
>>>>>>>> e -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt
>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared
>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um
>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt
>>>>>>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy-
>>>>>>>> "-DTHIS_FILE=\"\"" -c
>>>>>>>> -Fod:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/varia
>>>>>>>> nt-server/libjvm/objs/ad_x86_expand.obj
>>>>>>>> d:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-
>>>>>>>> server/gensrc/adfiles/ad_x86_expand.cpp<
>>>>>>>>
>>>>>>>> An interesting note is that make is rebuilding the pch file on every invocation so it too has trouble finding the file.
>>>>>>>>
>>>>>>>> /Erik