[PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows

Erik Joelsson erik.joelsson at oracle.com
Fri Dec 14 23:41:33 UTC 2018 

And now I've read up on case sensitivity in WSL. 
https://blogs.msdn.microsoft.com/commandline/2018/02/28/per-directory-case-sensitivity-and-wsl/

The build will certainly not work on Windows with case sensitive build 
directories.

I managed to fix this by modifying the MakeDir macro like this:

################################################################################
# Make directory without forking mkdir if not needed.
#
# If a directory with an encoded space is provided, the wildcard function
# sometimes returns false answers (typically if the dir existed when the
# makefile was parsed, but was deleted by a previous rule). In that 
case, always
# call mkdir regardless of what wildcard says.
#
# On Windows WSL, force new dirs to be case insensitive to stay 
compatible with
# Windows native tools used in the build.
#
# 1: List of directories to create
MakeDir = \
     $(strip \
         $(eval MakeDir_dirs_to_make := $(strip $(foreach d, $1, \
           $(if $(findstring ?, $d), '$(call DecodeSpace, $d)', \
             $(if $(wildcard $d), , $d) \
           ) \
         ))) \
         $(if $(MakeDir_dirs_to_make), \
           $(shell $(MKDIR) -p $(MakeDir_dirs_to_make)) \
           $(if $(call equals, $(OPENJDK_TARGET_OS_ENV), windows.wsl), \
              $(foreach d, $(MakeDir_dirs_to_make), \
                 $(shell $(FSUTIL) file setCaseSensitiveInfo `$(WSLPATH) 
-w $d` disable > /dev/null) \
              ) \
           ) \
         ) \
     )

I also had to add FSUTIL to basics.m4 as well as add WSLPATH and FSUTIL 
to spec.gmk.in.

Now I'm hitting the same rc.exe problem with PCH enabled as before without.

/Erik

On 2018-12-14 15:11, Erik Joelsson wrote:
> On 2018-12-14 11:33, Erik Joelsson wrote:
>> On 2018-12-14 11:05, Andrew Luo wrote:
>>> Odd, it builds fine on my system.  Did you sync down the code on 
>>> Windows or WSL, and to a Windows or WSL directory?  My code actually 
>>> lives in Windows under /mnt/c/...
>> Yes, otherwise it wouldn't have worked at all since Windows can't 
>> reach the WSL paths. The src was cloned in Cygwin originally.
>>> I believe there is a difference (regarding case sensitivity) 
>>> depending on if you are on a Windows filesystem or a WSL filesystem.
>>
>> I don't think this is really about case sensitivity, but it could be 
>> a symptom.
>>
> It does seem to be about being case sensitive. I extracted a failing 
> command line and pasted into a VS env CMD window, it reproduces. I 
> then tried to build in Cygwin, into a different output directory (this 
> failed for other reasons later, but I got far enough that I had some 
> object files). If I changed the compile command to use the pch-file 
> from the Cygwin based build, the command succeeded. One notable 
> difference between these files, the file from the Cygwin build is 
> accessible using both upper and lower case name, while the one from 
> WSL is not.
>
> File from WSL build:
>
> ---
>
> D:\>dir 
> d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\BUILD_LIBJVM.pch
>  Volume in drive D is Work
>  Volume Serial Number is 4ED4-C471
>
>  Directory of 
> d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs
>
> 2018-12-14  12:58        36,634,624 BUILD_LIBJVM.pch
>                1 File(s)     36,634,624 bytes
>                0 Dir(s)  192,267,493,376 bytes free
>
> D:\>dir 
> d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch
>  Volume in drive D is Work
>  Volume Serial Number is 4ED4-C471
>
>  Directory of 
> d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs
>
> File Not Found
>
> ---
>
>
> File from Cygwin build:
>
> ---
>
> D:\>dir /x 
> d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs\BUILD_LIBJVM.pch
>  Volume in drive D is Work
>  Volume Serial Number is 4ED4-C471
>
>  Directory of 
> d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs
>
> 2018-12-14  14:41        36,634,624              BUILD_LIBJVM.pch
>                1 File(s)     36,634,624 bytes
>                0 Dir(s)  192,267,493,376 bytes free
>
> D:\>dir /x 
> d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs\build_libjvm.pch
>  Volume in drive D is Work
>  Volume Serial Number is 4ED4-C471
>
>  Directory of 
> d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs
>
> 2018-12-14  14:41        36,634,624              BUILD_LIBJVM.pch
>                1 File(s)     36,634,624 bytes
>                0 Dir(s)  192,267,493,376 bytes free
>
> ---
>
> /Erik
>>
>>> Thanks,
>>>
>>> -Andrew
>>>
>>> -----Original Message-----
>>> From: Erik Joelsson <erik.joelsson at oracle.com>
>>> Sent: Friday, December 14, 2018 10:42 AM
>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; Andrew Luo 
>>> <andrewluotechnologies at outlook.com>
>>> Cc: build-dev at openjdk.java.net
>>> Subject: Re: [PATCH] Support for building using WSL (Windows 
>>> Subsystem for Linux) on Windows
>>>
>>>
>>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote:
>>>>
>>>> On 2018-12-14 19:23, Erik Joelsson wrote:
>>>>> Hello,
>>>>>
>>>>> I took your patch for a spin, and configure passes, but I get the
>>>>> same build error I got with my patch:
>>>>>
>>>>> fatal error C1083: Cannot open compiler intermediate file:
>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': 
>>>>>
>>>>> No such file or directory
>>>>>
>>>>> This is repeated for every C++ file in Hotspot. I see two issues
>>>>> here. First of all, I need to figure out why the compiler will not
>>>>> find the file, which is clearly there. Second, why isn't this failure
>>>>> picked up by make? Somewhere the return value of cl.exe is 
>>>>> disappearing.
>>>> Can you build without errors if you disable PCH?
>>>>
>>>> Also, a wild guess: can it be related to file permissions? Can you
>>>> read the file properly from both WSL and Windows?
>>>>
>>> It is readable, but it could be something with case. The file is 
>>> actually called BUILD_LIBJVM.pch, but that is also how it's given to 
>>> the compiler command line. Here is the output from DEBUG_FIXPATH:
>>>
>>> Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line 
>>> >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe 
>>> -w 
>>> /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe 
>>> -showIncludes 
>>> -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch 
>>> -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS 
>>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo 
>>> -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 
>>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 
>>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 
>>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 
>>> "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 
>>> -DINCLUDE_ZGC=0 
>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles 
>>> -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share 
>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share 
>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows 
>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 
>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 
>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc 
>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled 
>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include 
>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include 
>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base 
>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 
>>> -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 
>>> -d2Zi+ -wd4800 -WX 
>>> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include 
>>> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include 
>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt 
>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared 
>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um 
>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt 
>>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy- 
>>> "-DTHIS_FILE=\"\"" -c 
>>> -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/ad_x86_expand.obj 
>>> /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles/ad_x86_expand.cpp< 
>>>
>>> fixpath using wsl mode, with path list:
>>> fixpath converted line
>>> >c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe 
>>> -showIncludes 
>>> -Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch 
>>> -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS 
>>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo 
>>> -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 
>>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 
>>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 
>>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 
>>> "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 
>>> -DINCLUDE_ZGC=0 
>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles 
>>> -Id:/erik/jdk-wsl/closed/src/hotspot/share 
>>> -Id:/erik/jdk-wsl/open/src/hotspot/share 
>>> -Id:/erik/jdk-wsl/open/src/hotspot/os/windows 
>>> -Id:/erik/jdk-wsl/open/src/hotspot/cpu/x86 
>>> -Id:/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 
>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc 
>>> -Id:/erik/jdk-wsl/open/src/hotspot/share/precompiled 
>>> -Id:/erik/jdk-wsl/open/src/hotspot/share/include 
>>> -Id:/erik/jdk-wsl/open/src/hotspot/os/windows/include 
>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base 
>>> -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 
>>> -Id:/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 
>>> -d2Zi+ -wd4800 -WX 
>>> -Ic:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include 
>>> -Ic:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include 
>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt 
>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared 
>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um 
>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt 
>>> -Ic:/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy- 
>>> "-DTHIS_FILE=\"\"" -c 
>>> -Fod:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/ad_x86_expand.obj 
>>> d:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles/ad_x86_expand.cpp< 
>>>
>>>
>>> An interesting note is that make is rebuilding the pch file on every 
>>> invocation so it too has trouble finding the file.
>>>
>>> /Erik
>>>
>>

More information about the build-dev mailing list