[foreign] Running tests on Windows

Thu Sep 27 18:15:29 UTC 2018

On 2018-09-27 17:55, Maurizio Cimadamore wrote:
> NVM,
> I think the real issue is how AC_CHECK_LIB invokes the compiler - e.g. 
> with -lclang
>
> In MSVC this should be /link libclang.lib
>
> Not sure how you tell AC_CHECK_LIB to use a different command, but 
> frankly it seems like AC_XYZ checks are GNU-centric and they don't 
> play well with Windows - it might be a lost cause to try and make them 
> work under cygwin.
That might very well be the case. We have not used AC_CHECK_LIB much (at 
all?), and I vaguely remember that we've tried to actually stay clear of 
AC_CHECK_LIB because of issues; that might be that they do not work on 
Microsoft.

For other external libraries on Windows, like freetype, if I remember 
correctly we have just provided the absolute path to the .LIB file as 
*_LIBS. So instead of the unix standard of using a LD_FLAGS -L<path> and 
LIBS -l<lib>, we just use <path>/<lib>.lib as LIBS.

/Magnus

>
> Maurizio
>
>
> On 27/09/18 16:11, Maurizio Cimadamore wrote:
>> Btw, the issue in my case seems to be cause by the fact that cl.exe 
>> cant locate <time.h> (which is included by clang-c/Index.h header)
>>
>> Maurizio
>>
>>
>> On 27/09/18 15:58, Maurizio Cimadamore wrote:
>>> Got the same on my windows box. I also replaced Magnus's CC 
>>> suggestion with CXX which got me a bit further.
>>>
>>> Maurizio
>>>
>>>
>>>
>>> On 27/09/18 15:37, Jorn Vernee wrote:
>>>> AC_CHECK_HEADER seems to use CXX instead of CC. I wasn't seeing the 
>>>> fixpath.exe call in config.log with `CC = "$FIXPATH $CC"`, but it 
>>>> is there when using `CXX="$FIXPATH $CXX"".
>>>>
>>>> Checking the header now works, but it's breaking when using 
>>>> `AC_CHECK_LIB`:
>>>>
>>>> configure:60545: checking for clang_getClangVersion in -lclang
>>>> configure:60570: 
>>>> /cygdrive/j/cygwin64/home/Jorn/cygwin-projects/panama/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe 
>>>> -c 
>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>> -o conftest.exe    -I/cygdrive/j/LLVM/include 
>>>> -L/cygdrive/j/LLVM/lib conftest.cpp -lclang   >&5
>>>> conftest.cpp
>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>
>>>> /out:conftest.exe
>>>> /out:conftest.exe
>>>> conftest.obj
>>>> conftest.obj : error LNK2019: unresolved external symbol 
>>>> clang_getClangVersion referenced in function main
>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>
>>>> cl : Command line warning D9035 : option 'o' has been deprecated 
>>>> and will be removed in a future release
>>>> cl : Command line warning D9002 : ignoring unknown option 
>>>> '-Lj:/LLVM/lib'
>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>> configure:60570: $? = 2
>>>>
>>>> I'm also seeing the following warnings:
>>>>
>>>> The following warnings were produced. Repeated here for convenience:
>>>> WARNING: clang-c/Index.h: accepted by the compiler, rejected by the 
>>>> preprocessor!
>>>> WARNING: clang-c/Index.h: proceeding with the compiler's result
>>>>
>>>> FWIW Not sure if the lib check can work with AC_CHECK_LIB, which 
>>>> seems to add the `-l` on it's own (note the unrecognized option 
>>>> warnings in the log). For the MS linker AFAIK you'd need 
>>>> `/DEFAULTLIB:clang`, or to pass a .lib or .pdb file on the command 
>>>> line instead.
>>>>
>>>> Jorn
>>>>
>>>> [1] : 
>>>> https://docs.microsoft.com/en-us/cpp/build/reference/linker-options?view=vs-2017
>>>>
>>>> Maurizio Cimadamore schreef op 2018-09-27 15:41:
>>>>> On 27/09/18 14:35, Magnus Ihse Bursie wrote:
>>>>>
>>>>>> On 2018-09-25 01:38, Maurizio Cimadamore wrote:
>>>>>>
>>>>>> On 20/09/18 10:29, Magnus Ihse Bursie wrote:
>>>>>>
>>>>>> On 2018-09-20 11:02, Maurizio Cimadamore wrote:
>>>>>> Try this:
>>>>>>
>>>>>> diff -r d5dbb455a6b1 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4    Tue Sep 11 18:39:11 2018 +0100
>>>>>> +++ b/make/autoconf/lib-clang.m4    Thu Sep 20 09:59:08 2018 +0100
>>>>>> @@ -60,6 +60,9 @@
>>>>>> VER=`ls $with_libclang/lib/clang/`
>>>>>> CLANG_INCLUDE_AUX_PATH="$with_libclang/lib/clang/$VER/include"
>>>>>> CLANG_LIB_PATH="$with_libclang/lib"
>>>>>> +      BASIC_FIXUP_PATH([CLANG_INCLUDE_PATH])
>>>>>> +      BASIC_FIXUP_PATH([CLANG_LIB_PATH])
>>>>>> +      BASIC_FIXUP_PATH([CLANG_INCLUDE_AUX_PATH])
>>>>>> fi
>>>>>>
>>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>>
>>>>>> Now, if you use only the --with-libclang option with a Unix-style
>>>>>> path (the thing you were trying at first), I believe it should do
>>>>>> the right thing.
>>>>>>
>>>>>> I have not followed the entire conversation, but this part looks
>>>>>> sane. As Maurizio says, we should use BASIC_FIXUP_PATH on all paths
>>>>>> from configure argument. (And these should, yes indeed, be in unix
>>>>>> style). However, this might not be all fixes that are needed. I can
>>>>>> help take a look at it, if someone points me to where the
>>>>>> problematic code resides.
>>>>>  Magnus, I followed up a bit on this, and managed to reproduce the
>>>>> issue.
>>>>>
>>>>> The underlying issue is that our lib-clang.m4 conf file checks for
>>>>> headers in the usual way with AC_CHECK_HEADER. This check relies on
>>>>> setting CPPFLAGS and LDFLAGS accordingly. Now, even with the above
>>>>> fixups, what you end up with is this:
>>>>>
>>>>> CLANG_INCLUDE_PATH=/cygdrive/c/progra~1/llvm/include
>>>>>
>>>>> This is the correct _cygwin_ path, but it's not the correct Windows
>>>>> path. In fact, if you try to compile a dummy header that contains 
>>>>> this
>>>>> (this was also observed by Jorn)
>>>>>
>>>>> #include <clang-c/Index.h>
>>>>>
>>>>> with a -I option that includes the above path using the cl.exe
>>>>> compiler within cygwin, you get an error because the include file
>>>>> cannot be found. The only solution is to use a Windows path, which in
>>>>> cygwin can only be done using double backslashes (ugh).
>>>>>
>>>>> Now, it seems that BASIC_FIXUP_PATH turns windows paths into unix
>>>>> path, while here we need the opposite operation. I was under the
>>>>> impression that the build configure framework needed to solve this
>>>>> particular issue anyway, to check for other headers, but I realized
>>>>> that all calls to AC_CHECK_HEADER seems to be Unix-style only. So, is
>>>>> AC_CHECK_HEADER even supposed to work under Windows? If not, what
>>>>> should we use in order to make our configure script portable?
>>>>>
>>>>> P.S.
>>>>> I note that there's also a FIXPATH script which does exactly what we
>>>>> need here - but this script seems just to be set by the configure 
>>>>> step
>>>>> and is probably only used by the proper build, so I don't think we 
>>>>> can
>>>>> refer to it from here?
>>>>>
>>>>> Hi,
>>>>>
>>>>> Sorry for not having had time to look at this yet.
>>>>>
>>>>> The solution should be to use FIXPATH at all times when compiling on
>>>>> Windows if using local paths, even in the configure script. It might
>>>>> be the case that we have not had to do this as of yet, so it might 
>>>>> not
>>>>> work out of the box. :-( But I thought we set up CC to "$FIXPATH $CC"
>>>>> on Windows, so I thought it should work by itself.
>>>>>
>>>>> To get around the mess of windows and unix style paths, the decision
>>>>> was made to use only unix paths *all the time*. The duct tape needed
>>>>> to get this to work is the fixpath launcher, which takes a
>>>>> command-line with unix paths and converts it to windows paths at the
>>>>> very last moment before launching the executable.
>>>>>
>>>>> Does things work better if you to:
>>>>> CC="$FIXPATH $CC"
>>>>> before your AC_CHECK_HEADER?
>>>>>
>>>>> I'll give this a try - thanks
>>>>>
>>>>> Maurizio
>>>>>
>>>>>> /Magnus
>>>>>>
>>>>>> Thanks
>>>>>> Maurizio
>>>>>> /Magnus
>>>>>>
>>>>>> Note that we call this BASIC_FIXUP_PATH thingie on all incoming
>>>>>> paths which can possibly contain forward slashes - e.g.
>>>>>>
>>>>>>
>>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/autoconf/source-dirs.m4#l49 
>>>>>
>>>>>>
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> On 20/09/18 09:44, Maurizio Cimadamore wrote:
>>>>>> FTR, as I'm looking at other configure files, I've seen
>>>>>>
>>>>>> "xwindows" used not "xmicrosoft"
>>>>>>
>>>>>> As for the need for double backslash, I don't see any special
>>>>>> processing done in other configure files prior to the
>>>>>> AC_CHECK_HEADER call.
>>>>>>
>>>>>> The build guide [1] is very explicit that forward slashes should be
>>>>>> used in options (unlike what you did) and mentions some 'fixpath'
>>>>>> tool which is used by configure to convert Unix paths into Windows
>>>>>> ones. I wonder if this could be a bug in that tool?
>>>>>>
>>>>>> Seems like this tool is compiled in make/autoconf/basic_windows.m4 -
>>>>>> it could be worth chasing down as to where the compiled 'fixpath'
>>>>>> executable is put (should be somewhere in
>>>>>> build/<conf>/configure-support/) call it with the clang include
>>>>>> folder with Unix-style forward slash and see whether it spits the
>>>>>> correct path.
>>>>>>
>>>>>> The source code for this tool can be found here:
>>>>>>
>>>>>>
>>>>> http://hg.openjdk.java.net/jdk/jdk/file/43668e3cae4d/make/src/native/fixpath.c 
>>>>>
>>>>>>
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> [1] -
>>>>>>
>>>>> http://hg.openjdk.java.net/jdk/jdk/raw-file/43668e3cae4d/doc/building.html#windows 
>>>>>
>>>>>>
>>>>>> On 20/09/18 00:41, Henry Jen wrote:
>>>>>> Actually, -I works, and link option need to be passed with /link,
>>>>>> but you got the idea…
>>>>>>
>>>>>> However, -lclang is added by the AC_CHECK_LIB macro, so I am not
>>>>>> sure it would work. Need to find a better way to check the lib.
>>>>>>
>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018
>>>>>> -0700
>>>>>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:38:06 2018
>>>>>> -0700
>>>>>> @@ -68,7 +68,11 @@
>>>>>> LIBCLANG_CPPFLAGS=""
>>>>>> fi
>>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>> +      else
>>>>>> +        LIBCLANG_LDFLAGS="/link /LIBPATH $CLANG_LIB_PATH"
>>>>>> +      fi
>>>>>> else
>>>>>> LIBCLANG_LDFLAGS=""
>>>>>> fi
>>>>>>
>>>>>> Cheers,
>>>>>> Henry
>>>>>>
>>>>>> On Sep 19, 2018, at 4:17 PM, Henry Jen <henry.jen at oracle.com> wrote:
>>>>>>
>>>>>>
>>>>>> Haven’t test it, but try this,
>>>>>>
>>>>>> diff -r 3fedd3afcd98 make/autoconf/lib-clang.m4
>>>>>> --- a/make/autoconf/lib-clang.m4        Mon Sep 17 22:17:08 2018
>>>>>> -0700
>>>>>> +++ b/make/autoconf/lib-clang.m4        Wed Sep 19 16:16:27 2018
>>>>>> -0700
>>>>>> @@ -63,12 +63,20 @@
>>>>>> fi
>>>>>>
>>>>>> if test "x$CLANG_INCLUDE_PATH" != "x"; then
>>>>>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> LIBCLANG_CPPFLAGS="-I$CLANG_INCLUDE_PATH"
>>>>>> +      else
>>>>>> +        LIBCLANG_CPPFLAGS="/I $CLANG_INCLUDE_PATH"
>>>>>> +      fi
>>>>>> else
>>>>>> LIBCLANG_CPPFLAGS=""
>>>>>> fi
>>>>>> if test "x$CLANG_LIB_PATH" != "x"; then
>>>>>> +      if test "x$TOOLCHAIN_TYPE" != xmicrosoft; then
>>>>>> LIBCLANG_LDFLAGS="-L$CLANG_LIB_PATH"
>>>>>> +      else
>>>>>> +        LIBCLANG_LDFLAGS="/LIBPATH $CLANG_LIB_PATH"
>>>>>> +      fi
>>>>>> else
>>>>>> LIBCLANG_LDFLAGS=""
>>>>>> fi
>>>>>>
>>>>>> We still need to fix the copy of DLL.
>>>>>>
>>>>>> Cheers,
>>>>>> Henry
>>>>>>
>>>>>> On Sep 19, 2018, at 3:57 PM, Maurizio Cimadamore
>>>>>> <maurizio.cimadamore at oracle.com> wrote:
>>>>>>
>>>>>> Looks like good progress; tomorrow I'll take a look at some of our
>>>>>> build files and see if something special is done for mangling
>>>>>> windows include paths.
>>>>>>
>>>>>> Cheers
>>>>>> Maurizio
>>>>>>
>>>>>> On 19/09/18 23:12, Jorn Vernee wrote:
>>>>>> If I use the following flags:
>>>>>>
>>>>>> $ bash configure --disable-warnings-as-errors
>>>>>> --with-jtreg='/cygdrive/j/Libraries And Tools/jtreg-4.2-b13'
>>>>>> --with-libclang-include='J:\\LLVM\\include'
>>>>>> --with-libclang-include-aux='J:\\LLVM\\lib'
>>>>>> --with-libclang-lib='J:\\LLVM\\lib'
>>>>>>
>>>>>> (Notice that I'm having to use different path styles)
>>>>>>
>>>>>> It's detecting the header files, but it's failing on being passed
>>>>>> the wrong flags. from config.log (see at the bottom):
>>>>>>
>>>>>> configure:60248: checking for clang_getClangVersion in -lclang
>>>>>> configure:60273:
>>>>>>
>>>>> /cygdrive/j/progra~2/micros~2/2017/buildt~1/vc/tools/msvc/1414~1.264/bin/hostx86/x64/cl 
>>>>>
>>>>>> -o conftest.exe -IJ:\\LLVM\\include -LJ:\\LLVM\\lib conftest.cpp
>>>>>> -lclang   >&5
>>>>>> conftest.cpp
>>>>>> Microsoft (R) Incremental Linker Version 14.14.26430.0
>>>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>>>
>>>>>> /out:conftest.exe
>>>>>> /out:conftest.exe
>>>>>> conftest.obj
>>>>>> conftest.obj : error LNK2019: unresolved external symbol
>>>>>> clang_getClangVersion referenced in function main
>>>>>> conftest.exe : fatal error LNK1120: 1 unresolved externals
>>>>>> Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
>>>>>> Copyright (C) Microsoft Corporation.  All rights reserved.
>>>>>>
>>>>>> cl : Command line warning D9035 : option 'o' has been deprecated and
>>>>>> will be removed in a future release
>>>>>> cl : Command line warning D9002 : ignoring unknown option
>>>>>> '-LJ:\\LLVM\\lib'
>>>>>> cl : Command line warning D9002 : ignoring unknown option '-lclang'
>>>>>>
>>>>>> That seems to be a simple problem of casing-off windows and passing
>>>>>> the right flags, but I call it a night here :)
>>>>>>
>>>>>> Jorn
>>>
>>
>