Building OpenJDK9 on MSYS2
Magnus Ihse Bursie
magnus.ihse.bursie at oracle.com
Thu Oct 26 09:00:08 UTC 2017
On 2017-10-26 00:01, Peter Budai wrote:
>
> OK, I have found what was missing, it was actually my fault with a
> missing exception handler.
>
> So finally OpenJDK build has finished on Windows using gcc toolchain
> running in MSYS2/MINGW64 shell. I ran hotspot unit tests, and it looks
> promising:
>
> ./build/windows-x86_64-normal-server-fastdebug/hotspot/variant-server/libjvm/gtest/gtestLauncher.exe
> --jdk=/home/peterbud/jdk9/build/windows-x86_64-normal-server-fastdebug/jdk
>
> ….
>
> ….
>
> ….
>
> [----------] Global test environment tear-down
>
> [==========] 346 tests from 54 test cases ran. (3859 ms total)
>
> [ PASSED ] 346 tests.
>
I'm impressed! :-)
Would you care to share your current patchset, just to still my
curiosity? :-)
> What is the best way to test the current build more thoroughly?
>
"make run-test-tier1". As Erik says, you'll need jtreg, and call
"configure --with-jtreg=...". You can get it from the Adopt OpenJDK
group here:
https://adopt-openjdk.ci.cloudbees.com/job/jtreg/lastSuccessfulBuild/artifact/
/Magnus
>
> P.
>
> *From: *Bob Vandette <mailto:bob.vandette at oracle.com>
> *Sent: *Tuesday, October 24, 2017 8:10 PM
> *To: *Peter Budai <mailto:peterbudai at hotmail.com>
> *Cc: *David Holmes <mailto:david.holmes at oracle.com>; Erik Joelsson
> <mailto:erik.joelsson at oracle.com>; Magnus Ihse Bursie
> <mailto:magnus.ihse.bursie at oracle.com>; build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net>
> *Subject: *Re: Building OpenJDK9 on MSYS2
>
> Can you provide some details about your toolchain, processor and os plus
>
> a dump of the native instructions around the SEGV. This might give us
>
> enough info to be able to figure out what’s going on.
>
> Bob.
>
> On Oct 24, 2017, at 1:21 PM, Peter Budai <peterbudai at hotmail.com
> <mailto:peterbudai at hotmail.com>> wrote:
>
> I get that error running in the debugger but also running
> without/outside of the debugger as well.
>
> I saw the comment in the code about generating SEGV in function
> generate_get_cpu_info(), however the debugger can execute that,
> and the SEGV I experience is coming later in the
> VM_Version::get_processor_features() function
>
> Peter
>
> *From:*Bob Vandette <mailto:bob.vandette at oracle.com>
> *Sent:*Tuesday, October 24, 2017 6:28 PM
> *To:*Peter Budai <mailto:peterbudai at hotmail.com>
> *Cc:*David Holmes <mailto:david.holmes at oracle.com>;Erik Joelsson
> <mailto:erik.joelsson at oracle.com>;Magnus Ihse Bursie
> <mailto:magnus.ihse.bursie at oracle.com>;build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net>
> *Subject:*Re: Building OpenJDK9 on MSYS2
>
> Was this a SEGV while you were running the debugger?
>
> There is an intentional SEGV generated. This is used to determine
> if we can use some of newer
> CPU features. Try to allow the SEGV to continue to see if you run
> normally.
>
> Bob.
>
>
> > On Oct 24, 2017, at 11:37 AM, Peter Budai
> <peterbudai at hotmail.com <mailto:peterbudai at hotmail.com>> wrote:
> >
> > It seems that the compile is progressing well, I have 49
> executables/tools and 38 compiled shared libraries already in the
> JDK folder.
> >
> > I have tried to run the product with the simplest ‘java
> -version’ command, however I get a SIGSEGV at get_cpu_info_stub()
> in vm_version_x86.cpp, VM_Version::get_processor_features()
> >
> > get_cpu_info_stub(&_cpuid_info);
> >
> > Thread 5 received signal SIGSEGV, Segmentation fault.
> > 0x000000002d9a072f in ?? ()
> >
> > I can debug using gdb, but unfortunately this is pure ASM, and
> my knowledge on this is close to 0.
> >
> > Any idea help or pointer how could I tackle this?
> >
> > Peter
> >
> > From: David Holmes<mailto:david.holmes at oracle.com>
> > Sent: Sunday, October 15, 2017 10:37 PM
> > To: Peter Budai<mailto:peterbudai at hotmail.com>; Erik
> Joelsson<mailto:erik.joelsson at oracle.com>; Magnus Ihse
> Bursie<mailto:magnus.ihse.bursie at oracle.com>
> > Cc:build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net>
> > Subject: Re: Building OpenJDK9 on MSYS2
> >
> > On 16/10/2017 12:41 AM, Peter Budai wrote:
> >> A quick status update:
> >>
> >> * Hotspot successfully compiled without warnings
> >> * I’d like to run the unit tests, but as I see ‘make check’
> does not work, and gtestlauncher expects a command line parameter
> jdk. Tried to look up some documentation on this, but have not
> found. So the question is: how can I run unit tests for hotspot?
> Do I need also JDK compiled for that? Or the bootJDK is good
> enough? Any help would be appreciated.
> >
> > Hotspot has to be tested as part of a full JDK - you can't load
> the JVM
> > without having the "J" part :)
> >
> > You should be able to drop your built dll into an existing JDK 9
> windows
> > JDK and test it that way.
> >
> > David
> > -----
> >
> >> * Also I’m making progress on compiling jdk, but there are
> some very interesting solutions on windows linking which makes a
> bit more difficult to compile with gcc: LIBS_windows contains
> sometimes simple library names (which I believe is correct) and
> other times library names with full path (which I believe is not
> the best solution). I’m trying to rework those places and use
> simple library names and passing search path for libraries
> -L<path> (for gcc toolchain) and /LIBPATH:<path> (for Microsoft
> toolchain). Also I was surprised by a few manual function name
> exports…
> >> * jdk code base contains apparently more MSVC specific
> part, many places casts/lack of casts are generating errors,
> static attributes were missing etc. a bit tedious work.
> >>
> >> P.
> >>
> >> From: Erik Joelsson<mailto:erik.joelsson at oracle.com>
> >> Sent: Wednesday, October 11, 2017 4:16 PM
> >> To: Magnus Ihse Bursie<mailto:magnus.ihse.bursie at oracle.com>;
> Peter Budai<mailto:peterbudai at hotmail.com>
> >> Cc:build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net>
> >> Subject: Re: Building OpenJDK9 on MSYS2
> >>
> >> Hello,
> >>
> >> On 2017-10-11 15:48, Magnus Ihse Bursie wrote:
> >>
> >> For gcc, we let the compiler generate the .d file. For the
> Microsoft tool chain, we use a clever sed script to extract and
> create it ourself.
> >>
> >> I think that logic is checking for "Windows", not "Microsoft".
> That might be your cause of trouble.
> >>
> >> Look in NativeCompilation.gmk.
> >>
> >> That was my initial thought as well, but we do correctly check
> for microsoft. Also it's not the .d files that are the problem. As
> Peter just wrote, they look fine. It's the .d.target files which
> we create using the same technique on all platforms. What we don't
> account for is the compiler putting Windows mixed paths in the .d
> files.
> >>
> >> /Magnus
> >>
> >> 11 okt. 2017 kl. 14:43 skrev Peter Budai
> <peterbudai at hotmail.com
> <mailto:peterbudai at hotmail.com><mailto:peterbudai at hotmail.com>>:
> >> Hi Erik,
> >>
> >> The .d file looks like this:
> >>
> C:/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.obj:
> \
> >>
> C:/msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlparse.cpp \
> >>
> >> I have checked .d.targets file, and looks like it has the first
> line has not been deleted, and the file names below are also wrong:
> >>
> /msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.obj
> : :
> >> /msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlparse.cpp :
> >> /msys64/home/peterbud/jdk9/hotspot/src/share/vm/adlc/adlc.hpp :
> >>
> >> I guess this part in the DEPENDENCY_TARGET_SED_PATTERN is
> fooled by the “C:/”
> >> -e 's/^[^:]*: *//'
> >>
> >> Yes, that does indeed look like the problem. I suppose the
> regexp is unnecessarily strict. It should be ok to rewrite it as
> something like this:
> >> -e 's/^.*: *//'
> >>
> >> Basically just make sure it ends with : and any number of spaces.
> >>
> >> /Erik
> >>
> >>
> >>
> >> Peter
> >>
> >> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986>
> for Windows 10
> >>
> >> From: Erik Joelsson<mailto:erik.joelsson at oracle.com>
> >> Sent: Wednesday, October 11, 2017 12:16 PM
> >> To: Peter Budai<mailto:peterbudai at hotmail.com>; Magnus Ihse
> Bursie<mailto:magnus.ihse.bursie at oracle.com>;build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net>
> >> Subject: Re: Building OpenJDK9 on MSYS2
> >>
> >>
> >> Hello Peter,
> >>
> >> On 2017-10-11 00:18, Peter Budai wrote:
> >> Thanks Magnus & Erik
> >>
> >> First thanks for your support and kind words!
> >>
> >> Magnus, I have checked .bash_profile, .bashrc but they seem to
> be empty (everything is commented out). You can check with a
> default MSYS2 install, I have not changed these files at all. If
> you find thee something specific I can give a try here as well.
> >>
> >> Let me give also a quick status update, where am I with
> building hotspot:
> >> · I guess its still the beginning, but I have managed to
> compile jvm.dll with almost 700 object file: with debug info the
> dll is around 700 MB😊
> >> · I made only surgical, minimal changes to the source,
> and so far it looks reasonable. I have encountered 3 scenarios
> where changes were necessary:
> >> o When in makefiles conditionals were using assuming that if
> target_os is windows then it is visual studio compiler/linker.
> Obviously these conditionals had to be reviewed in a few places
> and if necessary were changes to check the toolchain=Microsoft
> >> These are not surprising and should be pretty straight forward
> to fix and it seems you know what to do.
> >>
> >>
> >> ·
> >> o I got a few warnings as gcc 7.2 uncovered some code
> problems in windows specific codes, where before that MSVC I guess
> did not say a word…
> >> To get around this you can configure with
> --disable-warnings-as-errors until you get things working
> properly. This is commonly needed when using compiler versions
> that we normally don't use.
> >>
> >>
> >> ·
> >> o And I had like 6-7 places where the code was using MSVC
> specific __try … __except structures which gcc does not know. Do
> you have a suggestion how to approach them? I can do ugly #ifdefs
> (I would avoid that) but I have also seen some solutions to
> replace them with a code which gcc can compile
> (http://www.programmingunlimited.net/siteexec/content.cgi?page=mingw-seh)
> – but before doing that though I would ask first you on the
> purpose of those
> >> This kind of question is probably best to bring to the hotspot
> mailing list.
> >> · What bothers me is that I was not able to do
> incremental builds: when an error occurs, and build stops, then
> after making change in the CPP source the build cannot continue, I
> always got an error message:
> >>
> >>
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/hotspot/variant-server/tools/adlc/objs/adlparse.d.targets:1:
> *** missing target pattern. Stop.
> >> make[2]: *** [make/Main.gmk:256: hotspot-server-gensrc] Error 2
> >>
> >> If I do a ‘make clean’ and restart the build then it nicely
> compiles.
> >>
> >> Question 1: Is there a way to resume such builds without ‘make
> clean’?
> >> Well, incremental builds is supposed to work well. We have
> several extra tricks in there to handle cases where normal make
> builds would fail. The *.d.targets files is one such trick and it
> seems to backfire for you. The contents of that file should be
> something like:
> >>
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlparse.cpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlc.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/opcodes.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/classes.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/arena.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/opto/adlcVMDeps.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/filebuff.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/dict2.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/forms.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/formsopt.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/formssel.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/archDesc.hpp :
> >> /localhome/hg/jdk9-dev/hotspot/src/share/vm/adlc/adlparse.hpp :
> >>
> >> Basically an empty rule for each dependency for the
> corresponding object file. Declaring these rules makes it possible
> to delete source files without having to build clean. It seems
> your file is not generated correctly so please have a look inside
> it. The file is in make/common/NativeCompilation.gmk, look for
> DEPENDENCY_TARGET_SED_PATTERN.
> >>
> >>
> >>
> >> Question 2: What would be the best way to submit/share the
> patches for your thorough review?
> >>
> >> Well, first of all, have you signed the OCA?
> >>
> >> As for publishing patches and reviews, there is a bit of
> chicken and egg problem. Once you become an "author" in any of the
> OpenJDK projects, you get a user name and should be able to
> publish reviews oncr.openjdk.java.net
> <http://cr.openjdk.java.net/><http://cr.openjdk.java.net
> <http://cr.openjdk.java.net/>>. Before that, if the patch is
> small, it can be posted inline in an email to the list. If it's
> large, you will need a current OpenJDK user to host it for you. At
> least that's how I understand it. Hopefully someone who knows the
> process better can chime in here.
> >>
> >> I should also let you know that getting this into JDK 9 is most
> likely not going to happen. AFAIK we are only doing security
> updates for 9. It would have to go into the currently active
> release. I should also warn you that new ports generally need a
> certain amount of backing to be accepted. It may be that this
> would have to live in a porting side project. Hopefully someone
> who knows this better can chime in here as well.
> >>
> >> /Erik
> >>
> >>
> >> P.
> >>
> >> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986>
> for Windows 10
> >>
> >> From: Magnus Ihse Bursie<mailto:magnus.ihse.bursie at oracle.com>
> >> Sent: Tuesday, October 10, 2017 10:04 AM
> >> To: Peter Budai<mailto:peterbudai at hotmail.com>; Erik
> Joelsson<mailto:erik.joelsson at oracle.com>;build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net>
> >> Subject: Re: Building OpenJDK9 on MSYS2
> >>
> >> On 2017-10-07 10:14, Peter Budai wrote:
> >> The configure of OpenJDK overwrites the SHELL. Actually it is
> using bash, but for the arguments it was using “-e -o pipefail”. I
> have figured that for MSYS2 bash what is needed as bash arguments
> is “-e -l -c -o pipefail”
> >>
> >> That looks like solving this problem, and now the real issues
> are surfacing.
> >>
> >> FWIW, "-l" makes bash behave like a login shell. Most likely
> you are changing bash's behavior in one of your login scripts, and
> that change is what's really needed.
> >>
> >> /Magnus
> >>
> >>
> >>
> >>
> >>
> >> Peter
> >>
> >> From: Peter Budai<mailto:peterbudai at hotmail.com>
> >> Sent: Friday, October 6, 2017 6:43 PM
> >> To: Magnus Ihse Bursie<mailto:magnus.ihse.bursie at oracle.com>;
> Erik
> Joelsson<mailto:erik.joelsson at oracle.com>;build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net>
> >> Subject: RE: Building OpenJDK9 on MSYS2
> >>
> >> Magnus,
> >>
> >> I have followed your suggestion and removed the fixpath
> prefixes from gcc related compile tools, and left only the fixpath
> prefix _only_ for the Boot JDK related tools in place.
> >>
> >> 1) As I follow the process, all java and javac related
> compile steps are running properly
> >> 2) When the process reaches gcc related steps I got the
> error message at the same place as before (no fixpath). If I
> execute that command from the bash prompt, it creates the output:
> >>
> >> $ ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
> /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
> && /C/msys64/mingw64/bin/gcc -E -x c
> /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
> 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2)
> | /usr/bin/gawk '/@@START_HERE@@/,0' | /usr/bin/sed -e
> 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT
> EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) >
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
> >>
> >> As I have mentioned the parameters are replaced by the bash
> automatically
> >> 3) Then build continues, then little later stops at a
> super simple command:
> >>
> >> mv
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java.tmp
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/java/nio/ByteBuffer.java
> >> Needless to say, the ByteBuffer.java.tmp file DOES
> exist. And running the above command from the bash works, and
> build continues.
> >> 4) A few similar cases (stops) with DirectByteBuffer and
> DirectByteBufferR
> >>
> >>
> >> Currently I try to explore how that might relate to the MSYS2
> bash and make, somehow it behaves differently
> >>
> >> If you have any other suggestion, let me know.
> >>
> >> Best regards,
> >>
> >> Peter
> >>
> >> From: Peter Budai<mailto:peterbudai at hotmail.com>
> >> Sent: Thursday, October 5, 2017 3:52 PM
> >> To: Magnus Ihse Bursie<mailto:magnus.ihse.bursie at oracle.com>;
> Erik
> Joelsson<mailto:erik.joelsson at oracle.com>;build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net>
> >> Subject: RE: Building OpenJDK9 on MSYS2
> >>
> >> Hi Magnus,
> >>
> >> So first of all, here is the current patch, which I was not
> able to attach:https://pastebin.com/pwT4Ynxc
> >>
> >>>> That's surprising, since gcc is prefixed with fixpath, which
> it should not.
> >> Actually you DO need fixpath IMHO.
> >> This is a mingw64 version of the gcc
> (/C/msys64/mingw64/bin/gcc), which is a fully functional Windows
> executable, which expects Windows formatted path arguments.
> >> As the updated build process uses EXPORT MSYS2_ARG_CONV_EXCL=*
> (see that patch), none of the command line arguments are
> converted from the unix path to Windows, but fixpath does that
> conversion. There is a wiki describing more details on this:
> >>https://github.com/msys2/msys2/wiki/Porting#user-content-filesystem-namespaces
> >>
> >>
> >>
> >>>> I have a hard time believing this is a race condition. On the
> other hand, this stuff is weird, we're misusing the C preprocessor
> to process defines in java code, so I'm not surprised it breaks down.
> >>>> I don't know why it succeeded when run on the command line,
> though.
> >> When I execute that command from the bash command line there is
> no EXPORT MSYS2_ARG_CONV_EXCL, but the bash itself does the
> automatic conversion of the arguments. Maybe it has something to
> do how fixpath does CreateProcess?
> >>
> >> Does that help?
> >>
> >> Best regards,
> >>
> >> Peter
> >>
> >> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986>
> for Windows 10
> >>
> >> From: Magnus Ihse Bursie<mailto:magnus.ihse.bursie at oracle.com>
> >> Sent: Thursday, October 5, 2017 12:13 PM
> >> To: Peter Budai<mailto:peterbudai at hotmail.com>; Erik
> Joelsson<mailto:erik.joelsson at oracle.com>;build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net>
> >> Subject: Re: Building OpenJDK9 on MSYS2
> >>
> >>
> >> On 2017-10-05 11:59, Peter Budai wrote:
> >> Hi Magnus and Erik,
> >>
> >> I really appreciate your quick feedback. I assumed that it
> won’t be easy, but I just don’t feel I should give up now - maybe
> later when I see the real scale of work. So bear with me for a
> time being.
> >>
> >> Attached is a patch which already includes Magnus’ changes,
> plus a few which I have added:
> >> · basically enabling gcc for windows,
> >> · and modifying a logic for compiling fixpath (before
> that it was using hard-coded MS VSC compile flags)
> >>
> >> Actually, you must make sure fixpath is *not* used for the
> toolchain, since gcc uses unix style paths.
> >> (However, other tools such as java will still need it.)
> >>
> >>
> >>
> >>
> >>
> >>
> >> So here is what I have as the result of configure:
> >> ====================================================
> >> The existing configuration has been successfully updated in
> >>
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release
> >> using configure arguments '--disable-freetype-bundling
> --disable-javac-server'.
> >>
> >> Configuration summary:
> >> * Debug level: release
> >> * HS debug level: product
> >> * JDK variant: normal
> >> * JVM variants: server
> >> * OpenJDK target: OS: windows, CPU architecture: x86, address
> length: 64
> >> * Version string: 9-internal+0-adhoc.peterbud.jdk9 (9-internal)
> >>
> >> Tools summary:
> >> * Environment: msys version 2.9.0(0.318/5/3) (root at /C/msys64)
> >> * Boot JDK: java version "1.8.0_144" Java(TM) SE Runtime
> Environment (build 1.8.0_144-b01) Java HotSpot(TM) 64-Bit Server
> VM (build 25.144-b01, mixed mode) (at /c/progra~1/java/jdk18~1.0_1)
> >> * Toolchain: gcc (GNU Compiler Collection)
> >> * C Compiler: Version 7.2.0 (at /C/msys64/mingw64/bin/gcc)
> >> * C++ Compiler: Version 7.2.0 (at /c/msys64/mingw64/bin/g++)
> >>
> >> Build performance summary:
> >> * Cores to use: 4
> >> * Memory limit: 16216 MB
> >>
> >> Its clear says that the toolchain is gcc 7.2 (BTW there is no
> Visual Studio on this machine)
> >>
> >> Now for the details of the config log, you can see
> here:https://pastebin.com/MN2ZYcHH
> >>
> >> And about the build process and the error I get:
> >>
> >> $ make JOBS=1
> >> Building target 'default (exploded-image)' in configuration
> 'windows-x86_64-normal-server-release'
> >> Compiling 8 files for BUILD_TOOLS_LANGTOOLS
> >> Compiling 17 properties into resource bundles for jdk.compiler
> >> Parsing 1 properties into enum-like class for jdk.compiler
> >> Compiling 19 properties into resource bundles for jdk.javadoc
> >> Compiling 12 properties into resource bundles for jdk.jdeps
> >> Compiling 7 properties into resource bundles for jdk.jshell
> >> Compiling 117 files for BUILD_INTERIM_java.compiler
> >> Compiling 396 files for BUILD_INTERIM_jdk.compiler
> >> Compiling 61 files for BUILD_INTERIM_jdk.jdeps
> >> Compiling 457 files for BUILD_INTERIM_jdk.javadoc
> >> Note: Some input files use or override a deprecated API.
> >> Note: Recompile with -Xlint:deprecation for details.
> >> Compiling 159 files for BUILD_TOOLS_JDK
> >> Note: Some input files use unchecked or unsafe operations.
> >> Note: Recompile with -Xlint:unchecked for details.
> >> make[3]: *** [GensrcMisc.gmk:78:
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
> Error 1
> >> make[3]: *** Deleting file
> '/C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java'
> >> make[2]: *** [make/Main.gmk:115: java.base-gensrc-jdk] Error 2
> >>
> >> ERROR: Build failed for target 'default (exploded-image)' in
> configuration 'windows-x86_64-normal-server-release' (exit code 2)
> >>
> >> No indication of failed target found.
> >> Hint: Try searching the build log for '] Error'.
> >> Hint: See common/doc/building.html#troubleshooting for assistance.
> >>
> >> make[1]: *** [/home/peterbud/jdk9/make/Init.gmk:296: main] Error 2
> >> make: *** [/home/peterbud/jdk9/make/Init.gmk:185: default] Error 2
> >>
> >> If I run here
> >> make JOBS=1 LOG=debug
> >> The failing line seems to be this:
> >>
> >> ( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1'
> /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
> &&
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/configure-support/bin/fixpath.exe
> -m/C/msys64/@/c/msys64/@/c/progra~ /C/msys64/mingw64/bin/gcc -E -x
> c
> /C/msys64/home/peterbud/jdk9/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template
> 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2)
> | /usr/bin/gawk '/@@START_HERE@@/,0' | /usr/bin/sed -e
> 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT
> EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) >
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
> >> make[3]: *** [GensrcMisc.gmk:78:
> /C/msys64/home/peterbud/jdk9/build/windows-x86_64-normal-server-release/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java]
> Error 1
> >>
> >> Now the interesting is: if I copy this line above to the bash
> prompt, it runs without problem, and the file
> support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java
> >> That's surprising, since gcc is prefixed with fixpath, which it
> should not.
> >>
> >> I have a hard time believing this is a race condition. On the
> other hand, this stuff is weird, we're misusing the C preprocessor
> to process defines in java code, so I'm not suprised it breaks
> down. I don't know why it succeeded when run on the command line,
> though. My suggestion is to just do some quick and dirty hack
> around this: take the file you manage to generate and just copy it
> in during the build instead. If you can get round this, you might
> start seeing some *real* problems. :-)
> >>
> >> Also, my suggestion is that you try running "make hotspot" to
> cut to the chase. Compiling hotspot will likely be the hardest
> thing. Or even "make -k hotspot" to get an assessment of the
> amount of work ahead of you.
> >>
> >> /Magnus
> >> Is produced.
> >>
> >> Then I can again issue
> >> make JOBS=1 LOG=debug
> >>
> >> And the compile process is being continued, until a similar
> error pops up with a different generated file. I have an
> assumption that this happens because make is still running
> parallel jobs, despite JOBS=1 but I’m not sure.
> >>
> >> How could I best tackle this?
> >>
> >> Thank you and best regards,
> >>
> >> Peter
> >>
> >> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986>
> for Windows 10
> >>
> >> From: Magnus Ihse Bursie<mailto:magnus.ihse.bursie at oracle.com>
> >> Sent: Thursday, October 5, 2017 11:33 AM
> >> To: Erik Joelsson<mailto:erik.joelsson at oracle.com>; Peter
> Budai<mailto:peterbudai at hotmail.com>;build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net>
> >> Subject: Re: Building OpenJDK9 on MSYS2
> >>
> >> On 2017-10-05 10:10, Erik Joelsson wrote:
> >>> Hello Peter,
> >>>
> >>>
> >>> On 2017-10-04 21:15, Peter Budai wrote:
> >>>> Hi Magnus,
> >>>>
> >>>> Thanks for the quick reply I’ll check these patches with msys2.
> >>>>
> >>>> Let me specify with more details what I’d like to achieve:
> I’d like
> >>>> to build OpenJDK9 with MSYS2 MINGW64 environment using gcc
> toolchain.
> >>>> (I’m not sure how familiar are you with MSYS2, but there are 3
> >>>> different environments: MSYS2, MINGW32 and MINGW64). In theory
> >>>> MINGW64 with gcc is the closes you can get on Windows
> platform as a
> >>>> gcc unix like build environment, which produces still a
> native 64-bit
> >>>> executable on Windows.
> >>>>
> >>>> I’m not very familiar with OpenJDK yet, so therefore I’d like
> to hear
> >>>> your opinion: how realistic is that?
> >>> Sorry to disappoint, but I would say that requires major work.
> There
> >>> is a strong historic assumption that windows builds are done using
> >>> Visual Studio. We have abstracted away some of it in configure
> (see
> >>> TOOLCHAIN_TYPE), but it's very far from enough to change compiler
> >>> environment for a Windows build. The native sources are also
> bound to
> >>> make a lot of such assumptions. I would expect the changes
> needed to
> >>> be in the thousands of lines of code.
> >>
> >> I agree that it requires hard work (even if "thousands" might be an
> >> overestimation I think, but "hundreds" is not enough, so it's
> the right
> >> magnitude). On the other hand, it would be really good if we
> did sort
> >> things out, so that we had proper conditions based on OS vs
> >> compiler/toolchain.
> >>
> >> If you really want to start, the first thing is to patch
> toolchain.m4 to
> >> VALID_TOOLCHAINS_windows="microsoft gcc"
> >> and then call configure using "bash configure
> --with-toolchain-type=gcc".
> >>
> >> As Erik, I doubt you will come very far before things starts
> tumbling down.
> >>
> >>>
> >>> When we say supporting the build in msys2 instead of cygwin,
> we just
> >>> mean using msys2 as the unix emulating layer for our tools like
> >>> make/bash/grep/sed etc.
> >>>
> >>> One think I have done successfully is running the build in WSL
> >>> (Windows Subsystem for Linux), but that isn't all that helpful
> as WSL
> >>> for practical purposes is more or less like running Linux in a
> VM, so
> >>> the build sees a Linux system and builds a Linux binary.
> >>>> As a side note: with MINGW64 I have managed to run configure
> phase
> >>>> successfully for OpenJDK. The compile process has also
> started and
> >>>> went for a while, but interestingly I run into some kind of race
> >>>> conditions as make stopped with an error. Using LOG=debug I
> have fond
> >>>> the failing line and then copying the failed command and
> pasting it
> >>>> to the bash prompt it successfully generated the output
> target, and
> >>>> then the build process run further when a similar situation
> happened.
> >>>> Also pasting the failed command run in the bash without any
> problem,
> >>>> and build continued… until the next.
> >>> Without seeing the errors I can't say much. I very much doubt
> that you
> >>> are running with gcc as the compiler though. Configure isn't
> easily
> >>> fooled into using a different compiler to what it prefers, and
> I would
> >>> expect things to crash and burn pretty early if you actually did.
> >>>
> >>> /Erik
> >>>>
> >>>> I have tried to run make JOBS=1, but did not help, strangely
> I have
> >>>> still seen in the log make[3] and make[4] logs which
> suggested that
> >>>> there are more than one make jobs were running. Also tried
> .configure
> >>>> --with-output-sync=recurse without success (same symptoms)
> >>>>
> >>>> Let me know your thoughts.
> >>>>
> >>>> Best regards,
> >>>>
> >>>> Peter
> >>>>
> >>>> Sent from
> Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
> >>>> Windows 10
> >>>>
> >>>> From: Magnus Ihse Bursie<mailto:magnus.ihse.bursie at oracle.com>
> >>>> Sent: Wednesday, October 4, 2017 1:04 AM
> >>>> To: Peter Budai<mailto:peterbudai at hotmail.com>;
> >>>>build-dev at openjdk.java.net
> <mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net><mailto:build-dev at openjdk.java.net>
> >>>> Subject: Re: Building OpenJDK9 on MSYS2
> >>>>
> >>>> Actually, it wasn't so much remaining trouble. :-) I fired up
> msys2 and
> >>>> checked out where I left off. It turned out that the
> remaining snag was
> >>>> that msys2 tries to convert command lines automatically, from
> "unix"
> >>>> style paths to "windows" style paths. Unfortunately, it does
> not do this
> >>>> very well and it breaks all sorts of things. We already have
> a FIXPATH
> >>>> solution in place which deals with this, so basically all I
> had to do
> >>>> was disable this (by setting MSYS2_ARG_CONV_EXCL to "*").
> However, this
> >>>> broke our cygpath replacement hack (!) so I had to disable it
> there.
> >>>> Sigh. Anyway, with those fixes it ran and worked well. (I also
> >>>> discovered and fixed a bug related to how we set up the
> FIXPATH variable
> >>>> on msys, but it only triggers in certain circumstances).
> >>>>
> >>>> With this patch I now jdk9 seems to build fine on msys2. It
> should apply
> >>>> cleanly on jdk9/jdk9. Since it turned out to be so trivial,
> I'll try to
> >>>> get it in in jdk10.
> >>>>
> >>>> Here's the patch if you want to apply it yourself:
> >>>>
> >>>> diff -r a08cbfc0e4ec common/autoconf/basics_windows.m4
> >>>> --- a/common/autoconf/basics_windows.m4 Thu Aug 03
> 18:56:56 2017
> >>>> +0000
> >>>> +++ b/common/autoconf/basics_windows.m4 Wed Oct 04
> 00:53:58 2017
> >>>> +0200
> >>>> @@ -42,7 +42,7 @@
> >>>> windows_path=`$CYGPATH -m "$unix_path"`
> >>>> $1="$windows_path"
> >>>> elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
> >>>> - windows_path=`cmd //c echo $unix_path`
> >>>> + windows_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $unix_path`
> >>>> $1="$windows_path"
> >>>> fi
> >>>> ])
> >>>> @@ -136,6 +136,16 @@
> >>>> fi
> >>>> ])
> >>>>
> >>>> +AC_DEFUN([BASIC_MSYS_UPDATE_FIXPATH],
> >>>> +[
> >>>> + # Take all collected prefixes and turn them into a
> >>>> -m/c/foo@/c/bar at ... command line
> >>>> + # @ was chosen as separator to minimize risk of other
> tools messing
> >>>> around with it
> >>>> + all_unique_prefixes=`echo "${all_fixpath_prefixes@<:@@@:>@}" \
> >>>> + | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
> >>>> + fixpath_argument_list=`echo $all_unique_prefixes | tr ' '
> '@'`
> >>>> + FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
> >>>> +])
> >>>> +
> >>>> AC_DEFUN([BASIC_FIXUP_PATH_MSYS],
> >>>> [
> >>>> path="[$]$1"
> >>>> @@ -143,7 +153,7 @@
> >>>> new_path="$path"
> >>>> if test "x$has_colon" = x; then
> >>>> # Not in mixed or Windows style, start by that.
> >>>> - new_path=`cmd //c echo $path`
> >>>> + new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $path`
> >>>> fi
> >>>>
> >>>> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
> >>>> @@ -155,6 +165,8 @@
> >>>>
> >>>> # Save the first 10 bytes of this path to the storage,
> so fixpath
> >>>> can work.
> >>>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
> >>>> "${new_path:0:10}")
> >>>> + # We might need to re-evaluate FIXPATH.
> >>>> + BASIC_MSYS_UPDATE_FIXPATH
> >>>> ])
> >>>>
> >>>> AC_DEFUN([BASIC_FIXUP_EXECUTABLE_CYGWIN],
> >>>> @@ -293,7 +305,7 @@
> >>>> # Do not save /bin paths to all_fixpath_prefixes!
> >>>> else
> >>>> # Not in mixed or Windows style, start by that.
> >>>> - new_path=`cmd //c echo $new_path`
> >>>> + new_path=`MSYS2_ARG_CONV_EXCL= cmd //c echo $new_path`
> >>>> BASIC_MAKE_WINDOWS_SPACE_SAFE_MSYS([$new_path])
> >>>> # Output is in $new_path
> >>>> BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(new_path)
> >>>> @@ -302,6 +314,8 @@
> >>>>
> >>>> # Save the first 10 bytes of this path to the storage,
> so fixpath
> >>>> can work.
> >>>> all_fixpath_prefixes=("${all_fixpath_prefixes@<:@@@:>@}"
> >>>> "${new_path:0:10}")
> >>>> + # We might need to re-evaluate FIXPATH.
> >>>> + BASIC_MSYS_UPDATE_FIXPATH
> >>>> fi
> >>>> ])
> >>>>
> >>>> @@ -347,6 +361,10 @@
> >>>> WINDOWS_ENV_VENDOR='msys'
> >>>> WINDOWS_ENV_VERSION="$MSYS_VERSION"
> >>>>
> >>>> + # Prohibit msys2 path conversion from trying to be
> "intelligent",
> >>>> and rely
> >>>> + # on fixpath instead.
> >>>> + export MSYS2_ARG_CONV_EXCL="*"
> >>>> +
> >>>> AC_MSG_CHECKING([msys root directory as unix-style path])
> >>>> # The cmd output ends with Windows line endings
> (CR/LF), the grep
> >>>> command will strip that away
> >>>> MSYS_ROOT_PATH=`cd / ; cmd /c cd | $GREP ".*"`
> >>>> @@ -391,10 +409,7 @@
> >>>> elif test "x$OPENJDK_BUILD_OS_ENV" = xwindows.msys; then
> >>>> # Take all collected prefixes and turn them into a
> >>>> -m/c/foo@/c/bar at ... command line
> >>>> # @ was chosen as separator to minimize risk of
> other tools
> >>>> messing around with it
> >>>> - all_unique_prefixes=`echo
> "${all_fixpath_prefixes@<:@@@:>@}" \
> >>>> - | tr ' ' '\n' | $GREP '^/./' | $SORT | $UNIQ`
> >>>> - fixpath_argument_list=`echo $all_unique_prefixes | tr
> ' ' '@'`
> >>>> - FIXPATH="$FIXPATH_BIN -m$fixpath_argument_list"
> >>>> + BASIC_MSYS_UPDATE_FIXPATH
> >>>> fi
> >>>> FIXPATH_SRC_W="$FIXPATH_SRC"
> >>>> FIXPATH_BIN_W="$FIXPATH_BIN"
> >>>> diff -r a08cbfc0e4ec common/autoconf/build-aux/config.sub
> >>>> --- a/common/autoconf/build-aux/config.sub Thu Aug 03 18:56:56
> >>>> 2017 +0000
> >>>> +++ b/common/autoconf/build-aux/config.sub Wed Oct 04 00:53:58
> >>>> 2017 +0200
> >>>> @@ -30,7 +30,7 @@
> >>>> DIR=`dirname $0`
> >>>>
> >>>> # First, filter out everything that doesn't begin with
> "aarch64-"
> >>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
> >>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
> >>>> . $DIR/autoconf-config.sub "$@"
> >>>> # autoconf-config.sub exits, so we never reach here,
> but just in
> >>>> # case we do:
> >>>> @@ -38,13 +38,17 @@
> >>>> fi
> >>>>
> >>>> while test $# -gt 0 ; do
> >>>> - case $1 in
> >>>> + case $1 in
> >>>> -- ) # Stop option processing
> >>>> shift; break ;;
> >>>> aarch64-* )
> >>>> config=`echo $1 | sed 's/^aarch64-/arm-/'`
> >>>> sub_args="$sub_args $config"
> >>>> shift; ;;
> >>>> + *-msys )
> >>>> + config=`echo $1 | sed 's/msys/mingw32/'`
> >>>> + sub_args="$sub_args $config"
> >>>> + shift; ;;
> >>>> - ) # Use stdin as input.
> >>>> sub_args="$sub_args $1"
> >>>> shift; break ;;
> >>>> diff -r a08cbfc0e4ec common/autoconf/spec.gmk.in
> >>>> --- a/common/autoconf/spec.gmk.in Thu Aug 03 18:56:56 2017
> +0000
> >>>> +++ b/common/autoconf/spec.gmk.in Wed Oct 04 00:53:58 2017
> +0200
> >>>> @@ -120,6 +120,13 @@
> >>>> # On Windows, the Visual Studio toolchain needs the PATH
> to be
> >>>> adjusted
> >>>> # to include Visual Studio tools (this needs to be in
> cygwin/msys
> >>>> style).
> >>>> export PATH:=@VS_PATH@
> >>>> +
> >>>> +endif
> >>>> +
> >>>> +ifeq ($(OPENJDK_TARGET_OS_ENV), windows.msys)
> >>>> + # On msys2, prohibit msys path conversion from trying to be
> >>>> + # "intelligent", and rely on fixpath instead.
> >>>> + export MSYS2_ARG_CONV_EXCL:=*
> >>>> endif
> >>>>
> >>>> SYSROOT_CFLAGS := @SYSROOT_CFLAGS@
> >>>>
> >>>> /Magnus
> >>>>
> >>>> On 2017-10-03 22:34, Magnus Ihse Bursie wrote:
> >>>>> I gave msys2 a shot some time ago, but it ended up too much
> trouble.
> >>>>> I'll share some of my notes from that attempt, for what it's
> worth.
> >>>>>
> >>>>> To install package X/Y, run "pacman -S X/Y". Missing tools and
> >>>>> packages where to find them:
> >>>>> cmp: msys/diffutils
> >>>>> tar: msys/tar
> >>>>> make: msys/make
> >>>>> unzip: msys/unzip
> >>>>> zip: msys/zip
> >>>>>
> >>>>> config.sub reports msys as "x86_64-pc-mingw32" but msys2 as
> >>>>> "x86_64-pc-msys". This patch adds postprocessing in "our"
> config.sub
> >>>>> to report msys2 similar to msys. (Opinions, including my own
> :-) may
> >>>>> vary if this really is the best way..)
> >>>>>
> >>>>> diff -r b88023f46daa common/autoconf/build-aux/config.sub
> >>>>> --- a/common/autoconf/build-aux/config.sub Fri Jan 27
> 10:15:41
> >>>>> 2017 +0100
> >>>>> +++ b/common/autoconf/build-aux/config.sub Fri Feb 03
> 05:00:25
> >>>>> 2017 -0700
> >>>>> @@ -30,7 +30,7 @@
> >>>>> DIR=`dirname $0`
> >>>>>
> >>>>> # First, filter out everything that doesn't begin with
> "aarch64-"
> >>>>> -if ! echo $* | grep '^aarch64-' >/dev/null ; then
> >>>>> +if ! echo $* | grep -e '^aarch64-' -e 'msys' >/dev/null ; then
> >>>>> . $DIR/autoconf-config.sub "$@"
> >>>>> # autoconf-config.sub exits, so we never reach here,
> but just in
> >>>>> # case we do:
> >>>>> @@ -45,6 +45,10 @@
> >>>>> config=`echo $1 | sed 's/^aarch64-/arm-/'`
> >>>>> sub_args="$sub_args $config"
> >>>>> shift; ;;
> >>>>> + *-msys )
> >>>>> + config=`echo $1 | sed 's/msys/mingw32/'`
> >>>>> + sub_args="$sub_args $config"
> >>>>> + shift; ;;
> >>>>> - ) # Use stdin as input.
> >>>>> sub_args="$sub_args $1"
> >>>>> shift; break ;;
> >>>>>
> >>>>> If I remember correctly, this got me past the configure
> stage at the
> >>>>> time.
> >>>>>
> >>>>> I don't think it's very hard to get it to work on msys2, I
> just ran
> >>>>> into one snag too many and didn't think msys2 would be used
> by anyone.
> >>>>>
> >>>>> /Magnus
> >>>>>
> >>>>> On 2017-10-03 17:20, Peter Budai wrote:
> >>>>>> Hello,
> >>>>>>
> >>>>>> According to
> >>>>>>http://hg.openjdk.java.net/jdk9/jdk9/file/a08cbfc0e4ec/common/doc/building.html
> >>>>>>
> >>>>>> “msys2 and the new Windows Subsystem for Linux (WSL) would
> likely be
> >>>>>> possible to support in a future version but that would
> require a
> >>>>>> community effort to implement”
> >>>>>>
> >>>>>> I’d like to help making the OpenJDK 9 build working on
> msys2. What is
> >>>>>> the best way to move forward? Is there a similar effort in
> progress?
> >>>>>>
> >>>>>> Thank you and best regards,
> >>>>>>
> >>>>>> Peter
> >>>>>>
> >>>>>>
> >>>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
>
More information about the build-dev
mailing list