OpenJDK successfully built with Visual Studio express edition

[Andrew John Hughes]

2009/12/10 Kelly O'Hair <Kelly.Ohair at sun.com>:
>
> Damjan Jovanovic wrote:
>>
>> Hi
>>
>> I'm documenting my (unbelievably difficult but ultimately) successful
>> experience of building OpenJDK on Windows with Visual Studio express
>> edition.
>
> Thanks for sending this out. You do deserve some kind of medal, building
> on Windows is very difficult.
>
> First off, where and how did you get the sources?
> It's not exactly clear what sources we are dealing with here.
>

Errr... he says in the next paragraph 'the awt tree from about a week
ago' i.e. http://hg.openjdk.java.net/jdk7/awt

My congratulations too on the successful build.  I wouldn't even want
to contemplate this...

>>
>> I built the awt tree from about a week ago, with Windows Vista, Cygwin
>> 1.5.25-15, Microsoft Visual Studio 2008 Express Edition SP1, Freetype
>> 2.3.5-1, jdk-7-ea-plug-b77-windows-i586-03_dec_2009.jar, DirectX SDK
>> as per README-builds.html, Ant copied from Netbeans 6.7, and make from
>> the cmake.org website (as per README-builds.html). The import JDK was
>> 1.6.0.
>
> So the devil is in the details, but first off, you have picked a few things
> that differ from our recommended set of things:
>
>  * Windows Vista - My condolences, but I don't know of anyone using Vista
>    to build. I know Windows 2000 is almost 10 years old, and Windows XP is
>    pretty old, but XP is the newest OS we can recommend. Anything newer is
>    in the 'on your own' category. But I am glad you were ultimately
> successful,
>    at least we know it is possible.
>
>  * Microsoft Visual Studio 2008 Express Edition SP1 - We recommend the
>    purchased copy, but we understand why you are using this version and
>    we would like to see it work, but it's the ATL issue that is the killer.
>    Whatever adjustments we can make to allow it's use we will do.
>
>  * Freetype (on windows) - A royal pain in my side. More later.
>
>  * The jdk-7-ea-plug-b77-windows-i586-03_dec_2009.jar should not be needed
>    anymore for a successful build. I would skip it. (Binary plugs are
> optional).
>

Reminds me we need to forwardport the doc. update for this.  OpenJDK6
lists them as optional, but 7 doesn't.

>  * DirectX SDK - Which version did you use? Just curious.
>
>  * Ant copied from Netbeans 6.7 - If that worked, fine. But I would really
>    rather people download and install the official 1.7.1 version of ant.
>    NetBeans may be sifting out parts of ant in it's install, so using the
>    ant from NetBeans may not be the same behavior as from an official ant
>    install.  But if you got it to work, that's great.
>

This is especially true with the recent jaxp/jaxws changes which rely
on the regexp extension.  I know that's not in the base package on
most GNU/Linux distros.

>  * Was the cmake you got 3.81? Just curious.
>
>>
>> Firstly the README-builds.html talks a lot about path problems,
>> without offering any good solutions. After much effort, I discovered
>> that you have to use /cygpath/c paths only in $PATH and use double
>> quote quoted C:/ style paths everywhere else (eg. export
>> WindowsSdkDir="C:/Program Files/Microsoft SDKs/Windows/v6.0A/").
>
> If this isn't clear I'll try and improve it.
> By the way, the Microsoft SDK should probably be v6.1, but I don't
> think it will matter much. You did not list installing the Microsoft SDK
> above, did you explicitly install v6.0A or did that come with the Visual
> Studio compiler install?
>
>>
>> Microsoft's quality C compilers apparently need environment variables
>> set before they work properly, otherwise they crash (not complain, but
>> actually crash). The vcvars32.bat doesn't work from inside cygwin, so
>> I had to run "set > a.txt", then run "vcvars32.bat", then "set >
>> b.txt", and finally diff a.txt with b.txt to find out what these
>> mysterious variables were, then convert them into C:/ and /cygpath/c
>> style paths that get set in cygwin. Running
>> jdk/make/jdk_generic_profile.sh loses the variables again, so I
>> avoided it.
>
> Sigh...  I should change jdk/make/jdk_generic_profile.sh, it was never
> made cygwin compliant. That file was mainly meant for documentation
> purposes, not sure how I feel about people using it as part of their
> build process, :^(
>
> On vcvars32.bat, I have created a shell script called vsvars.sh
> (See http://blogs.sun.com/kto/entry/windows_visual_studio_stupid_pet)
> that I plan on adding to the jdk repository at some point.
> It will (from a cygwin shell) effectively do the vcvars32.bat for
> you and set PATH correctly. All you need to do is run
>  eval `vsvars.sh -v9`
>
> I hate vcvars32.bat. And you are right about the crashes and unfriendly
> behavior, it's windows. Sorry.
>
>>
>> Compilation misdetects the Ant version and complains it is too old,
>> but works fine.
>
> "Compilation misdetects"? Can you run 'ant -version' from the command line?
>
> The ant startup scripts are a bit crufty on Windows, better in ant 1.7.1,
> but I have no idea what NetBeans has done with the Windows ant.bat or
> ant shell script startup files.
> On Windows, you usually need at least three things to get 'ant -version'
> to work: 1. ant in PATH, 2. ANT_HOME set, and 3. JAVA_HOME set.
> Unfortunately, when building the jdk, we don't allow JAVA_HOME to be
> set right now due to the potential build problems it can introduce.
> So getting 'ant -version' on windows has been tricky.
> With the older ant versions (like 1.6.*, this ant startup was worse).
>
> When ant is broken up and spread around, like on some Linux systems,
> it's hard to set ANT_HOME, and I had that problem with NetBeans, and
> on Windows, sometimes you just have to set ANT_HOME because the startup
> script can't seem to find where it is living. :^(
> So I always just install my own ant, makes life easier.
>

Indeed, my own GNU/Linux builds set ANT to /usr/bin/ant explicitly and
so does IcedTea.

>>
>> Even "make sanity" rapidly died as freetype_versioncheck didn't build.
>> http://mail.openjdk.java.net/pipermail/jdk6-dev/2008-March/000023.html
>> claims you need OPENJDK=true set, an undocumented fact that doesn't
>> help. After much effort I discovered
>> http://www.mail-archive.com/build-dev@openjdk.java.net/msg02319.html
>> where someone got it to work with Freetype 2.3.5 by copying
>> bin/freetype.dll into lib/freetype6.dll and applying a patch; I also
>> had to copy zlib1.dll into the build/windows-i586/btbins. Please at
>> least document the fact Freetype 2.3.4 is the last working version?
>
> Sigh... freetype... :^(  What a pain on windows.
> I'm going to file a bug on this, see if we can't clean up this dependency
> at least as far as building or setting up goes.
> I have been down this path, and finally published freetype binary bundles
> at http://cr.openjdk.java.net/~ohair/FreeType/
>
>>
>> It then compiled for a few minutes further and died in the corba/
>> subdirectory, with "COMPILER_PATH cannot be empty here". I found
>> http://mail.openjdk.java.net/pipermail/build-dev/2009-March/001767.html
>> and edited the file in question to fix RC (I had to use the full C:/
>> style path, the one from that email didn't work for me), unfortunately
>> it still didn't help, so I just commented out lines 96-98 and it
>> compiled much further.
>
> Usually when you get "COMPILER_PATH cannot be empty here", one of
>  * PATH is not setup correctly (according to vcvars32.bat)
>  * ALT_COMPILER_PATH (if you set it) is not set correctly
>  * The system environment variable VS90CMNTOOLS is not defined
>    or the Makefiles can't find the default location for the compilers.
>    If you ssh into a windows system, VS90CMNTOOLS will not be set.
>    If you installed in a non-standard location, that could be the issue.
>
> So I'm a little confused as how you could continue at this point and
> would like to understand that.
>
> Can you send me the "C:/Program Files/" directory where the
> cl.exe compiler file was found in your install?
>
>>
>> The problem repeated itself under the jdk/ directory, and I had to
>> edit another file and fix the path to RC and RCS.
>
> Yes, the path to RC/RCS is problematic, I think we have a bug on that.
> It appears to be a moving and sometimes replicated exe that moves
> with each release. :^(
>
>>
>> Then, still in jdk/, came missing headers. The first was afxres.h
>> which I got from mingw and copied into Visual Studio's include
>> directory. This got it to compile further.
>
> Humm... I thought we replaced a bunch of these afxres.h uses with
> just windows.h or winres.h. Ah... jkernel... new stuff.
>
>>
>> Next missing header was atlbase.h which lead me to discover an email
>>
>> (http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000654.html)
>> stating that the Visual Studio express editions are not supposed to
>> work (again, undocumented in README-builds.html). Indeed I found that
>> atlbase.h is part of ATL, which isn't part of any Visual Studio
>> express edition or the Platform SDK (6 or 7). Visual Studio is
>> expensive to buy just to compile OpenJDK. But am I really going to let
>> that stop me, after the previous 8 hour battle?
>
> Well, we can't document everything that doesn't work, but I understand
> your situation. We had been trying to allow for the Express compiler to
> work, but we could not guarantee it.
> At one point, I thought we were ATL #include free, but I see jkernel changes
> were added recently and re-introduced an atl dependency.
>
>>
>> $ grep atl * -iR | grep include
>> Binary file hotspot/.hg/store/data/src/share/vm/include_d_b__core.i
>> matches
>> jdk/make/jdk_generic_profile.sh:
>>  include4sdk="${vc7_root}/atlmfc/include"
>> jdk/make/jdk_generic_profile.sh:
>> include4sdk="${include4sdk};${platform_sdk}/Include/atl"
>> jdk/src/windows/native/sun/jkernel/kernel.cpp:#include <atlbase.h>
>> jdk/src/windows/native/sun/jkernel/kernel.cpp:#include <atlwin.h>
>> jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlbase.h>
>> jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlcom.h>
>> jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlwin.h>
>> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlbase.h>
>> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlcom.h>
>> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlwin.h>
>> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlhost.h>
>> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlbase.h>
>> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlcom.h>
>> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlwin.h>
>> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlhost.h>
>> jdk/src/windows/native/sun/jkernel/stdafx.cpp:#include <atlimpl.cpp>
>>
>
> I see this bug has been filed recently on this problem:
> http://bugs.sun.com/view_bug.do?bug_id=6903970
>
>> So in other words only jkernel uses the ATL for anything. Pity that it
>> uses it extensively, so there's no easy way to take ATL out as a
>> dependency. Someone should rewrite it without ATL: 4000 lines of code
>> shouldn't hold 7 million lines hostage.
>
> There may be a way to skip the jkernel build. Perhaps if you
> removed the jkernel from SUBDIRS in jdk/make/sun/Makefile?
> I have no idea if that would work, just a guess.
>
>>
>> But if ATL can't come out of jkernel, maybe jkernel can come out of
>> Java. I put in lots of #if 0 around the entire contents of all the
>> jkernel .cpp files, and made preJVMStartup() into a no-op.
>
> That's another way I suppose.
>
>>
>> It compiled further, complaining in at least one place about missing
>> /usr/bin/diff which isn't documented as a dependency, but luckily that
>> doesn't stop the build.
>
> What? Where did it complain about /usr/bin/diff? That's strange.
> You have one I assume, but Windows processes may not accept some
> cygwin files as being valid. Let me know if you still have
> the error message on this, I'd like to get to the bottom of this.
>
>>
>> When it got to compiling fonts, it broke again, this time looking for
>> freetype.dll instead of freetype6.dll. I copied the one to the other
>> and typed make with crossed fingers. After 20 minutes of compiling it
>> got to the fonts again and they compiled.
>>
>
> Sigh... freetype again... I'm glad you got it to compile, it
> should not be this hard. Sorry about that.
>
>> After a very long wait, it finally compiled successfully...
>>
>> ...and the hello world worked :-).
>
> Glad it was finally successful.
>
>>
>> Now I'm onto what I thought would be the hard part: the actual patch I
>> want to write :-).
>>
>> Patches I used to build it are attached.
>
> I'll try and make sure there are bugs filed on these issues, can't
> promise your exact patches can be used.
>
>>
>> Hope this helps someone.
>
> I'm sure it will.
>
> Now you can go get a little devil tatoo that says
>   "I built OpenJDK on Windows". ;^)
>
> -kto
>
>>
>> Regards
>> Damjan
>>
>



-- 
Andrew :-)

Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net

PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA  7927 142C 2591 94EF D9D8