OpenJDK successfully built with Visual Studio express edition

[Kelly O'Hair]

Andrew John Hughes wrote:
> 2009/12/10 Kelly O'Hair <Kelly.Ohair at sun.com>:
>> Damjan Jovanovic wrote:
>>> Hi
>>>
>>> I'm documenting my (unbelievably difficult but ultimately) successful
>>> experience of building OpenJDK on Windows with Visual Studio express
>>> edition.
>> Thanks for sending this out. You do deserve some kind of medal, building
>> on Windows is very difficult.
>>
>> First off, where and how did you get the sources?
>> It's not exactly clear what sources we are dealing with here.
>>
> 
> Errr... he says in the next paragraph 'the awt tree from about a week
> ago' i.e. http://hg.openjdk.java.net/jdk7/awt

Ah.. You are right. When I saw "awt tree" I was thinking the awt sources
in some jdk tree.... :^(  Clearly it is the awt forest.

> 
> My congratulations too on the successful build.  I wouldn't even want
> to contemplate this...
> 
>>> I built the awt tree from about a week ago, with Windows Vista, Cygwin
>>> 1.5.25-15, Microsoft Visual Studio 2008 Express Edition SP1, Freetype
>>> 2.3.5-1, jdk-7-ea-plug-b77-windows-i586-03_dec_2009.jar, DirectX SDK
>>> as per README-builds.html, Ant copied from Netbeans 6.7, and make from
>>> the cmake.org website (as per README-builds.html). The import JDK was
>>> 1.6.0.
>> So the devil is in the details, but first off, you have picked a few things
>> that differ from our recommended set of things:
>>
>>  * Windows Vista - My condolences, but I don't know of anyone using Vista
>>    to build. I know Windows 2000 is almost 10 years old, and Windows XP is
>>    pretty old, but XP is the newest OS we can recommend. Anything newer is
>>    in the 'on your own' category. But I am glad you were ultimately
>> successful,
>>    at least we know it is possible.
>>
>>  * Microsoft Visual Studio 2008 Express Edition SP1 - We recommend the
>>    purchased copy, but we understand why you are using this version and
>>    we would like to see it work, but it's the ATL issue that is the killer.
>>    Whatever adjustments we can make to allow it's use we will do.
>>
>>  * Freetype (on windows) - A royal pain in my side. More later.
>>
>>  * The jdk-7-ea-plug-b77-windows-i586-03_dec_2009.jar should not be needed
>>    anymore for a successful build. I would skip it. (Binary plugs are
>> optional).
>>
> 
> Reminds me we need to forwardport the doc. update for this.  OpenJDK6
> lists them as optional, but 7 doesn't.

I added a note to 6896978 for jdk7 to fix this. You are right.

> 
>>  * DirectX SDK - Which version did you use? Just curious.
>>
>>  * Ant copied from Netbeans 6.7 - If that worked, fine. But I would really
>>    rather people download and install the official 1.7.1 version of ant.
>>    NetBeans may be sifting out parts of ant in it's install, so using the
>>    ant from NetBeans may not be the same behavior as from an official ant
>>    install.  But if you got it to work, that's great.
>>
> 
> This is especially true with the recent jaxp/jaxws changes which rely
> on the regexp extension.  I know that's not in the base package on
> most GNU/Linux distros.

I added a note to 6896978 for jdk7 on this. We should be clear about this.

You could spend a lifetime keeping these READMEs up-to-date. :^(

> 
>>  * Was the cmake you got 3.81? Just curious.
>>
>>> Firstly the README-builds.html talks a lot about path problems,
>>> without offering any good solutions. After much effort, I discovered
>>> that you have to use /cygpath/c paths only in $PATH and use double
>>> quote quoted C:/ style paths everywhere else (eg. export
>>> WindowsSdkDir="C:/Program Files/Microsoft SDKs/Windows/v6.0A/").
>> If this isn't clear I'll try and improve it.
>> By the way, the Microsoft SDK should probably be v6.1, but I don't
>> think it will matter much. You did not list installing the Microsoft SDK
>> above, did you explicitly install v6.0A or did that come with the Visual
>> Studio compiler install?
>>
>>> Microsoft's quality C compilers apparently need environment variables
>>> set before they work properly, otherwise they crash (not complain, but
>>> actually crash). The vcvars32.bat doesn't work from inside cygwin, so
>>> I had to run "set > a.txt", then run "vcvars32.bat", then "set >
>>> b.txt", and finally diff a.txt with b.txt to find out what these
>>> mysterious variables were, then convert them into C:/ and /cygpath/c
>>> style paths that get set in cygwin. Running
>>> jdk/make/jdk_generic_profile.sh loses the variables again, so I
>>> avoided it.
>> Sigh...  I should change jdk/make/jdk_generic_profile.sh, it was never
>> made cygwin compliant. That file was mainly meant for documentation
>> purposes, not sure how I feel about people using it as part of their
>> build process, :^(
>>
>> On vcvars32.bat, I have created a shell script called vsvars.sh
>> (See http://blogs.sun.com/kto/entry/windows_visual_studio_stupid_pet)
>> that I plan on adding to the jdk repository at some point.
>> It will (from a cygwin shell) effectively do the vcvars32.bat for
>> you and set PATH correctly. All you need to do is run
>>  eval `vsvars.sh -v9`
>>
>> I hate vcvars32.bat. And you are right about the crashes and unfriendly
>> behavior, it's windows. Sorry.
>>
>>> Compilation misdetects the Ant version and complains it is too old,
>>> but works fine.
>> "Compilation misdetects"? Can you run 'ant -version' from the command line?
>>
>> The ant startup scripts are a bit crufty on Windows, better in ant 1.7.1,
>> but I have no idea what NetBeans has done with the Windows ant.bat or
>> ant shell script startup files.
>> On Windows, you usually need at least three things to get 'ant -version'
>> to work: 1. ant in PATH, 2. ANT_HOME set, and 3. JAVA_HOME set.
>> Unfortunately, when building the jdk, we don't allow JAVA_HOME to be
>> set right now due to the potential build problems it can introduce.
>> So getting 'ant -version' on windows has been tricky.
>> With the older ant versions (like 1.6.*, this ant startup was worse).
>>
>> When ant is broken up and spread around, like on some Linux systems,
>> it's hard to set ANT_HOME, and I had that problem with NetBeans, and
>> on Windows, sometimes you just have to set ANT_HOME because the startup
>> script can't seem to find where it is living. :^(
>> So I always just install my own ant, makes life easier.
>>
> 
> Indeed, my own GNU/Linux builds set ANT to /usr/bin/ant explicitly and
> so does IcedTea.

The problem I've had with Linux builds is that depending on the Linux,
you might have a different version of ant. Some have newer ones than
others. Most work, but I just find it easier to have my own little
ant farm on all platforms. ;^)

I also work on JavaFX and it has a requirement of ant 1.7.1 because
it uses some NetBeans ant project files for building, and I think
NetBeans 6.7 started requiring ant 1.7.1, which is no sweat for NetBeans
itself because it ships it's own ant, but building outside of NetBeans
means you need ant 1.7.1 in your PATH. Oh well...

What a world ...  what a world... http://www.youtube.com/watch?v=qfV_ENR5IZE

-kto


> 
>>> Even "make sanity" rapidly died as freetype_versioncheck didn't build.
>>> http://mail.openjdk.java.net/pipermail/jdk6-dev/2008-March/000023.html
>>> claims you need OPENJDK=true set, an undocumented fact that doesn't
>>> help. After much effort I discovered
>>> http://www.mail-archive.com/build-dev@openjdk.java.net/msg02319.html
>>> where someone got it to work with Freetype 2.3.5 by copying
>>> bin/freetype.dll into lib/freetype6.dll and applying a patch; I also
>>> had to copy zlib1.dll into the build/windows-i586/btbins. Please at
>>> least document the fact Freetype 2.3.4 is the last working version?
>> Sigh... freetype... :^(  What a pain on windows.
>> I'm going to file a bug on this, see if we can't clean up this dependency
>> at least as far as building or setting up goes.
>> I have been down this path, and finally published freetype binary bundles
>> at http://cr.openjdk.java.net/~ohair/FreeType/
>>
>>> It then compiled for a few minutes further and died in the corba/
>>> subdirectory, with "COMPILER_PATH cannot be empty here". I found
>>> http://mail.openjdk.java.net/pipermail/build-dev/2009-March/001767.html
>>> and edited the file in question to fix RC (I had to use the full C:/
>>> style path, the one from that email didn't work for me), unfortunately
>>> it still didn't help, so I just commented out lines 96-98 and it
>>> compiled much further.
>> Usually when you get "COMPILER_PATH cannot be empty here", one of
>>  * PATH is not setup correctly (according to vcvars32.bat)
>>  * ALT_COMPILER_PATH (if you set it) is not set correctly
>>  * The system environment variable VS90CMNTOOLS is not defined
>>    or the Makefiles can't find the default location for the compilers.
>>    If you ssh into a windows system, VS90CMNTOOLS will not be set.
>>    If you installed in a non-standard location, that could be the issue.
>>
>> So I'm a little confused as how you could continue at this point and
>> would like to understand that.
>>
>> Can you send me the "C:/Program Files/" directory where the
>> cl.exe compiler file was found in your install?
>>
>>> The problem repeated itself under the jdk/ directory, and I had to
>>> edit another file and fix the path to RC and RCS.
>> Yes, the path to RC/RCS is problematic, I think we have a bug on that.
>> It appears to be a moving and sometimes replicated exe that moves
>> with each release. :^(
>>
>>> Then, still in jdk/, came missing headers. The first was afxres.h
>>> which I got from mingw and copied into Visual Studio's include
>>> directory. This got it to compile further.
>> Humm... I thought we replaced a bunch of these afxres.h uses with
>> just windows.h or winres.h. Ah... jkernel... new stuff.
>>
>>> Next missing header was atlbase.h which lead me to discover an email
>>>
>>> (http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000654.html)
>>> stating that the Visual Studio express editions are not supposed to
>>> work (again, undocumented in README-builds.html). Indeed I found that
>>> atlbase.h is part of ATL, which isn't part of any Visual Studio
>>> express edition or the Platform SDK (6 or 7). Visual Studio is
>>> expensive to buy just to compile OpenJDK. But am I really going to let
>>> that stop me, after the previous 8 hour battle?
>> Well, we can't document everything that doesn't work, but I understand
>> your situation. We had been trying to allow for the Express compiler to
>> work, but we could not guarantee it.
>> At one point, I thought we were ATL #include free, but I see jkernel changes
>> were added recently and re-introduced an atl dependency.
>>
>>> $ grep atl * -iR | grep include
>>> Binary file hotspot/.hg/store/data/src/share/vm/include_d_b__core.i
>>> matches
>>> jdk/make/jdk_generic_profile.sh:
>>>  include4sdk="${vc7_root}/atlmfc/include"
>>> jdk/make/jdk_generic_profile.sh:
>>> include4sdk="${include4sdk};${platform_sdk}/Include/atl"
>>> jdk/src/windows/native/sun/jkernel/kernel.cpp:#include <atlbase.h>
>>> jdk/src/windows/native/sun/jkernel/kernel.cpp:#include <atlwin.h>
>>> jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlbase.h>
>>> jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlcom.h>
>>> jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlwin.h>
>>> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlbase.h>
>>> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlcom.h>
>>> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlwin.h>
>>> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlhost.h>
>>> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlbase.h>
>>> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlcom.h>
>>> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlwin.h>
>>> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlhost.h>
>>> jdk/src/windows/native/sun/jkernel/stdafx.cpp:#include <atlimpl.cpp>
>>>
>> I see this bug has been filed recently on this problem:
>> http://bugs.sun.com/view_bug.do?bug_id=6903970
>>
>>> So in other words only jkernel uses the ATL for anything. Pity that it
>>> uses it extensively, so there's no easy way to take ATL out as a
>>> dependency. Someone should rewrite it without ATL: 4000 lines of code
>>> shouldn't hold 7 million lines hostage.
>> There may be a way to skip the jkernel build. Perhaps if you
>> removed the jkernel from SUBDIRS in jdk/make/sun/Makefile?
>> I have no idea if that would work, just a guess.
>>
>>> But if ATL can't come out of jkernel, maybe jkernel can come out of
>>> Java. I put in lots of #if 0 around the entire contents of all the
>>> jkernel .cpp files, and made preJVMStartup() into a no-op.
>> That's another way I suppose.
>>
>>> It compiled further, complaining in at least one place about missing
>>> /usr/bin/diff which isn't documented as a dependency, but luckily that
>>> doesn't stop the build.
>> What? Where did it complain about /usr/bin/diff? That's strange.
>> You have one I assume, but Windows processes may not accept some
>> cygwin files as being valid. Let me know if you still have
>> the error message on this, I'd like to get to the bottom of this.
>>
>>> When it got to compiling fonts, it broke again, this time looking for
>>> freetype.dll instead of freetype6.dll. I copied the one to the other
>>> and typed make with crossed fingers. After 20 minutes of compiling it
>>> got to the fonts again and they compiled.
>>>
>> Sigh... freetype again... I'm glad you got it to compile, it
>> should not be this hard. Sorry about that.
>>
>>> After a very long wait, it finally compiled successfully...
>>>
>>> ...and the hello world worked :-).
>> Glad it was finally successful.
>>
>>> Now I'm onto what I thought would be the hard part: the actual patch I
>>> want to write :-).
>>>
>>> Patches I used to build it are attached.
>> I'll try and make sure there are bugs filed on these issues, can't
>> promise your exact patches can be used.
>>
>>> Hope this helps someone.
>> I'm sure it will.
>>
>> Now you can go get a little devil tatoo that says
>>   "I built OpenJDK on Windows". ;^)
>>
>> -kto
>>
>>> Regards
>>> Damjan
>>>
> 
> 
>