OpenJDK successfully built with Visual Studio express edition

[Kelly O'Hair]

Damjan Jovanovic wrote:
> Hi
> 
> I'm documenting my (unbelievably difficult but ultimately) successful
> experience of building OpenJDK on Windows with Visual Studio express
> edition.

Thanks for sending this out. You do deserve some kind of medal, building
on Windows is very difficult.

First off, where and how did you get the sources?
It's not exactly clear what sources we are dealing with here.

> 
> I built the awt tree from about a week ago, with Windows Vista, Cygwin
> 1.5.25-15, Microsoft Visual Studio 2008 Express Edition SP1, Freetype
> 2.3.5-1, jdk-7-ea-plug-b77-windows-i586-03_dec_2009.jar, DirectX SDK
> as per README-builds.html, Ant copied from Netbeans 6.7, and make from
> the cmake.org website (as per README-builds.html). The import JDK was
> 1.6.0.

So the devil is in the details, but first off, you have picked a few things
that differ from our recommended set of things:

   * Windows Vista - My condolences, but I don't know of anyone using Vista
     to build. I know Windows 2000 is almost 10 years old, and Windows XP is
     pretty old, but XP is the newest OS we can recommend. Anything newer is
     in the 'on your own' category. But I am glad you were ultimately successful,
     at least we know it is possible.

   * Microsoft Visual Studio 2008 Express Edition SP1 - We recommend the
     purchased copy, but we understand why you are using this version and
     we would like to see it work, but it's the ATL issue that is the killer.
     Whatever adjustments we can make to allow it's use we will do.

   * Freetype (on windows) - A royal pain in my side. More later.

   * The jdk-7-ea-plug-b77-windows-i586-03_dec_2009.jar should not be needed
     anymore for a successful build. I would skip it. (Binary plugs are optional).

   * DirectX SDK - Which version did you use? Just curious.

   * Ant copied from Netbeans 6.7 - If that worked, fine. But I would really
     rather people download and install the official 1.7.1 version of ant.
     NetBeans may be sifting out parts of ant in it's install, so using the
     ant from NetBeans may not be the same behavior as from an official ant
     install.  But if you got it to work, that's great.

   * Was the cmake you got 3.81? Just curious.

> 
> Firstly the README-builds.html talks a lot about path problems,
> without offering any good solutions. After much effort, I discovered
> that you have to use /cygpath/c paths only in $PATH and use double
> quote quoted C:/ style paths everywhere else (eg. export
> WindowsSdkDir="C:/Program Files/Microsoft SDKs/Windows/v6.0A/").

If this isn't clear I'll try and improve it.
By the way, the Microsoft SDK should probably be v6.1, but I don't
think it will matter much. You did not list installing the Microsoft SDK
above, did you explicitly install v6.0A or did that come with the Visual
Studio compiler install?

> 
> Microsoft's quality C compilers apparently need environment variables
> set before they work properly, otherwise they crash (not complain, but
> actually crash). The vcvars32.bat doesn't work from inside cygwin, so
> I had to run "set > a.txt", then run "vcvars32.bat", then "set >
> b.txt", and finally diff a.txt with b.txt to find out what these
> mysterious variables were, then convert them into C:/ and /cygpath/c
> style paths that get set in cygwin. Running
> jdk/make/jdk_generic_profile.sh loses the variables again, so I
> avoided it.

Sigh...  I should change jdk/make/jdk_generic_profile.sh, it was never
made cygwin compliant. That file was mainly meant for documentation
purposes, not sure how I feel about people using it as part of their
build process, :^(

On vcvars32.bat, I have created a shell script called vsvars.sh
(See http://blogs.sun.com/kto/entry/windows_visual_studio_stupid_pet)
that I plan on adding to the jdk repository at some point.
It will (from a cygwin shell) effectively do the vcvars32.bat for
you and set PATH correctly. All you need to do is run
   eval `vsvars.sh -v9`

I hate vcvars32.bat. And you are right about the crashes and unfriendly
behavior, it's windows. Sorry.

> 
> Compilation misdetects the Ant version and complains it is too old,
> but works fine.

"Compilation misdetects"? Can you run 'ant -version' from the command line?

The ant startup scripts are a bit crufty on Windows, better in ant 1.7.1,
but I have no idea what NetBeans has done with the Windows ant.bat or
ant shell script startup files.
On Windows, you usually need at least three things to get 'ant -version'
to work: 1. ant in PATH, 2. ANT_HOME set, and 3. JAVA_HOME set.
Unfortunately, when building the jdk, we don't allow JAVA_HOME to be
set right now due to the potential build problems it can introduce.
So getting 'ant -version' on windows has been tricky.
With the older ant versions (like 1.6.*, this ant startup was worse).

When ant is broken up and spread around, like on some Linux systems,
it's hard to set ANT_HOME, and I had that problem with NetBeans, and
on Windows, sometimes you just have to set ANT_HOME because the startup
script can't seem to find where it is living. :^(
So I always just install my own ant, makes life easier.

> 
> Even "make sanity" rapidly died as freetype_versioncheck didn't build.
> http://mail.openjdk.java.net/pipermail/jdk6-dev/2008-March/000023.html
> claims you need OPENJDK=true set, an undocumented fact that doesn't
> help. After much effort I discovered
> http://www.mail-archive.com/build-dev@openjdk.java.net/msg02319.html
> where someone got it to work with Freetype 2.3.5 by copying
> bin/freetype.dll into lib/freetype6.dll and applying a patch; I also
> had to copy zlib1.dll into the build/windows-i586/btbins. Please at
> least document the fact Freetype 2.3.4 is the last working version?

Sigh... freetype... :^(  What a pain on windows.
I'm going to file a bug on this, see if we can't clean up this dependency
at least as far as building or setting up goes.
I have been down this path, and finally published freetype binary bundles
at http://cr.openjdk.java.net/~ohair/FreeType/

> 
> It then compiled for a few minutes further and died in the corba/
> subdirectory, with "COMPILER_PATH cannot be empty here". I found
> http://mail.openjdk.java.net/pipermail/build-dev/2009-March/001767.html
> and edited the file in question to fix RC (I had to use the full C:/
> style path, the one from that email didn't work for me), unfortunately
> it still didn't help, so I just commented out lines 96-98 and it
> compiled much further.

Usually when you get "COMPILER_PATH cannot be empty here", one of
   * PATH is not setup correctly (according to vcvars32.bat)
   * ALT_COMPILER_PATH (if you set it) is not set correctly
   * The system environment variable VS90CMNTOOLS is not defined
     or the Makefiles can't find the default location for the compilers.
     If you ssh into a windows system, VS90CMNTOOLS will not be set.
     If you installed in a non-standard location, that could be the issue.

So I'm a little confused as how you could continue at this point and
would like to understand that.

Can you send me the "C:/Program Files/" directory where the
cl.exe compiler file was found in your install?

> 
> The problem repeated itself under the jdk/ directory, and I had to
> edit another file and fix the path to RC and RCS.

Yes, the path to RC/RCS is problematic, I think we have a bug on that.
It appears to be a moving and sometimes replicated exe that moves
with each release. :^(

> 
> Then, still in jdk/, came missing headers. The first was afxres.h
> which I got from mingw and copied into Visual Studio's include
> directory. This got it to compile further.

Humm... I thought we replaced a bunch of these afxres.h uses with
just windows.h or winres.h. Ah... jkernel... new stuff.

> 
> Next missing header was atlbase.h which lead me to discover an email
> (http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000654.html)
> stating that the Visual Studio express editions are not supposed to
> work (again, undocumented in README-builds.html). Indeed I found that
> atlbase.h is part of ATL, which isn't part of any Visual Studio
> express edition or the Platform SDK (6 or 7). Visual Studio is
> expensive to buy just to compile OpenJDK. But am I really going to let
> that stop me, after the previous 8 hour battle?

Well, we can't document everything that doesn't work, but I understand
your situation. We had been trying to allow for the Express compiler to
work, but we could not guarantee it.
At one point, I thought we were ATL #include free, but I see jkernel changes
were added recently and re-introduced an atl dependency.

> 
> $ grep atl * -iR | grep include
> Binary file hotspot/.hg/store/data/src/share/vm/include_d_b__core.i matches
> jdk/make/jdk_generic_profile.sh:    include4sdk="${vc7_root}/atlmfc/include"
> jdk/make/jdk_generic_profile.sh:
> include4sdk="${include4sdk};${platform_sdk}/Include/atl"
> jdk/src/windows/native/sun/jkernel/kernel.cpp:#include <atlbase.h>
> jdk/src/windows/native/sun/jkernel/kernel.cpp:#include <atlwin.h>
> jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlbase.h>
> jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlcom.h>
> jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlwin.h>
> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlbase.h>
> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlcom.h>
> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlwin.h>
> jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlhost.h>
> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlbase.h>
> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlcom.h>
> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlwin.h>
> jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlhost.h>
> jdk/src/windows/native/sun/jkernel/stdafx.cpp:#include <atlimpl.cpp>
> 

I see this bug has been filed recently on this problem:
http://bugs.sun.com/view_bug.do?bug_id=6903970

> So in other words only jkernel uses the ATL for anything. Pity that it
> uses it extensively, so there's no easy way to take ATL out as a
> dependency. Someone should rewrite it without ATL: 4000 lines of code
> shouldn't hold 7 million lines hostage.

There may be a way to skip the jkernel build. Perhaps if you
removed the jkernel from SUBDIRS in jdk/make/sun/Makefile?
I have no idea if that would work, just a guess.

> 
> But if ATL can't come out of jkernel, maybe jkernel can come out of
> Java. I put in lots of #if 0 around the entire contents of all the
> jkernel .cpp files, and made preJVMStartup() into a no-op.

That's another way I suppose.

> 
> It compiled further, complaining in at least one place about missing
> /usr/bin/diff which isn't documented as a dependency, but luckily that
> doesn't stop the build.

What? Where did it complain about /usr/bin/diff? That's strange.
You have one I assume, but Windows processes may not accept some
cygwin files as being valid. Let me know if you still have
the error message on this, I'd like to get to the bottom of this.

> 
> When it got to compiling fonts, it broke again, this time looking for
> freetype.dll instead of freetype6.dll. I copied the one to the other
> and typed make with crossed fingers. After 20 minutes of compiling it
> got to the fonts again and they compiled.
> 

Sigh... freetype again... I'm glad you got it to compile, it
should not be this hard. Sorry about that.

> After a very long wait, it finally compiled successfully...
> 
> ...and the hello world worked :-).

Glad it was finally successful.

> 
> Now I'm onto what I thought would be the hard part: the actual patch I
> want to write :-).
> 
> Patches I used to build it are attached.

I'll try and make sure there are bugs filed on these issues, can't
promise your exact patches can be used.

> 
> Hope this helps someone.

I'm sure it will.

Now you can go get a little devil tatoo that says
    "I built OpenJDK on Windows". ;^)

-kto

> 
> Regards
> Damjan
>