A little progress report on Mercurial and build/source infrastructure changes

[Kelly O'Hair]

Re: A little progress report on Mercurial and build/source infrastructure changes

I thought I would take some time to list a few of the up and coming changes
with regards to Open JDK builds and source structure.
I don't want anyone blogging that we are keeping secrets from everyone. ;^)

We are converting OpenJDK to Mercurial but it's taking longer than I had planned.
There is a long list of issues we are trying to clear up internally, some in
the area of the keeping the legal eagles happy, and others just major cleanups
we want to do before starting out in a fresh new world of Mercurial.
We have used TeamWare for over 10 years (or longer?), so internally this is a
huge change for the JDK developers, and the fact that we will be externally
exposing our Mercurial repositories makes us nervous and very careful.

To anyone downloading source bundles, this impact should be minimal, but
the Mercurial conversion will cause many files to get moved around.
In the process of converting to Mercurial there will also be some basic build
infrastructure changes. So fair warning, things will be changing.

Internally we deal with many TeamWare workspaces, the big 2 primary ones
have historically been j2se and hotspot. These 2 workspaces do contain some
of what we are calling "closed" sources. The goal is to minimize these closed parts
of the workspaces, either by opening these files up where possible, or replacing
them. That will take time and will happen over the long term, perhaps faster
once we get public Mercurial repositories out in the open.

The current plan is that we would continue to keep components like hotspot
in a separate repository, and we are currently looking at separating out parts
of the existing j2se workspace into new repositories where it makes sense.
You may see a "langtools" repository that contains just javac/javah/javadoc/apt,
and you may see an "ee" repository that contains much of the Java EE contribution
to the OpenJDK. These two have been chosen because they are large chunks of the j2se
files that get delivered to more than just the JDK, and can be built and run with
just the BOOTJDK (no dependence on JDK7 apis or a running JDK7).

The build procedure becomes much simplier once the "langtools" get separated out.
e.g. (just a prediction)
    1. Build langtools with BOOTJDK
    2. Build hotspot with langtools javac and BOOTJDK and C/C++ compilers
    3. Build ee with langtools javac and BOOTJDK
    4. Build the rest of the j2se with ee, hotspot, langtools javac and BOOTJDK and C/C++ compilers
    5. Repeat 1-4 with this new JDK7 (which hasn't run yet) as the BOOTJDK

In step 4 one of the big changes will be that we won't need to run what we are building
as it happens now, that would be done in step 5. This should make cross development
easier, and certainly simplifies the build process from my perspective.

The "rest of j2se" and the current "control" files may become the primary "jdk"
repository, and the hotspot, langtools, and ee repositories may be nested inside
this jdk repository as nested repositories (TeamWare doesn't allow nesting, but
Mercurial does).
Each repository should be rather independent in terms of developers.

Many people have suggested breaking up the j2se even further, but we won't be
doing that initially. There are some serious issues with regards to the JDK7
runtime dependencies that needs to be taken into account with each separated
component.

Don't expect this to happen overnight, the current internal transition to
Mercurial is scheduled for roughly Oct2007, and our dates have slipped once already.
We have made great progress on the "langtools" separation and are now getting
started on the "ee" separation.

Many other smaller to medium sized details remain to be dealt with, but we are
progressing.

I hope this helps explain the current state. Now back to work. ;^)

-kto