Releases, releases, releases

[Andrew John Hughes]

Hi all,

I just want to post this while it's on my mind, IcedTea6 1.5 and
IcedTea7 1.10 having being released on Friday.  I think we need to
come up with a more clear process for releases than we have now, which
will hopefully avoid some of the confusion and tension that has arisen
recently.  As people regularly hacking on IcedTea are more than well
aware, IcedTea has grown far beyond its original designs of just being
a build infrastructure for OpenJDK and now incorporates a lot of
patches and build options.  It's been good to see lots of work going
on and input from various people across the community.  However,
people hacking on IcedTea will also be painfully aware of how long it
takes to do a full bootstrap build and the even longer length of time
it takes to run the jtreg tests.  As such, we really need to
prioritise which builds are tested as part of the release process and
which tests are run.  Of the top of my head, for IcedTea7 I think we
ended up testing:

* Intree build with default settings
* Intree build using --with-openjdk
* Intree build using --with-icedtea (I'm not convinced there is really
any need for a difference between this and openjdk, and I'd prefer we
drop one)
* Out-of-tree build with default settings
* Out-of-tree build using --with-icedtea
* Out-of-tree build with Zero
* Out-of-tree build with Shark
* Out-of-tree build with CACAO
* Out-of-tree build without the plugin

and probably some more we forgot.  By default settings, I tend to
include everything but things that specify paths to zips/dirs or
--with-parallel-jobs, which I always tend to use to speed up the
build.  However, these clearly will also affect things.
For clarity, an intree build is one where configure is run from the
checked out hg repo as ./configure.  An out-of-tree build is where
something like the following is done:

$ pwd
<blah>/icedtea
$ cd ..
$ mkdir icedtea-build
$ cd icedtea-build
$ ../icedtea/configure

so that the build directory is not the same as the source directory.
The advantage of this is cleanliness.  The build directory can easily
be removed to ensure new builds are always clean.  make distcheck
always does such a build, and I ensure this works before making a
release. It notably also tests that the build works when the source
directory is read-only, which required some changes to IcedTea that
may not have propagated from 7 to 6 (I'm as guilty for not testing 6
as others are for not testing 7 regularly).  For anyone interested, I
maintain my build scripts in a public Mercurial repository at
http://fuseyism.com/hg/scripts.  This does an out-of-directory build
and will do zero/shark/cacao builds automatically depending on the
name of the script that is actually run (e.g. symlink icedtea.sh to
cacao.sh, do ./cacao.sh and you get a cacao build).

This is clearly a long and tedious list for testing on every change so
we need to narrow down which ones are most release-critical and which
features really need to be tested, which are useful but not critical.
For the IcedTea7 release, we decided not to make the CACAO build
release-critical due to a bug
(http://server.complang.tuwien.ac.at/cgi-bin/bugzilla/show_bug.cgi?id=128)
that at least myself and one of the CACAO devs (Ringding on
#cacao at FreeNode) have reproduced.  I think something like that needs
regular testing, especially for 7, and support from upstream.

As to JTReg, I don't think either 6 or 7 yet have a clean run (i.e.
100% pass). I've tended to give less regards to results for 7 after
finding some tests wouldn't even compile, having been broken by API
changes (see https://bugs.openjdk.java.net/show_bug.cgi?id=100012 for
example).  If we're going to run this before releases, we need a clear
baseline so we can identify regressions, and ideally continuous
testing so we can spot regressions as soon as they happen.

As to the actual release process, I think we cleared most of it up but
for clarity, we now propose a release at least a week in advance and a
branch is created for release work once tip is considered stable
enough to undergo pre-release testing.  This branch can then also be
used for minor releases, while tip remains open for new major release
features.

If we have a clear release procedure, I think it will be easier for
everyone to collaborate on future releases and ensure a smoother and
more hassle-free process.   Clear goals for a release and easy
delineation of work are thus the order of the day.

-- 
Andrew :-)

Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net

PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA  7927 142C 2591 94EF D9D8