Releases, releases, releases

[Andrew Haley]

Andrew John Hughes wrote:

> This is clearly a long and tedious list for testing on every change so
> we need to narrow down which ones are most release-critical and which
> features really need to be tested, which are useful but not critical.
> For the IcedTea7 release, we decided not to make the CACAO build
> release-critical due to a bug
> (http://server.complang.tuwien.ac.at/cgi-bin/bugzilla/show_bug.cgi?id=128)
> that at least myself and one of the CACAO devs (Ringding on
> #cacao at FreeNode) have reproduced.  I think something like that needs
> regular testing, especially for 7, and support from upstream.

Yes.  Cacao is really the Cacao maintainers' job to keep working: others
are in no position to debug it when it breaks.

At a minimum we need to test Zero, x86/32, and x86/64.  We may not
need to test Zero on a wide range of architectures; that's up for
discussion.

Re in-tree/out-of-tree builds: I'm not sure that we need to support
(and therefore test) both.  I think we should stick as close as possible
to the way Sun work upstream, which AFAIK is in-tree.  I am quite
aware of the advantages of building otside the source tree, but IMO
the disadvantages of building in a way that's not commonly used
upstream outweigh them.  What do other people think?

> As to JTReg, I don't think either 6 or 7 yet have a clean run (i.e.
> 100% pass). I've tended to give less regards to results for 7 after
> finding some tests wouldn't even compile, having been broken by API
> changes (see https://bugs.openjdk.java.net/show_bug.cgi?id=100012 for
> example).  If we're going to run this before releases, we need a clear
> baseline so we can identify regressions, and ideally continuous
> testing so we can spot regressions as soon as they happen.

OK.

> As to the actual release process, I think we cleared most of it up but
> for clarity, we now propose a release at least a week in advance and a
> branch is created for release work once tip is considered stable
> enough to undergo pre-release testing.  This branch can then also be
> used for minor releases, while tip remains open for new major release
> features.

That is standard practice in most of the free software projects I know
about.  We need to ask whether a week's notice is enough, though.  There
are a great many Debian platforms that take a long time to build and test.

> If we have a clear release procedure, I think it will be easier for
> everyone to collaborate on future releases and ensure a smoother and
> more hassle-free process.   Clear goals for a release and easy
> delineation of work are thus the order of the day.

I think the basic release process is now pretty good, but it would be
better if it were described online.  A Wiki page detailing the process
can be discussed and edited by all IcedTea stakeholders.  So, if someone
thinks that testing on a particular platform in a particular way is
needed, they can propose it on the list and add it to the Wiki page
when agreed.  I must emphasize that anyone insisting that Platform X
must be tested is in effect promising to do the testing.

The Wiki page should also detail all known JTreg failures.

Andrew.