JDK8 Preliminary Repository Layout

Tue Mar 15 22:07:21 UTC 2011

On 09:14 Thu 10 Mar     , Kelly O'Hair wrote:
> 
> On Mar 9, 2011, at 4:48 PM, Dr Andrew John Hughes wrote:
> 
> >> 
> >> Other ideas were considered:
> >>  * Folding jaxp/jaxws into the root or jdk8/jdk repo
> > 
> > Sounds good.  jdk8/jdk would make more sense as jaxws depends on some classes that are in the jdk
> > tree (com.sun.net.httpserver) and we could then get rid of the Ant junk.  It would be good to have
> > the code there too so we can once again track changes.  The recent security issue was a nightmare
> > due to the unavailability of the source code.
> 
> Humm "Ant junk" ok...  I'm not in love with it either, but it does provide a very fast build of simple pure java code.

I haven't seen much noticeable difference and one would have thought that loading just javac would be quicker than
loading javac+ant classes+xml file parsing.

If you're comparing with the JDK build, then it's a bad comparison as it's designed so that javac is invoked
many times, sometimes on just a couple of files.  As a result, HotSpot never really gets to kick in.  To compare
like-with-like, you'd need to build all the JDK java files with one javac invocation, which would also avoid
some classes being built against bootstrap classes.

> But having separated out jaxp/jaxws/corba from the rest of the jdk, we are not considering folding it all
> back in. The consideration was moving the use of the drop bundle, not forking the source again.
> The separation of these sources from the basic jdk has been a major benefit to the developers and was also meant
> to ease the fork issues of this code, that is and has been maintained as separate open source projects and
> delivered into multiple projects, licensed differently, and re-packaged in different ways.
> The jdk is downstream on these sources.
> There will be a separate discussion on this but it will focus on improving the source drop complications.
> 

So where are the upstreams?  What is in these tarballs?

The problem is that you can't track down bugs because we just move from one random snapshot to another
with no history.  We need more transparency in this area.

> > 
> >>  * Separating out the jdk demos from the jdk repo to a separate "demos" repository
> > 
> > Makes sense.  Having an option to disable them would be a good first step.
> 
> A 'do not build demos' option would be a fine RFE. Maybe do some preliminary surgery to at least
> isolate the sources that are strictly 'demo or sample' material.  IF someone has the time. ;^)
> 
> > 
> >>  * Separating out the client (awt/swing/etc) code from the jdk repo into a separate repo
> > 
> > Why would we want to do this?  IME, there are lots of interdependencies with the other code and
> > this would make the build a nightmare.
> 
> It was suggested and talked about a little, nobody stepped forward to champion it.
> I assume it would be done to categorize the sources and make the developers lives easier
> when focusing on their own area.
> 

While making the build harder.

> > 
> >>  * Updating the corba sources changing it to an ant build
> > 
> > Please, no!  These Ant builds are already a nightmare and an order of magniture harder to debug than
> > the Makefiles.  Why you want to require a build system that requires an entire JDK setup (possibly
> > bringing another into the mix beyond the bootstrap JDK) is beyond me.
> > 
> 
> Ok. I had no idea is was that bad. Well, yes, ant script logging and debugging is a bit bad, and the syntax
> is a little ...ah... bizarre... don't want to insult any xml people... :^(  The pains with ant scripts seemed worth
> it with regards to the speed it can compile/zip/jar all in one VM process. And of course the potential NetBeans
> IDE usage benefits, just ask the langtools development team.
> The current corba makefile build can take 10-15 minutes on some systems, and I would estimate that an ant build could
> cut that in half or maybe a quarter of that time. Especially on Windows systems which aren't exactly exec demons.
> 

I don't know what the Oracle setup is but I would guess (especially from the '/java' stuff and other defaults)
that there is one well-defined setup.  I also guess there's only ever Oracle JDKs in the mix.

Out in the real world, there are all manner of possible Ant setups and they could be using any JDK to run on.
At one point, I think Kaffe got pulled into an OpenJDK build I was doing due to Ant.  In short, it brings in
a heap of dependencies and setup variances that just using make (which is required anyway) doesn't.

I guess all these problems come down to the fact that everything is designed around known sets of developers
with largely homogenous requirements rather than being designed for the plethora of possibilities that could
arise due to this now being a Free Software code base.  The numerous build questions on the list suggest
people don't always get too far or find it too easy.

> But regardless, this won't be happening, corba will remain 'as is' unless we decide to upgrade the entire corba
> source base, in which case I would advocate using the default build mechanism recommended and used by the
> Corba team. I would like to defer to the development team that has to work on this code and the upstream
> project that has what is considered the master sources.
> 
> > What would make much more sense is to just add corba into the jdk tree.  It actually requires sun.tools
> > classes to compile rmic anyway so it would make much more sense there.
> 
> No, we won't be doing that as far as I know. Unless the Corba team wants to lobby for that, and I suspect they won't.
> 

The problem is that they are still all interdependent.  CORBA may be stored separately but it requires
classes in the JDK tree.

I finally managed to get it all building yesterday and the changes required are shown by the patches in this
changeset:

http://icedtea.classpath.org/hg/icedtea/rev/0811a305cdb3

> > 
> > Hey, I'd just make it all one repository as they all interdepend on each other, but the HotSpot folks
> > probably wouldn't like that... ;-)   Moving the HotSpot servicability agent into the JDK, where the
> > interfaces it uses lives, might be a good idea though.  I'm currently working on a patch which fixes
> > up a mismatch between the HotSpot SA implementation and the interfaces in the JDK which has existed
> > for who knows how long (it's certainly in OpenJDK6)...
> 
> When we initially started the OpenJDK project, one single repository was suggested. It was quickly rejected.
> 
> The comments about the Serviceability Agent is puzzling, it has a very tight interface between it and the Hotspot
> VM data structures, or did as I recall. What JDK interfaces are you talking about?
> 

com.sun.jdi.  The serviceabiliy agent provides an implementation of this interface, which sounds like it
should probably be in the HotSpot repository.  

At present, the use of -source/-target 1.4 is hiding a mismatch.

See http://hg.openjdk.java.net/icedtea/jdk7/hotspot/rev/e452443363a7 for the fix.

> > 
> >> 
> >> None of this seemed urgent to do out of the gate, or delay getting a preliminary jdk8 layout defined.
> >> 
> > 
> > No, I agree with that.  Splitting off OpenJDK8 already seems overdue.
> > 
> >> I know there is some interest in pulling the actual jaxp/jaxws sources back into their repos, that will
> >> be discussed separately, we have multiple issues with that, but I am well aware of the pains that the
> >> source drop zip files have created.
> >> 
> > 
> > The problem is less technical and more social; there is no obvious way to interact with the JAXP and JAXWS
> > side of things.  We just get these zips of code with no idea of what changes are in there or why.
> > 
> > Basically, they shouldn't work either like they do now or did before, but like everything else in OpenJDK
> > with visible commits.  Hey, maybe we could even have some mailing list discussion if we're very lucky...
> 
> We will try and have these discussions in the open. And try and make the source more visible, but
> we do want to have the changes flowing from the master sources on these projects. Maybe we can come up
> with a better way.
> 
> > 
> >> As always, we would like to get comments, or additional ideas.
> >> 
> >> Separate topics:
> >>  * Forest Extension and it's replacement
> > 
> > Do we really need one?  Current status quo (get_source.sh) works fine.
> > 
> >>  * Mercurial server update to 1.8 or newer
> > 
> > I guess this is related to the forest issue.
> > 
> >>  * Build&Test system [1]
> >>  * Open bug tracking system [2]
> >> 
> > 
> > You missed off open sourcing that jcheck thing... that's the biggest problem I have with
> > the Mercurial server.  :-)
> 
> Yes. I annoy Mark about this all the time too.  ;^)
> 
> -kto
> 
> 

-- 
Andrew :)

Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

Support Free Java!
Contribute to GNU Classpath and IcedTea
http://www.gnu.org/software/classpath
http://icedtea.classpath.org
PGP Key: F5862A37 (https://keys.indymedia.org/)
Fingerprint = EA30 D855 D50F 90CD F54D  0698 0713 C3ED F586 2A37