6856630: Restructure jaxp/jaxws repository

Fri Oct 23 00:49:47 UTC 2009

Kelly O'Hair wrote:
> Andrew,
>
> Sorry I'm so slow on this, ...
>
> I'm looking at updating the jaxp bundle. The webrev is:
>    
> http://cr.openjdk.java.net/~ohair/openjdk7/jdk7-tl-jaxp-6861589/webrev/
>
> Updated jaxp source bundle, and complete removal of anything
> that would use the original source. The final changeset will also
> delete the jaxp/src directory so we never accidently use it.
> It also adds a md5 checksum check on the bundle in the ant script,
> a property value contains the expected checksum.
>
> I'm not done, so I could add more if you want me to add the changes
> mentioned here... The merge should be trivial.
>
> One snag is that I plan on pushing this to the TL forest so that
> the TL integrator (Tim Bell) can get the appropriate tests done
> on the jaxp changes. In the future jaxp changes will probably
> be going through the TL area.
>
> ...  a few more comments below...
>
> Andrew John Hughes wrote:
>> 2009/10/12 Kelly O'Hair <Kelly.Ohair at sun.com>:
>>> I saw the webrev for a while, and now I can't :^(, but I saw enough...
>>>
>>>>> Andrew John Hughes wrote:
>>>>>> 2009/10/7 Andrew John Hughes <gnu_andrew at member.fsf.org>:
>>> ...[snip]...
>>>>>> I found a number of issues with the current version:
>>>>>>
>>>>>> * The drop zips are expected to be in a share/jdk7-drops 
>>>>>> subdirectory
>>>>>> of the devtools directory.  This doesn't really work outside Sun's
>>>>>> internal setup.  I've added support for ALT_DROP_DIR which can be 
>>>>>> used
>>>>>> to specify an exact path to the zips.  If this is not set, the old
>>>>>> devtools behaviour is used.
>>> I'm concerned about the name dropdir (your master copy location) vs.
>>> drop.dir
>>> (destination for the exploded contents) in the property names. Seems
>>> confusing.
>>> I hesitate to go on a name hunt, but what about ALT_DROP_BUNDLE_DIR and
>>> drop.bundle.dir?
>>>
>>
>> Yes, I noticed drop.dir after posting the webrev and agree it's
>> confusing.  Would just making them ALT_DROPS_DIR and drops.dir not be
>> sufficient?  drop.bundle seems a little verbose and both mean the same
>> thing to me at least.
>
> I'm ok with ALT_DROPS_DIR and drops.dir.
>
>>
>>>>>> * The zips are being downloaded and extracted in the source 
>>>>>> directory,
>>>>>> which could be read only.  I've changed this so they are written to
>>>>>> the build directory like everything else.
>>> It turns out that we need this drop.dir to be in the source area if
>>> we want the openjdk source zip bundles, created by RE, to include
>>> the drop sources. These source bundles are created by bundling up
>>> everything in the forest, prior to building, so the drop.dir needs
>>> to be in the forest somehow. Otherwise, the bundling logic has to
>>> merge two separate source trees, or different roots.
>>> I was given a requirement that the openjdk source bundles include these
>>> drop sources, easiest answer was drop.dir in the repo.
>>> Not an unsurmountable problem, I avoided the more complicated path.
>>>
>>> I know that becomes an issue if the repository or source is in a
>>> read-only area, but that's not the only situation in the openjdk I 
>>> suspect.
>>> I assumed someone building from a read-only area could set
>>> drop.dir to some other location.
>>>
>>
>> Would it not make more sense for RE to set the drop.dir property into
>> the source directory, given it's that build that's abnormal (needing
>> to distribute a post build tree in effect)?
>
> I will look into that. You have convinced me. ;^)
>
>>
>>> Not sure what to say.
>>>
>>> How important is it for the community that the openjdk source bundles
>>> include these drop sources? e.g. ones at
>>> http://download.java.net/openjdk/jdk7/
>>> If they are not included, when the openjdk source bundles are built,
>>> they will need to download the jaxp and jaxws drop bundles at build 
>>> time.
>>>
>>
>> I obviously can't speak for everyone, but for IcedTea7 we don't even
>> use the download bundles any more.  We grab from the IcedTea forest
>> instead, which allows patches which didn't make a bxx release to be
>> applied there once and used in releases.  We did this for the security
>> fixes with Milestone 4/1.11.
>>
>> I've already added downloading of the jaxp, jaxws and jaf bundles to
>> the IcedTea build; it's this process that resulted in this webrev.
>> This has two advantages; we check all the zips/tarballs at the same
>> point in the build (the JAXP and JAXWS builds are then pointed at the
>> directory IcedTea uses for them) and we check the MD5 sums of the
>> downloads or pre-existing zips.  I don't know if the latter could be
>> incorporated into the OpenJDK build somehow.  At present, there's no
>> guarantee that these zips are valid.
>>
>> The rules look like this:
>>
>> stamps/download-jaxp-drop.stamp:
>>         mkdir -p drops
>> if USE_ALT_JAXP_DROP_ZIP
>>         ln -sf $(ALT_JAXP_DROP_ZIP) drops/$(JAXP_DROP_ZIP)
>> endif
>>         if ! echo "$(JAXP_DROP_MD5SUM)  drops/$(JAXP_DROP_ZIP)" \
>>           | $(MD5SUM) --check ; \
>>         then \
>>           if [ -f drops/$(JAXP_DROP_ZIP) ] ; \
>>           then \
>>             mv drops/$(JAXP_DROP_ZIP) drops/$(JAXP_DROP_ZIP).old ; \
>>           fi ; \
>>           $(WGET) $(DROP_URL)/$(JAXP_DROP_ZIP) -O
>> drops/$(JAXP_DROP_ZIP); \
>>         fi ;
>>         mkdir -p stamps
>>     touch stamps/download-jaxp-drop.stamp
>>
>> ALT_JAXP_DROP_ZIP gets set if the user specifies
>> --with-jaxp-drop-zip=<path> to configure.  WGET and MD5SUM are the
>> respective binaries as located by configure and DROP_URL is the
>> kenai.com URL as in the jaxws.properties and jaxp.properties files:
>>
>> DROP_URL = http://kenai.com/projects/jdk7-drops/downloads/download
>> JAXWS_DROP_ZIP = jdk7-jaxws-2009_09_28.zip
>> JAXWS_DROP_MD5SUM = debb949440c5a15ce999cfefbbc56526
>> JAF_DROP_ZIP = jdk7-jaf-2009_08_28.zip
>> JAF_DROP_MD5SUM = eb8cb7a4a7f14e211fbe2354878a2472
>> JAXP_DROP_ZIP = jdk7-jaxp-2009_09_28.zip
>> JAXP_DROP_MD5SUM = 8800970d03bab1fff188dcfafc346f5d
>
> Ah... so my checksum logic in the ant script will serve as a
> double check.
>
> Unless you are changing the bundles and rebundling them...
> nobody does that, ...  do they???
> Ah, if they do, they would need to change the property value
> holding the checksum.
>
> -kto
>
>>
>>> -kto
>>>
>>>>>> The webrev for these changes is here:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~andrew/drops/webrev.01/
>>>>>>
>>>>>> I've committed this to IcedTea already.  Does this look ok for build
>>>>>> also?
>>>>
>>>>
>>
>>
>>
Kelly,

Does all this mean we can somewhat simplify the list of forests on 
hg.o.j.n, by removing jdk7/{corba,jaxp,jaxws} and the corresponding 
-gate forests?

-- Jon