jvmti and endorsed mechanism

Andrew John Hughes ahughes at redhat.com
Thu Mar 17 11:41:05 PDT 2011 

On Thu, Mar 17, 2011 at 10:28:06AM +0100, Jiri Vanek wrote:
> On 03/16/2011 07:11 PM, Dr Andrew John Hughes wrote:
> >On 08:52 Wed 16 Mar     , Jiri Vanek wrote:
> >>On 03/15/2011 11:21 PM, Dr Andrew John Hughes wrote:
> >>>On 13:29 Tue 15 Mar     , Jiri Vanek wrote:
> >>>>Does anybody know why endorsed mechanism is used during build of jvmti?
> >>>>    - exepct the one I found: that -classapth seems to me as being ignored
> >>>>by xslt transformation done by jvmtiGen (or???)
> >>>>
> >>>>Current (part of) jvmti build looks like this:
> >>>>in configure (configure.ac? - I'm not used for auto-tools) is defined
> >>>>XALAN2_JAR nad XERCES2_JAR. Those  two files are then linked from
> >>>>icedtea6-1.10/bootstrap/ecj/lib/endorsed.
> >>>>icedtea6-1.10/bootstrap/ecj/lib/endorsed is set (somewhere) into
> >>>>ENDORSED variable, which is added by jvmtiEnv.patch into jvmti.make.
> >>>>result in jvmti.make:
> >>>>XSLT = $(QUIETLY) $(REMOTE) $(RUN.JAVA) $(ENDORSED) -classpath
> >>>>$(JvmtiOutDir) jvmtiGen
> >>>>...
> >>>>$(XSLT) -IN $(JvmtiSrcDir)/jvmti.xml -XSL $(JvmtiSrcDir)/jvmtiEnter.xsl
> >>>>-OUT $(JvmtiOutDir)/jvmtiEnter.cpp -PARAM interface jvmti
> >>>>
> >>>>where the build was failing with  Exception in thread "main"
> >>>>java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal.
> >>>>
> >>>>This was caused by new version of xalan, which left xml-commons-apis
> >>>>from its "inner" classpath.
> >>>>
> >>>>linking xml-commons-* from endorsed(^^) dir solved this problem
> >>>>
> >>>>This is exactly the state endorsed dir mechanism  should protect from -
> >>>>there should be clases which should be used prior all other classes. In
> >>>>our case there shoud be drop of xalan/xerces which are working instead
> >>>>of links to /usr/share/java/... .
> >>>>
> >>>>Most easy fix to make icedea 1.10 build run on F15 is to link ^^
> >>>>mentioned files from endorsed dir in makefile (if anybody can handle
> >>>>./configure part I will be more then happy).
> >>>>
> >>>>Better fixes are two - use endorsed dir properly and use working drop of
> >>>>xalan and xerces (I'm afraid with all of those xml-commons-* beause
> >>>>commons used by by "our-working" xalan willnot be avaiable in f15) or
> >>>>get rid of it and use normal -classpath (which can be difficult if
> >>>>endorsed dir was used with reason)
> >>>>
> >>>>Hope that I have written this clearly:)
> >>>>
> >>>>Regards J.
> >>>>
> >>>>
> >>>>
> >>>
> >>>You don't provide any background here:
> >>>
> >>>* What JDK are you building with?
> >>
> >>Openjdk upon Released 1.10  on f15
> >>
> >
> >Err... no, that sounds like what you're building, not what you're using to build it.
> 
> I'm building in mock.
> 

Yes, but which JDK is it using to build?  This will be in the output of configure.

The main reason I ask is that Xerces/Xalan is only used in bootstrap mode.  It won't
be used if --disable-bootstrap is used, which I think was the case for most previous
Fedora releases.  It's not a long-term solution, but at least it will workaround
the problem for now.

> >
> >>>* What version of Xerces?
> >>>* What version of Xalan?
> >>
> >>The last ones avaiable for f15:
> >>xalan-j2-2.7.1-3.fc15
> >>xerces-j2-2.11.0-3.fc15
> >
> >I also have 2.7.1 of Xalan but 2.9.1 of Xerces.
> >
> >I presume the issue is:
> >
> >'Added support for the Element Traversal API (org.w3c.dom.ElementTraversal). [Michael Glavassevich]'
> >
> >from http://xerces.apache.org/xerces2-j/releases.html
> >
> >org.w3c.dom.ElementTraversal is not provided as part of the 1.6 API or any earlier version.  Neither
> >is it in 1.7.
> >
> >In depending on it, Xerces looks broken to me and so whoever is packaging that needs to fix it
> >as this will break other packages too.
> >
> If this is right, and your conclusions about it are right then it is
> ok:) And it will fix itsef alone :D
> 

Doubtful, someone has to fix it.  I just think this is a general issue
that will affect more than just us and it needs to be fixed at the source
(Xerces) rather than us working around it.

> If no, then I will fix it by add commons to endorsed dir.
> 
> Although my doubts about using of endorsed dir were not explained,
> at least Dr. Andrew replied ... (community :D ).. Thanx a lot! I
> don't feel myself to fix whole issue as written above:(
> 

Xerces and Xalan are put in the endorsed directory so they override the XML
implementation in the JDK.  http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47843
is the reason.

I've been considered just patching the JDK during bootstrap to use xsltproc
instead of the JDK.  We could also use this to generate NEWS, AUTHORS, etc.
as we discussed in relation to the javaws about app.  I'll take a look next
week but I'm not sure about putting that in 1.10.1.  It will probably be a 1.11
feature.

> 
> I have some doubts about this issue being fixed. As mbooth is
> writing at bottom of this email, he see no reason to add xml-commons
> to "xalan/xerces classapth".
> 

Yes and I don't agree with them or upstream.  This is going to break more than
just OpenJDK as users will expect to just put Xerces on the classpath before and
for it to work.

This is why we need a module system with proper dependency management.

> 
> >>
> >>>
> >>>A full build and config.log would be good.
> >>
> >>The background is longer stuff:
> >>
> >>1.10 build for f15 was failing with Exception in thread "main"
> >>java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal during
> >>jvmti staff building.
> >>
> >>After some digging i have found it is xalan/xercess problem (my last
> >>week status and java-team irc log 9.3.2011)
> >>
> >>
> >>Also reply of mbooth on fedora channel attached.
> >>
> >>Regards J.
> >>
> >>
> >
> >>Date: Wed, 9 Mar 2011 21:24:24 +0000
> >>From: Mat Booth<mbooth at fedoraproject.org>
> >>To: Alexander Kurtakov<akurtakov at fedoraproject.org>, Stanislav Ochotnicky
> >>  <sochotnicky at redhat.com>
> >>Subject: Re: Changed xalan/xerces package in f15
> >>
> >>On 9 March 2011 16:08, Matthew Booth<mbooth at redhat.com>  wrote:
> >>>Mat,
> >>>
> >>>Pretty sure this was meant for you.
> >>>
> >>>Regards,
> >>>
> >>>Matt
> >>>
> >>>-------- Original Message --------
> >>>Subject: Changed xalan/xerces package in f15
> >>>Date: Wed, 09 Mar 2011 16:58:51 +0100
> >>>From: Jiri Vanek<jvanek at redhat.com>
> >>>To: mbooth at redhat.com
> >>>
> >>>Hi!
> >>>
> >>>I need to take a talk with you because of broken openjdk build caused by
> >>>xalan. You have been blamed by koji;)
> >>>
> >>>Can wee? I'm from central European timezone and mi nick upon #devel,
> >>>#0day, #java, #brno #fedora-devel, and #openjdk is jvanek
> >>>
> >>>Or we can take an emails. This is background:
> >>>
> >>><jvanek>  akurtakov_, ping
> >>><akurtakov_>  jvanek: yes?
> >>><jvanek>  hi, i'm missing ElementTraversal interface in xalan
> >>><jvanek>  I have seen your name between builders. Maybe you know whats
> >>>going on
> >>><akurtakov_>  jvanek:  I would need a bit more info
> >>><akurtakov_>  what are you build ? buildlog?
> >>><akurtakov_>  xalan/xerces versions
> >>><jvanek>  openjdk build is feiling with Info: jvmtiGen using
> >>>javax.xml.transform.TransformerFactory =
> >>>org.apache.xalan.processor.TransformerFactoryImpl
> >>><jvanek>  Exception in thread "main" java.lang.NoClassDefFoundError:
> >>>org/w3c/dom/ElementTraversal
> >>><jvanek>  when i look into xalan build for f15 (xalan-j2-2.7.1-3.fc15)
> >>>there is another tar.gz with  org/w3c/dom/ package, but ElementTraversal
> >>>is missing
> >>><akurtakov_>  do you have xml-common-apis in the classpath?
> >>>* akurtakov_ is pretty sure that org/w3c/dom should be there
> >>><akurtakov_>  jvanek: does it help?
> >>><jvanek>  mmnt
> >>><akurtakov_>  jvanek: /usr/share/java/xml-commons-apis.jar
> >>><jvanek>  hmmm... xml-commons-apis.jar are installed and elementtraversal
> >>>is inside
> >>><jvanek>  thanx for this info.
> >>><jvanek>  Will follow this ...
> >>><jvanek>  akurtakov_, ^
> >>><jvanek>  Are you aware about some changes here beteween f15/14?
> >>><akurtakov_>  jvanek: nope, sorry
> >>><jvanek>  np. thanx a lot
> >>><akurtakov_>  it's mbooth who was taking care of it lately
> >>><jvanek>  yy
> >>><akurtakov_>  jvanek: note that there are 2 mbooth guys and I'm speaking
> >>>for the one that is not working for Red Hat
> >>><jvanek>  akurtakov_, ok:)
> >>><sochotni>  jvanek: I actually just remembered...that was some problem
> >>>with some xml parser implementation....but I can't remember what was it
> >>>exactly. It involved xerces/xalan and ElementTraversal so it seems it's
> >>>the same problem..
> >>><sochotni>  it was a problem with alternatives
> >>><jvanek>  sochotni, hhmmm
> >>><sochotni>  I think it was /etc/alternatives/jaxp_parser_impl
> >>><jvanek>  sochotni, this looks correct lrwxrwxrwx. 1 root root 29 Mar  9
> >>>08:49 jaxp_parser_impl ->  /usr/share/java/xerces-j2.jar
> >>><jvanek>  lrwxrwxrwx. 1 root root 28 Mar  9 08:49 jaxp_transform_impl ->
> >>>/usr/share/java/xalan-j2.jar
> >>><sochotni>  yes, that does...but I thing xerces changed something and
> >>>wasn't really providing jaxp_parser_impl
> >>><akurtakov_>  jvanek: sochotni:
> >>>http://koji.fedoraproject.org/koji/packageinfo?packageID=482
> >>><akurtakov_>  mbooth pushed latest xerces just to F-15
> >>><akurtakov_>  and this is the released that removed ElementTraversal in
> >>>favor of the one in xml-commons-apis
> >>><sochotni>  yup, so the alternative is incorrect now
> >>><akurtakov_>  sochotni: why?
> >>><akurtakov_>  xerces is still a valid jaxp parser impl but it needs
> >>>additional jar
> >>><sochotni>  akurtakov_: right, my bad..
> >>><jvanek>  so i expect  xml-commons-apis.jar are missing inmy classpath
> >>>* ptisnovs is now known as ptisnovs_away
> >>><jvanek>  but i also expect this jar should be added to classpath by
> >>>xerces co I can blame mbooth O:)
> >>><akurtakov_>  jvanek: sadly there is no way to do this except by
> >>>hardcoding the classpath in the manifest
> >>><akurtakov_>  which opens a number of other problems
> >>>
> >>>
> >>>
> >>>
> >>>Regards J.
> >>>
> >>
> >>
> >>Yes, I guess this must be my fault, sorry about that.
> >>
> >>(If Redhat ever offer me a job, I assume I will have to fight the
> >>other mbooth to the death before I get his email address...)  ;-)
> >>
> >>However, I'm not clear why you expect the JAXP implementation (xerces)
> >>to include the JAXP apis (xml-commons-apis)?
> >>
> >>Perhaps it's because xml-commons-apis and xerces are now providing
> >>JAXP 1.4 instead of 1.3 and the JDK has the JAXP 1.3 apis built in
> >>(and thus did not need xml-commons-apis when it was JAXP 1.3).
> >>
> >>Maybe you need to use an endorsed override?
> >>http://xerces.apache.org/xerces2-j/faq-general.html#faq-4
> >>
> >>Regards,
> >>Mat
> >>
> >>--
> >>Mat Booth
> >>http://fedoraproject.org/get-fedora
> >
> >
> 

-- 
Andrew :)

Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

Support Free Java!
Contribute to GNU Classpath and IcedTea
http://www.gnu.org/software/classpath
http://icedtea.classpath.org
PGP Key: F5862A37 (https://keys.indymedia.org/)
Fingerprint = EA30 D855 D50F 90CD F54D  0698 0713 C3ED F586 2A37

More information about the distro-pkg-dev mailing list