Unnamed module and duplicate package
Stephen Felts
stephen.felts at oracle.com
Fri Mar 11 04:02:42 UTC 2016
Yes, it is compiling and running on Jigsaw. It’s not pretty but generally getting better. I’m concerned about a number of behaviors that aren’t pinned down yet and I’m worried that a discussion like this one will introduce a non-upward-compatible behavior that will break things worse.
We have some classes that don’t run on Hotspot because they are already available but run on non-Hotspot JVMs.
The Java EE folks are aware of the problems with both the transaction and annotation jars. One of the outstanding issues I have is to get official copies of these module jar files. Also, the current approach requires two associated jar files on the module path,
javax.enterprise.cdi.api.jar and javax.interceptor.javax.interceptor.api.jar, because of dependencies. I think the plan is to handle the dependencies so those jar files are not needed.
From: Paul Benedict [mailto:pbenedict at apache.org]
Sent: Thursday, March 10, 2016 10:16 PM
To: Stephen Felts
Cc: Alex Buckley; ML OpenJDK Jigsaw Developers
Subject: Re: Unnamed module and duplicate package
Stephen, regarding your first paragraph, I would like some more detail. Are you running your application server with jigsaw?
Cheers,
Paul
On Thu, Mar 10, 2016 at 8:33 PM, Stephen Felts <HYPERLINK "mailto:stephen.felts at oracle.com" \nstephen.felts at oracle.com> wrote:
I'm aware of classes in our application server (not javax) that are unused when running with our own JVM and used when running with another JVM.
In that case, the packages are duplicate of classes in the JDK. I would not want that usage to generate a fatal error.
This is unrelated to endorsed jar files.
In addition to an upgradeable module for java.transaction.jar with Jigsaw, I'm also running with an upgradeable module for javax.annotation-api-1.2.jar.
-----Original Message-----
From: Alex Buckley
Sent: Thursday, March 10, 2016 9:15 PM
To: HYPERLINK "mailto:jigsaw-dev at openjdk.java.net"jigsaw-dev at openjdk.java.net
Subject: Re: Unnamed module and duplicate package
The subtlety is that the Endorsed Standards Override Mechanism interpreted "alternative implementations of the javax.* hierarchy" as "wholesale replacements of entire javax.* packages placed in the location specified by java.endorsed.dirs". It was never permissible to simply throw a handful of javax.* types on the classpath and have them augment JDK packages. As unsupported as sun.misc.Unsafe, as the saying might go.
Let's talk about the Java Transaction API, javax.transaction.xa plus some of javax.transaction. Unfortunately, JTA was never listed at http://docs.oracle.com/javase/8/docs/technotes/guides/standards/index.html
as a Standalone Technology capable of being replaced wholesale by the Endorsed Standards Override Mechanism. We believe this was an oversight; to be fair, it does rather excuse throwing javax.transaction.xa types on to the classpath. Anyway, in Java SE 9, the concept of Standalone Technology has turned into the concept of upgradeable module (where you specify a wholesale replacement of a Java SE module to -upgrademodulepath), and the ill-documented Standalone Technology of JTA has turned into the upgradeable module java.transaction.
tl;dr Since you were and are meant to replace javax.* packages in a wholesale fashion using a first-class mechanism, Paul's suggestion is morally backwards-compatible. Practically, it might be a different story.
Alex
On 3/10/2016 3:58 PM, Neil Bartlett wrote:
> Paul,
>
> This sounds like you are suggesting a backwards-incompatible change to
> the behaviour of the application classpath.
>
> For example, many apps include on their classpath a library containing
> javax.transaction.xa, since the version of this package provided by
> the JDK is broken (omits several types).
>
> In general it has always been permissible for applications to provide
> their own alternative implementations of the javax.* hierarchy, with
> only the java.* hierarchy being off-limits due to the
> SecurityException that would be thrown by ClassLoader.defineClass.
>
> Neil
>
>
>> On 10 Mar 2016, at 21:25, Paul Benedict <HYPERLINK "mailto:pbenedict at apache.org"pbenedict at apache.org
>> <mailto:HYPERLINK "mailto:pbenedict at apache.org"pbenedict at apache.org>> wrote:
>>
>> Alex,
>>
>> For the sake of usability, however, it would be nice if the JDK
>> verified that jars do not contain any exported JDK packages. This
>> would be an RFE. I understand that in order to avoid split packages
>> between modules and classpath, the module version takes precedent.
>> For developer vs. developer code, I find that reasoning fine.
>> However, I really would like to treat the JDK as "special" (your
>> words) because in my experience, I see developers constantly
>> perplexed by NoClassDefFoundError when something occurred like you
>> just detailed.
>>
>> I'd like to refer you to the Servlet 3.1 spec [1], section 10.7.2, as
>> an analogous concern to mine. This is the so-called "prohibited classes"
>> violation clause:
>>
>> "As described in the Java EE license agreement, servlet containers
>> that are not part of a Java EE product should not allow the
>> application to override Java SE platform classes, such as those in
>> the java.* and javax.* namespaces, that Java SE does not allow to be
>> modified. The container should not allow applications to override or
>> access the container’s implementation classes."
>>
>> I don't think it's good usability to let JDK packages in the
>> classpath go silently unchallenged and unloaded. I recommend they are
>> reported as an error.
>>
>>
>> [1]https://java.net/downloads/servlet-spec/Final/servlet-3_1-final.pd
>> f
>>
>> Cheers,
>> Paul
>>
>> On Thu, Mar 10, 2016 at 2:30 PM, Alex Buckley
>> <HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com>>
>> wrote:
>>
>>> I see xml-apis.jar (2.0.2) contains:
>>>
>>> - a javax.xml.parser package, which includes a FactoryFinder class
>>> that's not in Java SE, and
>>>
>>> - a javax.xml.transform package hierarchy, whose types at first
>>> glance look identical to those in Java SE except for yet another
>>> FactoryFinder class in javax.xml.transform.
>>>
>>> If you put xml-apis.jar on the classpath, its javax.xml.** packages
>>> will be ignored. The unnamed module reads the java.xml module which
>>> exports
>>> javax.xml.** packages (assuming java.xml in the system image, of
>>> course), so the application class loader delegates for javax.xml.**
>>> packages to the loader responsible for the java.xml module. User
>>> code that tries to access FactoryFinder will get a
>>> NoClassDefFoundError.
>>>
>>> There's nothing special about JDK modules here. The same
>>> NoClassDefFoundError would occur if the system image contained a
>>> module exporting some package, and a JAR on the classpath contained
>>> the same package with extra classes, and some code on the classpath
>>> tried to access those extra classes. Since the module in the system
>>> image is probably the rightful owner/exporter of the package, hard
>>> questions should be asked about the provenance of the JAR on the
>>> classpath.
>>>
>>> There has been some discussion of a jdeps-like tool that detects
>>> when a JAR on your classpath is trying to split a JDK package:
>>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-November/0052
>>> 27.html
>>> .
>>>
>>> Alex
>>>
>>> On 3/10/2016 10:27 AM, Paul Benedict wrote:
>>>
>>>> Alex, there are JARs that contain javax packages. Anyone in the web
>>>> development community knows how many people have included xml-apis
>>>> in their WEB-INF :-) only to find out it wasn't loaded or it took
>>>> precedent over the JDK versions.
>>>>
>>>> Has Jigsaw introduced any restrictions here on this front?
>>>> Honestly, I think the JDK should make it illegal for the classpath
>>>> to contain ANY packages that the jdk has. Please opine when it is convenient for you.
>>>>
>>>> Cheers,
>>>> Paul
>>>>
>>>> On Wed, Mar 9, 2016 at 5:13 PM, Alex Buckley
>>>> <HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com>
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com>> wrote:
>>>>
>>>> Paul, thank you for asking. The module system's treatment of the
>>>> unnamed module vis-a-vis named modules is probably the biggest
>>>> factor affecting usability of the module system. This is true almost
>>>> by definition because at JDK 9 GA the only named modules in the
>>>> world will be the JDK's while every other class will be in the
>>>> unnamed module of the application class loader.
>>>>
>>>> So please, ask more questions about the unnamed module. I am
>>>> especially interested to know if anyone has JARs that contain javax
>>>> packages (or heaven forbid, sun or com.sun packages) found in the
>>>> JDK -- such JARs are a mortal danger to interop between unnamed and
>>>> named modules.
>>>>
>>>> Alex
>>>>
>>>> On 3/9/2016 1:47 PM, Paul Benedict wrote:
>>>>
>>>> Thank you Alex. Since it's roughly the same as JDK 8, then it's
>>>> also not
>>>> worse. I defer to your explanation on that point.
>>>>
>>>> Cheers,
>>>> Paul
>>>>
>>>> On Wed, Mar 9, 2016 at 3:37 PM, Alex Buckley
>>>> <HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com><mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com>
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com>>> wrote:
>>>>
>>>> Presumably you would count the equivalent scenario on JDK 8
>>>> -- my
>>>> package A is in Alex.jar on the classpath and your package
>>>> A is in
>>>> Paul.jar on the classpath -- as a security issue too,
>>>> because some
>>>> of my classes may substitute for yours (or some of yours
>>>> for mine,
>>>> depending on how the classpath is constructed).
>>>>
>>>> On JDK 9, we do the "substitution" cleanly. Package A is
>>>> not split.
>>>> That avoids one category of error (ClassCastException).
>>>> What about
>>>> poor package B that finds itself accessing a different
>>>> package A
>>>> than it was compiled with? Well, since package A is
>>>> exported by a
>>>> named module, it's reasonable to assume that the named
>>>> module "owns"
>>>> package A [*], and that the developer of package B
>>>> co-bundled some
>>>> version of package A without renaming it. Dangerous in
>>>> JDK 8,
>>>> dangerous in JDK 9. (We're trying to encapsulate the
>>>> internals of a
>>>> module, which is different from trying to isolate modules
>>>> from each
>>>> other.)
>>>>
>>>> [*] Advanced scenario: the named module exporting A is
>>>> actually an
>>>> automatic module which happened to co-bundle package A. By
>>>> placing
>>>> this JAR on the modulepath to form an automatic module, it
>>>> dominates
>>>> the JAR left on the classpath which also co-bundled
>>>> package A.
>>>>
>>>> Alex
>>>>
>>>> On 3/9/2016 1:17 PM, Paul Benedict wrote:
>>>>
>>>> But isn't what your proposing a security issue? Let's
>>>> say my
>>>> package A
>>>> is in the unnamed module and your package A is in a
>>>> named
>>>> module. You
>>>> basically took over my code; your classes will be
>>>> substituted
>>>> for mine.
>>>>
>>>> Cheers,
>>>> Paul
>>>>
>>>> On Wed, Mar 9, 2016 at 2:38 PM, Alex Buckley
>>>> <HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com>
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com>>
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com>
>>>>
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com
>>>> <mailto:HYPERLINK "mailto:alex.buckley at oracle.com"alex.buckley at oracle.com>>>> wrote:
>>>>
>>>> On 3/9/2016 10:36 AM, Paul Benedict wrote:
>>>>
>>>> From the doc:
>>>> "If a package is defined in both a named
>>>> module and the
>>>> unnamed
>>>> module then
>>>> the package in the unnamed module is ignored.
>>>> This
>>>> preserves
>>>> reliable
>>>> configuration even in the face of the
>>>> chaos of the
>>>> class path,
>>>> ensuring
>>>> that every module still reads at most one
>>>> module defining a
>>>> given package.
>>>> If, in our example above, a JAR file on the
>>>> class path
>>>> contains
>>>> a class
>>>> file named
>>>> com/foo/bar/alpha/AlphaFactory.class then
>>>> that file
>>>> will never
>>>> be loaded, since the com.foo.bar.alpha
>>>> package is
>>>> exported by the
>>>> com.foo.bar module."
>>>>
>>>> I would like some clarification. Correct me if
>>>> wrong, but I
>>>> think this
>>>> entire paragraph is really meant to be
>>>> about the
>>>> perspective from a
>>>> modularized JAR? If a module has package A,
>>>> and the unnamed
>>>> module has
>>>> package A, then of course the module's package
>>>> A should
>>>> win.
>>>>
>>>> However, if it is meant to be absolute
>>>> language, then I
>>>> disagree.
>>>>
>>>> The unnamed module should be coherent among
>>>> itself. If the
>>>> unnamed module
>>>> has package B and relies on classes from
>>>> package A, it
>>>> should
>>>> still be able
>>>> to see its own package A. I don't think
>>>> modules should
>>>> be able
>>>> to impact
>>>> how the unnamed module sees itself. That's a
>>>> surprising
>>>> situation.
>>>>
>>>>
>>>> The unnamed module is not a root module during
>>>> resolution.
>>>> If your
>>>> main class is in the unnamed module (i.e. you did
>>>> java -jar
>>>> MyApp.jar rather than java -m MyApp), then the
>>>> module graph is
>>>> created by resolving various root modules (what
>>>> are they?
>>>> separate
>>>> discussion) and only then is the unnamed module
>>>> hooked up
>>>> to read
>>>> every module in the graph.
>>>>
>>>> Hope we're OK so far.
>>>>
>>>> If some named module in the graph exports package
>>>> A (more
>>>> than one
>>>> module exporting A? separate discussion), then
>>>> since the
>>>> unnamed
>>>> module reads that named module, the unnamed module
>>>> will
>>>> access A.*
>>>> types from that named module.
>>>>
>>>> It's hard to imagine the unnamed module NOT
>>>> accessing A.*
>>>> types from
>>>> that named module. Primarily, we need to avoid a
>>>> split package
>>>> situation where code in the unnamed module
>>>> sometimes
>>>> accesses A.*
>>>> types from the named module and sometimes from the
>>>> unnamed
>>>> module.
>>>>
>>>> You might say, OK, let code in the unnamed module
>>>> exclusively access
>>>> A.* in the unnamed module rather than exclusively
>>>> access
>>>> A.* in the
>>>> named module. Then you have two problems:
>>>>
>>>> 1. There are issues for named modules in the same
>>>> class
>>>> loader as
>>>> the unnamed module -- such named modules MUST get
>>>> A.* from
>>>> the named
>>>> module rather than the unnamed module, and the
>>>> class loading
>>>> mechanism is incapable of switching based on
>>>> accessor. It'll be
>>>> common for named modules to exist in the same
>>>> class loader
>>>> as the
>>>> unnamed module, as modular JARs on the
>>>> modulepath and
>>>> non-modular
>>>> JARs on the classpath all end up in the
>>>> application class
>>>> loader
>>>> (modular JARs as named modules; non-modular JARs
>>>> jointly as the
>>>> unnamed module).
>>>>
>>>> 2. While the module system is sure that package A
>>>> exists in the
>>>> named module, how would the module system possibly
>>>> know
>>>> that package
>>>> A exists in the unnamed module? Scanning every
>>>> class file
>>>> in every
>>>> non-modular JAR on the classpath at startup sounds
>>>> bad.
>>>>
>>>> Alex
>
More information about the jigsaw-dev
mailing list