Multiple versions of a non-exported dependency

[Gregg Wonderly]

Why I was referring to is how will modules find classes from other modules?  How will the different version of the same package namespace that Gili was talking about be hidden?   In other words, you can only make them all visible by using a single class loader, unless there is a DAG of dependencies that naturally falls out, because all we have is ClassLoaders parent as the pointer to other interesting detail.

How will definedModulesWithManyLoaders make it possible for all of the correct details to be visible? 

Some explicit example detail:

module A uses l4j logging in module B

module C uses l4j logging in module D

module E is a standalone module

A and C exchange data objects defined in E

+— B — A
 |
+ E
 |
 +— D — C

This graph you can draw today with ClassLoader parent references.   But, there are more complicated graphs that fall out from less simple needs.  It’s this specific issue of still not trying to provide support for versions and for arranging the module dependencies more explicitly through the use of version as part of the package name namespace which Gili is trying to speak to I feel.

Since B and D are the same package name, you have to hide them from each other in separate class loaders, obviously.   But the Graph from C to E and A to E is not always direct and can in some cases not be possible with a single instance of the jar.  That breaks static class data designs if you create two copies in separate loaders.

Gregg

> On Sep 1, 2016, at 8:37 AM, David M. Lloyd <david.lloyd at redhat.com> wrote:
> 
> You've missed the point I'm afraid.  I'm just talking about having the launcher use (the equivalent of) java.lang.reflect.Layer#defineModulesWithManyLoaders instead of (the equivalent of) java.lang.reflect.Layer#defineModulesWithOneLoader.  (The launcher actually uses the slightly lower level defineModules() method I think, but really what I'm suggesting is to update the function to assign new class loaders for named modules instead of reusing the same one.)
> 
> On 09/01/2016 08:23 AM, Gregg Wonderly wrote:
>> The important detail for me, is that ClassLoader per module, with the current Class resolution scheme (this ClassLoader and whatever I can find in the parent), provides a lot of issues. The “custom ClassLoaders” or “containers like OSGi” remarks point at the “us and them” attitude that is pretty prevalent in this conversation.  The majority of developers are looking for a module system that is not an “us or them” proposition.   These “all or nothing” compromises are what create the “hell” that dominates conversations here.  What we all want to be able to do, is write software once, target it to “THE Java platform”, and be done.
>> 
>> What Sun and now Oracle are continuing to do, is create more stuff that is nothing like what everyone else is doing with modularity and instead create something that is orthogonal to most peoples problem spaces and in the end creates tremendously more “work” for nothing more than compatibility with the new “JVM” environment.
>> 
>> The real goal here needs to be making all of the other module and container systems obsolete.  Those systems should “want” to provide support for the awesome, new module system that will make in unnecessary for them to roll their own details any longer.
>> 
>> Yes, that is a long road and a tall measure for success.  But frankly, even the lack of any visibility of the style of modules that Netbeans has used for decades makes it clear that this groups view at Oracle is extremely narrow and perhaps even more uninformed about what the community actually needs.
>> 
>> Gregg
>> 
>>> On Sep 1, 2016, at 7:29 AM, David M. Lloyd <david.lloyd at redhat.com> wrote:
>>> 
>>> It seems like there is no good reason why the application modules aren't loaded with classloader-per-module now.  The platform stuff could all be in one, but the application stuff?  Problems like this are going to come up a lot otherwise; let's consider making that change.
>>> 
>>> On 08/31/2016 07:45 PM, Neil Bartlett wrote:
>>>> Remi,
>>>> 
>>>> Actually I don’t think that statically linking will work. This would produce modules that have overlapping private (non-exported) packages, and such modules also cannot be used in Java 9 on the modulepath.
>>>> 
>>>> I tested this in build 9-ea+126-jigsaw-nightly-h5280-20160713 by creating two modules both containing a private package org.example.util. The following exception resulted:  java.lang.reflect.LayerInstantiationException: Package org.example.util in both module a and module b.
>>>> 
>>>> Again this could be “solved” by using custom ClassLoaders or a ClassLoader-based module system like OSGi on Java 9.
>>>> 
>>>> Neil
>>>> 
>>>> 
>>>> 
>>>>> On 31 Aug 2016, at 20:28, Remi Forax <forax at univ-mlv.fr> wrote:
>>>>> 
>>>>> The other solution is to statically link the right version of slf4j inside guava and jsoup.
>>>>> A tool like jarjar can be updated to merge two modular jars (merge two module-info).
>>>>> 
>>>>> cheers,
>>>>> Rémi
>>>>> 
>>>>> ----- Mail original -----
>>>>>> De: "Neil Bartlett" <njbartlett at gmail.com>
>>>>>> À: cowwoc at bbs.darktech.org, "Alex Buckley" <alex.buckley at oracle.com>
>>>>>> Cc: "ZML-OpenJDK-Jigsaw-Developers" <jigsaw-dev at openjdk.java.net>
>>>>>> Envoyé: Mercredi 31 Août 2016 20:54:44
>>>>>> Objet: Re: Multiple versions of a non-exported dependency
>>>>> 
>>>>>> Gili,
>>>>>> 
>>>>>> As Alex points out: your use-case can be supported in Java 9 but only with the
>>>>>> addition of custom ClassLoaders, or by using an existing ClassLoader-based
>>>>>> module system such as OSGi.
>>>>>> 
>>>>>> The same is also true of Java 8, and Java 7, etc.
>>>>>> 
>>>>>> Regards,
>>>>>> Neil
>>>>>> 
>>>>>> 
>>>>>>> On 31 Aug 2016, at 19:29, Alex Buckley <alex.buckley at oracle.com> wrote:
>>>>>>> 
>>>>>>> On 8/31/2016 10:56 AM, cowwoc wrote:
>>>>>>>> I recently became aware of the fact that the Jigsaw specification declared
>>>>>>>> "version-selection" as a non-goal. While I understand how we ended up here,
>>>>>>>> I am hoping that you were able to support the following (very common)
>>>>>>>> use-case:
>>>>>>>> 
>>>>>>>> * Module "HelloWorld" depends on modules "Guava" and "JSoup".
>>>>>>>> * Module "Guava" depends on module slf4j version 1 (requires but does not
>>>>>>>> export it).
>>>>>>>> * Module "JSoup" depends on module slf4j version 2 (requires but does not
>>>>>>>> export it).
>>>>>>>> * slf4j version 2 and is not backwards-compatible with version 1.
>>>>>>>> 
>>>>>>>> What happens at runtime? Will Jigsaw (out of the box, without 3rd-party
>>>>>>>> tools like Maven or OSGI) be smart enough to provide different versions of
>>>>>>>> slf4j to "Guava" and "JSoup"?
>>>>>>> 
>>>>>>> (You mean Guava/JSoup requires slf4j version 1/2 and does not "re-export" it
>>>>>>> a.k.a. 'requires public'.)
>>>>>>> 
>>>>>>> This use case isn't possible on JDK 8 for JARs on the classpath, and it's not
>>>>>>> supported on JDK 9 for modular JARs on the modulepath:
>>>>>>> 
>>>>>>> - If you have two versions of a modular JAR slf4j.jar in different directories
>>>>>>> on the modulepath, then the first one to be found will dominate, and that's
>>>>>>> what will be resolved for both Guava and JSoup.
>>>>>>> 
>>>>>>> - If you have two modular JARs slf4j_v1.jar and slf4j_v2.jar on the modulepath,
>>>>>>> and Guava requires slf4j_v1 and JSoup requires slf4j_v2, then launching 'java
>>>>>>> -m HelloWorld' will fail. The boot layer will refuse to map the "same" packages
>>>>>>> from different slf4j_v* modules to the application class loader.
>>>>>>> 
>>>>>>> The use case _is_ supported on JDK 9 for modular JARs loaded into custom loaders
>>>>>>> of custom layers. That is, the Java Platform Module System is perfectly capable
>>>>>>> of supporting the use case -- please see any of my "Jigsaw: Under The Hood"
>>>>>>> presentations. The use case just isn't supported "out of the box" by the 'java'
>>>>>>> launcher for JARs on the modulepath.
>>>>>>> 
>>>>>>> Alex
>>>> 
>>> 
>>> --
>>> - DML
>> 
> 
> -- 
> - DML