Conflicting module versions

[David M. Lloyd]

I agree that working dependency resolution is a necessity for a working 
software system.  But, it is my opinion that run time is far too late 
for this to occur (and I intend to carry this opinion into the JSR 
working group).  Ultimately it is up to the module distributor to ensure 
that their distribution contains a cohesive module set.  While it is 
possible to efficiently add certain types of validation checks at run 
time, by this time it is far too late to do anything about a violation 
other than just fail.

I don't know if the behavioral contract of module distributors will be 
specified in this JSR; my guess is that they will only be covered to the 
extent that modular compilation is to be supported and specified. 
Ensuring a cohesive version graph is (in my opinion) out of scope for 
the specification itself though, and falls to the module distributor, 
possibly with the help of a certain amount of reasonable build-time tooling.

I don't know that there is much more to say before the EG kicks off. 
IIRC Mark R targeted this for January, so I guess this translates into 
"real soon now".

On 02/05/2015 04:27 AM, Petter Måhlén wrote:
> Hi,
>
> This is something that I brought up with Mark Reinhold at Jfokus in
> Stockholm the other day. I don't think I was able to formulate my concern
> well, and while I've now seen it may be considered a non-requirement in
> Jigsaw
> <http://openjdk.java.net/projects/jigsaw/goals-reqs/03#multiple-versions>,
> I think it's important, so I'm going to see if I can be more coherent in
> text.
>
> One of the more common and also hard-to-solve problems that happen when you
> have a fairly large, Maven-based ecosystem is that your transitive
> dependencies depend on conflicting versions of some jar file. A common
> example at Spotify (where we use a lot of Cassandra) is that some
> application uses Cassandra version 2.0.5 and a new version of some internal
> library, let's call it X. Library X depends on Guava version 18 and uses,
> say, com.google.common.base.MoreObjects. Cassandra 2.0.5 depends on Guava
> version 15 and uses some method (maybe
> com.google.common.io.ByteStreams.newInputSupplier() or something) that has
> been removed in Guava 18.
>
> The problem is of course that if Maven decides to include Guava version 18
> on the classpath, then you will get an error at runtime due to the method
> on ByteStreams having been removed, and if it decides to include version
> 15, then library X will run into another error because the MoreObjects
> class didn't exist before version 18. Another problem is that it's not
> obvious to developers how Maven is going to choose which version to include
> (I think
> http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> is
> a correct description, but I'm not sure), but that's not really germane to
> this discussion.
>
> When this sort of problem happens, it's unpredictable because it depends on
> exactly which code paths get hit and it's hard enough to figure out that
> people can spend days on it, depending of course on how familiar they are
> with this type of thing. Note that since it's dependent on which code gets
> executed, it doesn't necessarily start happening immediately when you
> include library X, but maybe later on due to some seemingly innocuous code
> change that led to execution of a new code path. There are tools like
> the enforcer
> plugin <http://maven.apache.org/enforcer/maven-enforcer-plugin/> that help
> reduce the likelihood of this happening, but I think most Java developers
> are unaware of those.
>
> Some variations of the problem:
>
>     1. As above: two incompatible versions are specified; there is no choice
>     that will work for all code paths. The developer has to change the
>     dependencies somehow.
>     2. Library X doesn't depend on MoreObjects or anything else that has
>     been added since version 15 of Guava, so there is a best choice (pick
>     version 15).
>     3. Two Maven artifacts with different group/artifact ID:s define the
>     same classes. A real-world example is mockito-all
>     <http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.mockito%22%20AND%20a%3A%22mockito-all%22>,
>     which for instance defines Hamcrest classes that would normally be taken
>     from hamcrest-core
>     <http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.hamcrest%22%20AND%20a%3A%22hamcrest-core%22>.
>     Another variation is when an artifact changes names, such as when
>     google-collections became Guava. There are many other examples.
>
> The point I'm trying to make is that I agree with the statement from
> JSR-376 that, "Developers have long suffered with the brittle, error-prone
> class-path mechanism for configuring program components", but I think that
> not addressing the problems above means we will keep suffering. In
> practice, the only brittleness problems I have seen people suffering from
> in the last 10 years or so are due incorrect dependency resolution leading
> to runtime errors. As a developer of developer tools, I really like the
> idea of being able to do things like specify which packages I'm exporting
> from my modules, but I don't think the people using my modules suffer very
> much from me not being able to make the backwards-incompatible changes I
> would like in an API that should never have been public. I suffer because
> of that, and while that makes me less good at shipping improvements, I
> think they lose more from versioning conflicts.
>
> So if nothing else, perhaps the formulation in the Jigsaw goals and
> requirements could be changed - the need for a *working* dependency
> resolution mechanism is clearly there. Maybe the place where it is best
> solved is not in Jigsaw, but it does seem like something that would fit
> squarely with the goals of the project and JSR, and it's in fact the only
> important classpath-brittleness issue I tend to come across.
>
> Regards,
> Petter
>

-- 
- DML