Downloading and installing maven artifacts as modules

[Eric Johnson]

Hi Brian,

On 9/27/11 5:06 PM, Brian Pontarelli wrote:
> I've been sporadically following Jigsaw but I saw this thread and have recently been discussing a similar topic with some folks. The topic we have been discussing is the meta-data needed for pure dependency management compared to the meta-data provided by Maven, Jigsaw, Ivy, Savant and others.
>
> I wanted to bounce this idea off of this list as well as other Maven, Ivy and others. The idea would be to create an open standard format that defines the meta-data needed to do pure dependency management. This has a number of key benefits including:
>
> 	- Standardize dependency repositories
> 	- Clean dependency definitions
> 	- Simple way for tool providers to work with dependencies separate from the build and runtime systems
> 	- Easier migration between build tools (pick the right tool for the job not the repository or dependency management system)
>
> There are probably other benefits as well. I wanted to get your thoughts this concept. It looks like Jigsaw intends to support POMs and potentially support some of OSGi, but at a baseline, there are a number of concepts that can be defined and shared across all these projects.

If that is something that gets taken up by this group, I feel compelled 
to throw in my several cents worth.

a) Redundant metadata - extremely bad. Ideally, as much of the run-time 
dependency information should be used for build-time dependency 
resolution as possible. Only introduce build-time dependency metadata as 
needed, when it goes above and beyond what the run time data says. At my 
company, we've struggled with extra meta data driven by the use of an 
Eclipse-like notion of a feature, and the OSGi-level meta-data as well. 
This means that they can get out of sync, and do, and that can lead to 
nasty, annoying and subtle issues.

b) Build against the supported minimum. I think of this as the JDK 1.4 
problem. JDK 1.4 introduced "StringBuffer.append(StringBuffer sb);" You 
could take the same code and compile against both JDK 1.4 & 1.3. 
However, the version compiled against 1.4 would only run against 1.4, 
whereas the JDK 1.3 compiled version would run against both. What with 
auto-boxing, and newer language features, the opportunities for such 
mistakes are legion. So, if you want to assert that you *run* against 
version 2.3.0 and later of a dependency (be it a package, bundle, or 
module), you better make sure you build against that version.

c) Follow-on to (b) - sometimes the dependencies you state simply aren't 
consistent. If A depends on B & C, and B also depends on C, you might be 
in a spot where B requires a newer version of C than A requires. The 
correct thing to do here is flag a warning that the dependency that A 
states on C is actually a lie, and its lower end must be bumped.

d) What you need to build != what you need to run. For example, in OSGi 
terms, my bundle may require, at runtime, the ability to log. The API 
for the logging service may be fixed, but an implementation of that API 
must also be available. At build time, I don't need the implementation. 
In fact, if my build process does require an implementation, the one it 
grabs may trigger the problem indicated by (c), thus either incorrectly 
building against a newer version of some other API, or suggesting my 
bundle must raise its minimum dependencies unnecessarily.

e) What you need to run is a subset of what you need to test. Once we 
start specifying ranges for dependencies, we really should be making 
sure that at runtime we test against the lowest version of that 
dependency, the highest stable version, and any future versions in 
development. And that's just a minimum. In practice I suspect most 
people frequently shortcut this notion by continually raising dependency 
minimums, and forgoing testing of future versions until those future 
versions are just about declared stable.

Circling back to the notion that "there are a number of concepts that 
can be shared across all of these projects," I approach such an 
assertion with considerable trepidation. At some point you can boil this 
all down to a simple modeling notion of capabilities and requirements 
(as in code X provides capability, code Y requires some capability). 
However, we need to add modifiers to the predicates here. For example:

  * Version ranges
  * Security assertions (FIPS certified)
  * # of suppliers (I need > 1 implementation of service foo, I need
    exactly one supplier of service foo)
  * vendor
  * License (unless you intend to leave the open-source world out)

So, now, what you've got is a system that involves an arbitrary set of 
subjects and objects, and an arbitrary set of predicates that combine 
those two sets. That's right, you've got RDF. To the extent that you 
standardize an RDF-like approach, it only works if everyone agrees on 
the meanings of some core set of subjects, predicates, and objects.

And keep in mind, the more subjects and predicates you introduce, the 
more complexity you get.

Having said all that, yes, please, let's have at it, and finally try to 
establish what dependencies mean in the Java space, so that the platform 
can further thrive.

-Eric.