Optional service dependencies only

[Paul Sandoz]

On Jun 21, 2012, at 4:12 PM, Jaroslav Tulach wrote:

> Dne Čt 21. června 2012 14:14:29, Paul Sandoz napsal(a):
>> On Jun 21, 2012, at 11:35 AM, Jaroslav Tulach wrote:
>>> [1] http://wiki.apidesign.org/wiki/JigsawServices
>> Nice write up. We are definitely on the same page w.r.t. optionality.
> 
> Right. Although, I would not call it optional. Each individual service 
> implementation is optional, but the resolution fails if none can be enabled.
> 
>> What i was trying to do with my patch is change when those optional
>> dependencies are processed i.e. at the end, thus first the application
>> module dependencies are resolved as if no service provider modules are
>> present.
> 
> This is what should happen, with following clarification: I am still operating 
> in context of "complete repository" - e.g. I assume that the whole set of 
> transitive dependencies of a module has been computed and recorded at compile 
> time[1]. Only under such assumption range dependencies are acceptable, as far 
> as I understand the problem.
> 
> [1] http://wiki.apidesign.org/wiki/RangeDependenciesAnalysed#No_Longer_NP-
> Complete
> 

For the moment i am gently sweeping this aspect under my mental rug :-) Yes there is room, but it is getting rather lumpy.


>> My hunch is that because service consumers are decoupled from service
>> providers it can be harder to predict or understand the result when the
>> resolver mixes in the resolving of service provider dependencies.
> 
> Right. "Classpath" dependencies first, services later is going to be more 
> understandable to humans.
> 

OK. I am gonna refine my patch to make this more explicit. 

For a start we can unwind the call stack for those "Classpath" dependencies, then start again for services and essentially do:

  if (resolve(o, ch)) {  // "Classpath" dependencies
    // Service dependencies
    while(/has service dependencies/) {
      Set<ViewDependency> serviceDependencies = ... // copy
      // Reset service dependency state
      for (ViewDependency sd: serviceDependencies) {
        resolve(0, new Choice(null, vd, null));

        // Did the optional dependency get resolved? if not log warning as to why
      }
    }
    ensureServicesPresent();
  }


>>> However there should be a way to reconfigure these hints at the deployment
>>> time.
>> 
>> You raise a very interesting point!
>> 
>> I think this suggests control of versions of transitive dependencies, much
>> like what can be done with maven. Although the JAXP module may hint it
>> strongly wants a version of Xerces another module may strongly hint it
>> wants a different version. Perhaps the conflict could be mediated by
>> changing module processing order, or perhaps by declaring what version
>> should be used.
> 
> The general principle should be: the one that knows more should make the 
> decision. That is why:
> 
> #1 - a JAXP API knows it needs an implementation and knows Xerces module 
> provides one that is good enough
> 

Every module declares what it needs with "requires ...". Perhaps it is a lack of imagination on my part but i don't currently see how we can strengthen that need in a decentralized environment. It is not when it all comes together that such needs can be compared?


> #2 - the application vendor (providing the application module) knows all the 
> modules in its application and all the services and can decide certain 
> services should have different default (this is what Maven tried to do with 
> its "shortest" path dependency). Also when compiling the whole application, it 
> should be easy to decide whether different services are compatible with the 
> rest of modules or not as one knows the transitive closure of the whole 
> application.
> 
> #3 - a deployer should have the option to override the application defaults 
> without coding in Java, imho. E.g. something like:
> 
> $ java \
>  -module com.myapp.xsltproc \
>  -service javax.xml.parser.DocumentBuilderFactory \
>  -default org.apache.crimson.CrimsonBuilderFactory
> 
> ...of course one needs to install com.myapp.xsltproc module into the library 
> (is it the right term, I am still learning the terminology) as well as 
> org.apache.crimson module, so the resolver can enable both.
> 

Yes, "install module M into library L".


> #3a - the same using some configuration file which format is beyond the scope 
> of my current thinking
> 

Or part of a module-info that declares entry points.


> 
>>> I've noticed a discussion about security providers. You were in need of
>>> changing their order and that could be also considered a "hint". Just
>>> probably a hint provided externally, at the time the application is
>>> started (e.g. not compiled).
>> 
>> Yes, although in this case the order is associated with a service interface
>> not the module. 
> 
> Good point. The order needs to be associated with service interface types. 
> Btw. NetBeans have an order of modules defined by topological sort on top of 
> graph with "requires" edges (not "requires service"). This graph is used for 
> startup and shutdown order. But services are orthogonal to this concept, they 
> have their own ordering (per interface type, in general).
> 
> -jt
> 
>> Modules can be service provider modules for multiple
>> service interfaces. So hints on the order of service provider classes may
>> need to be something separate to hints on the order modules are resolved.
> 
> PS: What exactly order of modules is good for in Jigsaw?
> 

I was not referring to any topological sort of the module graph.

Order can affect the solution produced by the resolver. e.g. if module A depends on C, D, E in that order as per the declaration order of "requires ..." clauses in A's module declaration then perhaps one can override and say try E, C and D. I was assuming that this was one approach you were proposing.

I don't think the Jigsaw resolver will attempt to preserve a topological order (e.g. depth first traversal order) for the list of resolved modules, mainly because it does not use LinkedHashSet and LinkedHashMap and when building the set of context  HashSet is also used so that set would not be preserved either.

This means there is no consistent order, based on module graph traversal, for the service provider classes in the configuration. It  can also unnecessarily change each time the configuration is recalculated! (e.g. because of the hash code changes in SE8). We need to fix that.


> PPS: In NetBeans it is used for order of activators and also for order of URLs 
> returned from systemWideClassLoader.getResources(...). It is guaranteed 
> resources are returned in topological order with respect to order of modules.
> 

There is currently no concept of activators in Jigsaw and of course there is no system wide (or global) class loader.

However, i think we need to preserve a topological order for modules throughout the process of producing a configuration so that iteration of service instances is consistent with that order.

Paul.