Semantic versioning (was: NP-Completeness...)

[Jesse Glick]

On 12/23/2011 03:35 AM, Peter Kriens wrote:
> With an API you have a consumer of an API and a provider of that API [...]
> few people get the implications of APIs based programming and semantic versioning.

I guess you are referring to semver [1]. I agree that, while well-intentioned, that system does not adequately deal with modules which expose both a widely-used API and 
an SPI with a few implementations, or (more generally) multiple categories of interface intended for different clients with varying tolerance for incompatibility. If all 
these interfaces are presented indiscriminately as the "exports" of a versioned module, you get into a catch-22 when trying to introduce an incompatible SPI change while 
keeping the API fully compatible (or, generally, changing only those interfaces with a few well-known clients): you must mark the module version incompatible, to ensure 
that SPI implementors are updated to match the changes; but if you do so then the universe of API users will be gratuitously shut out of what would otherwise be a simple 
upgrade.

An unsatisfactory resolution is to version all SPI changes like compatible API changes, and just hope for the best. This seems to have been the policy of Java SE, with 
the result that the occasional implementor of DOM or JDBC interfaces (proxies, unit test mocks, ...) routinely becomes uncompilable with a new JDK (*). Such a policy 
becomes intolerable when pieces of the platform are downloaded according to version constraints.

Another resolution is to separate the SPI into its own package structure (as is common anyway) and version individual packages, as might be done in OSGi bundles. This is 
impossible in the current design of Jigsaw, of course, or generally in any module system aspiring to have a 1-1 mapping between module versions (and dependencies) and 
native package versions (resp. dependencies).

An arguably simpler resolution is to use a single module version but treat the semantics of its dotted components differently depending on the category of import. The 
OSGi "Semantic Versioning" section (3.7.3 in r4 4.3) advocates this, though it assumes that _any_ change to the SPI is automatically incompatible which seems overly 
pessimistic. The Jigsaw draft does not offer any real advice on version policy, or even describe exactly how version ranges may be expressed, but it seems to support 
(manual and unenforced) use of this policy. (**) In my draft proposal for updated versioning policy in the NetBeans module system [2] I take this approach, with the added 
twist of using the build system to enforce the correct kind of version range for an importer based upon its actual imports (under the condition that import categories map 
to package boundaries).


(*) Yes I have seen the proposal for interfaces with default method bodies in JDK 8. This provides a workaround for one class of incompatible changes - adding interface 
methods. Conscientious SPI designers could already avoid the problem in various ways, such as optional extension interfaces or abstract classes, but obviously many chose 
not to. (Nor are incompatible changes limited to this, of course - you may want a static factory method to take a list of something rather than a single object, ad 
nauseam.) I would urge the JLS [3] to be amended to say that adding an interface method (without a proposed default body) _does_ break binary compatibility, and be 
treated accordingly for purposes of versioning.

(**) For better or worse, Jigsaw also supports "permits" clauses by which only enumerated friend modules could use the SPI. The NetBeans module system has had a similar 
feature since 2005 but it has proven unworkable, and my proposal [2] essentially deprecates it. My understanding is that Eclipse continues to declare "friends", via 
package import constraints with partial tool enforcement.


[1] http://semver.org/
[2] http://wiki.netbeans.org/NbmPackageStability#Marking_stability_of_import
[3] http://java.sun.com/docs/books/jls/third_edition/html/binaryComp.html#13.5.3