Updated conformance text for Java SE 9

Tue Jul 25 19:09:46 UTC 2017

2017/7/25 10:17:06 -0700, volker.simonis at gmail.com:
> On Sat, Jul 22, 2017 at 1:46 AM,  mark.reinhold at oracle.com wrote:
>> 2017/7/21 7:15:44 -0700, volker.simonis at gmail.com:
>>> ...
>>> 
>>> So back to the specification. If we have a fixed list of standard
>>> modules which are upgradeable, I don't understand why all the other,
>>> non-SE modules are upgradeable by default (i.e. "A non-Java SE module
>>> is upgradeable unless the Implementation deems otherwise"). Why not
>>> define it the other way round that non-SE modules aren't upgradeable
>>> until the Implementation deems otherwise? Do you really think
>>> upgradeable non-standard modules will be useful? If yes, in which
>>> context?
>> 
>> Yes: Consider a custom linked image that contains some library and
>> application modules.  You might want to override some of those modules
>> in production, say, without having to (and maybe without being able to)
>> re-link the image.
>> 
>> I'll add some text to the specification to suggest this use case.
> 
> So you say that not only standard or platform modules can be
> upgradeable, but also application modules which are linked into a
> custom image?

Correct.

>               That's interesting - I haven't thought about that use
> case. Please see my further comments below.
> 
>>> And there's another problem with the upgradeable modules. Because they
>>> are upgradeable they obviously can't be hashed. So if somebody builds
>>> a custom image without an upgradeable module, a user of that image
>>> could easily provide his own module with the same name like the
>>> upgradeable module and use all of the qualified exports from the
>>> platform modules to the original version of the upgradeable module.
>>> Don't you think that this could compromise the integrity of the
>>> customized jlinked image?
>> 
>> Yes, it certainly could!  That's why we've taken care in the RI to make
>> sure that no interesting packages are exported to upgradeable modules.
>> In fact, the only packages that are exported with qualification to
>> upgradeable modules are from other upgradeable modules.
> 
> So if you add the section about overriding user modules from a custom
> linked image which you've proposed above you should also add a warning
> that creators of such images should be careful with the qualified
> exports to such modules because they can be upgraded by a user and
> compromise the integrity of their application. That's probably not as
> dangerous as compromising the integrity of the platform, but that may
> be not what users expect from modularization and custom images.

Good idea.  In the "Upgradeable modules" subsection [1] of the PFD draft
I already added, as promised:

  Custom configurations may contain non-Java SE library and application
  modules which it is convenient to upgrade in an atomic fashion.

To this I'll further append:

  Creators of such configurations should take care to ensure that
  qualified exports that target such modules cannot be abused if one or
  more of those modules is upgraded.

>>>                                            ... The standard can't
>>> actually say anything about non-standard modules, but it could mandate
>>> that every module is somehow tagged as upgradeable or not.
>> 
>> It could, but that would require that we make upgradeability an explicit
>> and standard property of all modules.
>> 
>> Should we delay the release in order to design and implement that feature?
> 
> Is this an attempt to blackmail me :)

No, it was an honest question.

> Of course I don't want to further delay the release. But I also don't
> want that we simply reject useful features just because of a release
> date. The fact that the feature of "upgradeability" also applies to
> user modules which are linked into custom images (which you mentioned
> above) makes this even more important.
> 
> The RI currently uses a non-standard feature (hashing) to enforce that
> only specific modules can really be upgraded. Somebody who creates a
> custom linked image that contains some library and application modules
> needs to carefully think which of them should be upgradeable and which
> should not. And he currently can not explicitly define that. If he
> doesn't do anything, then his modules will be all upgradeable by
> default (which may be not obvious). If he wants to make some of his
> modules not upgradeable, he has to hash them when he creates his
> custom image (which is also not obvious). And finally, he's not
> allowed to hash the modules which should be upgradeable (which is
> again not obvious). And we can't even advise that here in the
> specification because hashing is not a standard feature.

True.

> So I think we should at least record these two issues ("making
> upgradeability an explicit standard property" and "hashing of
> dependent modules in order to ensure integrity") somewhere (e.g.
> something like the JPMS issue summary) for future updates of this
> specification.

Rather than suggest the specific mechanism (i.e., hashing) for a future
release, let's just document the requirements in a new paragraph at the
end of the "Upgradeable modules" subsection:

  This Specification does not define a standard means to specify whether
  an arbitrary library or application module is upgradeable, nor does it
  define a standard means to ensure the integrity of a set of related
  library or application modules.  The creators of future versions of
  this Specification are encouraged to consider these requirements.

> And finally a new point which I only became aware of recently. What's
> the relation of upgradeable and automatic module? I.e. can a plain old
> jar file be used to upgrade a module? In the current RI it can and I
> think that this is crucial because the whole mechanism is actually
> designed for APIs which are usable on their own or that correspond to
> standards maintained outside of the JCP. The only requirement for this
> to work is that the jar file is either named in such a way that its
> automatically created module name matches with that of the module or
> that the jar file explicitly uses a corresponding
> Automatic-Module-Name attribute in its manifest. So maybe you could
> add a sentence like "a module can also be upgraded by a plain old jar
> file if the automatic module name of the jar file is the same like the
> name of the module"?

Sure.  To the existing paragraph:

  In order to replace an upgradeable module, the user supplies an
  alternate version which the Implementation observes in preference to
  the version included in the Implementation.  The alternate version
  typically exports APIs that are newer than those exported by the
  version in the Implementation, although an Implementation is not
  required to enforce this property.

I will append:

  The alternate version may be provided in the form of a JAR file that
  defines an automatic module.

- Mark

[1] http://cr.openjdk.java.net/~iris/se/9/java-se-9-pfd-spec-01/#Upgradeable-modules