Updated conformance text for Java SE 9 (JSR 379)

[mark.reinhold at oracle.com]

// I posted the draft conformance text to both the 376 and 379 EG lists for
// maximal transparency.  Tim replied on the 376 list but his comments are
// equally relevant to 379, and 379 is where this text resides anyway.  I'm
// replying here to both EG lists.  Please be sure to include both lists in
// any further replies, and if needed I'll moderate them through on the
// list of which you're not a member.

2017/7/24 6:33:34 -0700, tim_ellison at uk.ibm.com:
> Now that Java has introduced the concept of an explicit "link" stage for
> subsetting the platform code, I think the conformance definition should
> differentiate between an implementation of the specification that is
> providing a set of modules as open APIs, and an implementation of the
> specification that is solely used to power a "closed-world" application.
> 
> In the one case, a conformant platform may be a subset, and should
> provide a coherent set of modules that implement a well-defined
> development/runtime platform.  In the other case the implementation is
> being subset to provide a runtime fit for a particular purpose.
> 
> With this in mind, we can introduce two new definitions:
> 
> A “Platform Implementation“ of this specification means a set of modules
> created to be run with a variety of application programs, or linked with
> a specific application to produce an “Application Specific
> Implementation”, i.e. this is a library and associated runtime.
> 
> An “Application Specific Implementation” of this specification means a
> set of modules that are linked to a specific application such that the
> modules are not observable at runtime to code outside the application,
> i.e. this is a closed-world application.

What does it mean for code to be "outside" an application?

What if the application loads additional modules from the module path,
or additional classes from the class path?  Is that code outside the
application?

What if the application loads user modules, or JAR files, or plugins, or
whatever, using its own custom class loader?  Is that code outside the
application?

Even without loading new external code, what if the application spins
bytecodes at run time?  Is that code outside the application?

In all of these situations the outside code will depend upon at least
some Java SE APIs.  To the extent that the modules that define those
APIs are present in the linked application image, people will sensibly
expect them to conform to the Platform Specification, and to pass the
TCK.

> The conformance description would then go on to refer to these as
> follows:
> 
> A Platform Implementation of this specification must include the
> `java.base` module, _i.e._, must make that module observable.  A Platform
> Implementation may, further, include one or more additional Java SE
> modules so long as the set of Java SE modules that the Implementation
> includes is _closed_, …
> 
> An Application Specific Implementation of this specification is not
> subject to the same conformance requirements as a Platform Implementation
> of this specification.

The present Specification does not support "closed-world" applications.
There is no way to forbid the loading of additional classes at run time.
There is no way to make a set of modules unobservable to modules outside
of that set.  You can write code that simply avoids doing those things,
of course, but from the standpoint of the Platform Specification there
is no way to forbid those things from being done, and that's what you'd
need in order to guarantee a truly "closed world".

Perhaps a future revision will add those capabilities, but without them we
can't support a concept of "Application Specific Implementations".

> Please see further specific comments in-lined below.

My replies inline also.

> mark.reinhold at oracle.com wrote on 11/07/2017 18:10:38:
>> ...
>> 
>> Every Implementation of this Specification must include the `java.base`
>> module, _i.e._, must make that module observable.  An Implementation
>> may, further, include one or more additional Java SE modules so long as
>> the set of Java SE modules that the Implementation includes is _closed_,
>> i.e., contains every Java SE module required, via `requires transitive`
>> directives, by any member of the set.  For a specific example, an
>> Implementation that contains just the `java.base`, `java.sql`,
>> `java.logging`, and `java.xml` modules is closed because the only
>> `requires transitive` dependences are from `java.sql` to `java.logging`
>> and `java.xml`,
> 
> and `java.base` if you are being transitive.

No -- none of these modules `requires transitive java.base` since there
is no need for that, since every module implicitly `requires java.base`.

>> and the latter two modules are members of the set.
>> 
>> The Technology Compatibility Kit (TCK) for this Specification will be
>> able to test all of the Java SE modules included in an Implementation,
> 
> consider: strike out "be able to"

Done.

>> and it will require that set to be closed.  An Implementation that
>> passes the TCK is considered to fully implement this Specification even
>> if it does not include all of the Java SE modules.
> 
> consider: "...TCK is considered to implement this Specification correctly
> even if..."  Saying it fully implements the spec with missing modules
> doesn't make sense.

I understand that the use of "fully" here reads oddly, but it's needed
in order to be be consistent with the language of the Java Specification
Participation Agreement (JSPA [1]), and in particular with 5(B)a, which
enables independent Implementations.

>> If an Implementation of this Specification includes a Java SE module
>> then
> 
> How could an implementation of this specification *not* include an SE 
> module?

Yes, that's awkward; I've already fixed it in the next draft.

>> it must fully implement the corresponding part of this Specification. It
>> must, in particular, implement every API element of that module
>> including the module's name, its exported packages, and any `requires
>> transitive` dependences.  It must not export any of that module's other
>> packages without qualification.
>> 
>> This Specification defines the Licensor Name Space on a module-by-module
>> basis.  Provided that an Implementation that fully implements this
>> Specification includes the required Licensor Name Space for each
>> included
> 
> consider: replace "required" by "entire"?

Again, this is for consistency with the JSPA, in this case 5(B)b.

>> module then it is not considered to subset the Licensor Name Space.
> 
> <snip>
>> An Implementation of this Specification may provide a means to create
>> further Implementations.  If it does so then any new Implementation must
>> satisfy the constraints of the preceding paragraphs but need not include
>> all of the modules present in the original Implementation.
> </snip>
> 
> Why must it be an implementation of the specification that creates a
> further implementation?  Maybe it is a builder tool, linker, stripper,
> etc.

If it's a tool that's outside of an Implementation then there's no way
for this Specification to constrain it -- it's just an arbitrary tool.

>> As an aid to migration, an Implementation may relax the strong
>> encapsulation of the modules that it includes as follows:
>> 
>>  - An Implementation may provide a means to invoke its run-time system
>>    with one or more packages of one or more of its modules open to code
>>    in all unnamed modules, i.e., to code on the class path.
> 
> I suggest this should not be limited to invocation time, i.e. consider
> replacing this sentence with:
> 
>     - An Implementation may provide a means to open one or more packages
>       of one or more of its modules to code in all unnamed modules, i.e.,
>       to code on the class path.

That doesn't change the meaning of the statement, since opening a
package has no effect in any phase except run time.

>>    The first
>>    access to an element of such a package by a program via a reflection
>>    API must cause a warning to be issued on the standard error stream.
>>    Later accesses may also cause warnings to be issued.
> 
> This should be an implementation choice, not a spec requirement.  It
> is not going to be possible to define a meaningful "warning",
> especially in headless environments or places where the error console
> is not reified.

The "standard error stream" is perfectly meaningful (= `System.err`).
How it's logged or displayed is, of course, an implementation choice.

> While I see the value in warnings at development time, requiring a warning
> for every runtime invocation is unreasonable.

In light of the later statement that

  Future revisions of this Specification are expected to mandate that
  all of an Implementation's modules be strongly encapsulated by default
  and, eventually, to disallow the above-described relaxation.

it would be irresponsible in this version not to mandate some level of
warning.

> What will the JCK be checking to ensure this requirement is met?

The JCK will check the standard error stream.

>>  - An Implementation may, by default, open one or more packages of one
>>    or more of its modules to code in all unnamed modules.  If it does 
>>    so then it must issue warnings as described in the previous paragraph.
> 
> Consider: dropping this warnings requirement.

No, for the reasons previously stated.

>> ...
> 
> <snip>
>>  - An Implementation may provide a means to request that additional
>>    warnings or debugging information, such as stack traces, be issued
>>    when elements of its open packages are accessed via a reflection 
> API.
>> 
>>    (The Reference Implementation's run-time system can be invoked with
>>     the command-line options `--illegal-access=warn` or
>>     `--illegal-access=debug` in order to request warnings or both
>>     warnings and stack traces, respectively, on each unique reflective
>>     access to any open Implementation package.)
> </snip>
> 
> I suggest taking out the text between the <snip> markers.  This
> information is best provided in user docs rather than the conformance
> definition, otherwise there will be a very long list of what /may/ be
> done.

Good point -- done.

>> Future revisions of this Specification are expected to mandate that all
>> of an Implementation's modules be strongly encapsulated by default and,
>> eventually, to disallow the above-described relaxation.
>> 
>> As a further aid to migration, an Implementation may provide a means to
>> invoke its run-time system in a way that establishes additional
> 
> Change to remove reference to invocation time, e.g. "...provide a means to
> establish additional..."

Yes, though it's a bit more subtle than that -- adding reads edges and
exporting packages is relevant at both compile time and run time, but
opening packages is only relevant at run time.  Text corrected.

- Mark