Exporting things (Was: Re: Module-system requirements)

[David M. Lloyd]

On 03/16/2015 03:47 PM, mark.reinhold at oracle.com wrote:
> 2015/3/10 8:08 -0700, david.lloyd at redhat.com:
>> On 3/9/15 3:59 PM, mark.reinhold at oracle.com wrote:
>>> 2015/2/16 5:15 -0800, david.lloyd at redhat.com:
>>>> On 02/15/2015 07:24 PM, mark.reinhold at oracle.com wrote:
>>>>> ...
>>>>>
>>>>> Exposing all of a module's resources, as JAR files do today, is wrong
>>>>> from the perspective of encapsulation.  ...
>>>>>
>>>>> This suggests an additional requirement:
>>>>>
>>>>> - _Resource encapsulation_ --- The run-time system must ensure that the
>>>>> static resource files within a module are directly accessible only by
>>>>> code within that module.
>>>>
>>>> What you are saying here implies (to me) inspecting the call stack to
>>>> ensure that the caller actually exists within the module being
>>>> referenced.
>>>
>>> Yes.
>>>
>>>> I'm not certain we really want to go this far, if my
>>>> interpretation is accurate.  Such a rule is reasonable on the face of
>>>> it, but it does more or less break all the code in existence which
>>>> currently uses resources as a sort of API into a library, and this
>>>> includes very common patterns for loading configuration (among many
>>>> other things).
>>>
>>> True.  To what degree should it be a goal to preserve such mechanisms?
>>
>> I really feel that not having this compatibility will strongly diminish
>> adoption of this system, since a substantial body of existing work is
>> unlikely to function properly as modules.
>>
>>> Must Java modules be forever constrained to look like JAR files?
>>
>> No, but I think that isn't really related to the point at hand, which is
>> more related to the solution of specific problems rather than change for
>> its own sake.  If solving those problems means that modules have visible
>> resources, does that mean we've tripped an invisible failure criteria by
>> resembling JARs too closely in some specific way?
>>
>> Or put a nicer way, is there a specific problem or set of problems that
>> are solved by strictly encapsulating resources, or is it more of a way
>> to try a new general approach?
>
> Encapsulation of resources is valuable for the same reasons that
> encapsulation of types is valuable.  It allows you to conceal a module's
> implementation details so that external code does not depend them, giving
> you greater flexibility to evolve the module going forward.  It also
> allows you to keep adversaries from inspecting the internals of your
> module to discover information that might be useful in an attack.

I think you missed the thrust of my question which was "...*strictly* 
encapsulating...".  It seems clear just from our own experience that the 
user generally wants to keep resources private, but the question is 
whether we really want to kill off the practice of using specific 
resources as API (in this way).  But the new "Access to class files and 
resources" that you give should adequately cover this use case for new 
code at least.

> We'll encapsulate types by default, and also have a way to export them to
> other modules on a per-package basis.  We seemed to agree earlier that a
> similar style of export control for resources would be overkill, and my
> suggested requirement to ensure access to a module artifact's resources
> by suitably-privileged code, when such an artifact exists, appears to
> cover the use cases so far discussed (as you note later in your message,
> quoted below).

I agree that export control for resources adds complexity, however, 
without it we're definitely going to be looking at a potentially much 
more onerous conversion process for a broad category of existing code 
which wants to take advantage of the benefits of Java modularity but 
presently relies on using class loaders to access resources for the 
purposes of self-configuration, which does not (in my experience or 
opinion) violate encapsulation any more than exporting a public class 
does.  My agreement, such as it is, is based purely on the ensuing 
simplification of the module linkage algorithm, as well as the relative 
simplicity of *new* code (free of any legacy encumbrances) which isn't 
really the best of motives.

I guess what I'm getting at is, I'd like to be able to see some 
realistic mechanism to allow such code to work *unchanged* as modules 
(even if a customized dynamic configuration were required to do so). 
Whether or not your previously-proposed method of going through the 
class loader is a practical approach requires a bit more consideration, 
I guess.  It may depend on how difficult it is to control the 
relationship between modules and class loaders, assuming that we don't 
end up with a 1:1 relationship, or it may depend on making resource 
export control a feature of the module architecture which is not 
utilized by the default platform module configuration but can be used by 
higher layers.

>>> ...
>>>
>>> To ensure that a distributor of a set of modules can control how service
>>> dependences are resolved amongst those modules here's a new requirement,
>>> for the Fundamentals section:
>>>
>>> - _Selective binding_ --- It must be possible to control the binding
>>> process so that specific services are provided only by specific
>>> providers, and in a specific order.
>>>
>>> Does that make sense?
>>
>> In this requirement, what party is doing the controlling?
>
> Whatever party has invoked the binding process.  It might be the user,
> via command-line flags or build-system configuration settings, or it
> might be an application server, via the module-system API.

OK I think this makes sense to me.

>>> ...
>>>
>>> So a JBoss module can specify that it depends upon service providers from
>>> one or more specific modules rather than whichever modules happen to
>>> provide implementations of that service?  That seems contrary to the
>>> nature of services, if I understand you correctly.
>>
>> Right; this is however functionally similar to having the configuration
>> determine the service bindings without any sub-configurations being able
>> to override it, though at a whole-module granularity rather than a
>> per-module-service granularity.
>
> I can understand having a build system or a run-time framework control
> service bindings in this way, but it seems odd for a module itself to
> declare that it requires a specific provider of a service.

It's not too odd in practice.  Many projects (especially OSS frameworks 
which we consume) split their code into API and implementation aspects 
for encapsulation reasons (both at run time and at build time), where 
there are typically a small constant number of implementations (maybe 1, 
but sometimes one primary and one or two less common/specialized, whose 
usage is based on run time circumstances).  Then the implementations are 
simply imported into the API in order, the API uses its own class loader 
to spin up the service loader, and everything is fine.

This seems preferable to putting API and implementations into the same 
module (which is the only practical alternative that we provide), and 
also frees the user from having to create "obvious" bindings between 
service and provider (though if there could be a way for modules to 
influence the binding process by specifying a default or "preferred" set 
of provider modules for a given service, that could be a 
practical/satisfactory alternative I think).

>> ...
>>
>> In the configuration-level case though, establishment and predictability
>> of order (at least on a per distribution basis) is almost certainly
>> going to be needed I think, so I'd still maintain that it should be
>> worked into the requirements somehow.
>
> Is the proposed "selective binding" requirement (above) sufficient?

Probably, with the aforementioned caveat.

>>> When, in the future, an application is assembled by resolving a set of
>>> actual modules into a configuration, then we won't necessarily have to
>>> use class loaders to mediate resource access, so I'd rather not bake that
>>> into the requirements.  A configuration ought to be able to relate
>>> classes, modules, and module artifacts such that, given a module and a
>>> class name, you can read the corresponding class file (or any other kind
>>> of resource) from the module's defining artifact and load it however you
>>> want.
>>>
>>> To capture this as a requirement, in the Development section:
>>>
>>>     - _Access to class files and resources_ --- If a module is defined by
>>>       an artifact that contains class files and resources then it must be
>>>       possible, once that module is added to a configuration, for
>>>       suitably-privileged code to access those class files and resources.
>>
>> I like this requirement; I think it hits a good percentage of use cases.
>>
>>> (The "if a module is defined by an artifact" precondition allows for
>>>    ahead-of-time-compilation scenarios.)
>>>
>>> This requirement also addresses the problem of how to locate and read EE
>>> deployment descriptors and similar kinds of configuration files.
>>
>> Great.

-- 
- DML