Exporting things (Was: Re: Module-system requirements)

[David M. Lloyd]

On 02/16/2015 07:15 AM, David M. Lloyd wrote:
> On 02/15/2015 07:24 PM, mark.reinhold at oracle.com wrote:
>> 2015/2/11 11:27 -0800, david.lloyd at redhat.com:
>>> I notice that under Exports, it is defined that only packages (i.e.
>>> classes within those packages) are considered to be exportable, and not
>>> resources.  I'm not certain how I feel about that... in our
>>> implementation we flattened the concepts of packages and directories.
>>> This was mainly to solve a few specific difficulties of supporting plain
>>> JARs as module content; a JAR has a mix of classes and resources, and it
>>> is hard to deterministically tell the difference, especially when you
>>> can do such (commonly done) things as:
>>> classLoader.getResource("org/foo/bar/SomeClass.class").
>>>
>>> ...
>>>
>>> However this approach is not without its difficulties: ...
>>>
>>>     ...  To sum up though, I'd be glad to see (from my perspective) this
>>> behavior go away in favor of the simpler model of restricting exports to
>>> packages (and isolating classes and resources),
>>
>> I tend to agree.
>>
>> Exposing all of a module's resources, as JAR files do today, is wrong
>> from the perspective of encapsulation.  Extending the "exports"
>> requirement to allow control over which resources are exported leads to
>> both conceptual and implementation complexity (e.g., what would it mean
>> to export both a class, as a type, as well as the corresponding .class
>> file, or the latter and not the former?)  It seems sanest to treat
>> resources as strictly internal to modules.
>>
>> This suggests an additional requirement:
>>
>>    - _Resource encapsulation_ --- The run-time system must ensure that
>> the
>>      static resource files within a module are directly accessible
>> only by
>>      code within that module.
>
> What you are saying here implies (to me) inspecting the call stack to
> ensure that the caller actually exists within the module being
> referenced.  I'm not certain we really want to go this far, if my
> interpretation is accurate.  Such a rule is reasonable on the face of
> it, but it does more or less break all the code in existence which
> currently uses resources as a sort of API into a library, and this
> includes very common patterns for loading configuration (among many
> other things).

Going into a little more detail on this, I picture a number of scenarios:

1) A resource is fully internal to a module, and there's no reason that 
any external code ought to need or get access to this data.  The module 
could restrict access to the resource by caller as inferred above.

2) A resource is an internal part of the module, but other things might 
need access to the resource for specific purposes (like direct .class 
access for instrumentation).  The access restriction of this use case is 
"solved" by existing containers today by isolating class loaders, so the 
user has to have access to the target class loader in order to read the 
resource(s) in question.  You could subdivide this into (2a) the .class 
files themselves and (2b) anything else that might fall under this 
category, as I believe the use cases for (2a) are fairly specific to 
run-time analysis and instrumentation tooling and similar uses.

3) A resource is a part of the module's self-configuration, but is not 
of interest to importers.  This would include configuration files (like 
Java EE deployment descriptors, and other framework-XML-ish things). 
Access is typically presently solved similarly to #2 above, or by simply 
accessing the target resource archive directly, bypassing class loading.

4) A resource is a part of a public API intended to be imported and used 
by consumers.  Examples include framework configurations like logging 
and some other framework-XML-ish things, ServiceLoader, etc.  Today this 
is solved as an unavoidable consequence of flat classpaths, but also (in 
containers) by implicitly or explicitly exporting and importing some or 
all of a class loader's resource file/directory structure.

While JBoss Modules does in fact solve cases 2-4 well enough, there is 
no provision for things falling under (1).  This is not however due to 
any fundamental ideal; quite the opposite in fact.  I simply could not 
work out a consistent (and simple) set of rules for treating classes and 
packages separately from resources and their directories that didn't 
simply break everything*.

The questions are: Are there yet more use cases that I have failed to 
identify?  Is it possible to come up with a consistent set of rules 
which allows all these primary use cases to continue to function cleanly 
going forward?  Can we do so in a minimally-impacting manner that would 
allow most existing code to continue to function with (at most) only 
packaging changes?  If not, which use cases can or should be sacrificed, 
and why?

I could easily imagine adding a separate public resources concept which 
is wholly separate from the module internal content, where importers 
could import some or all of these resources into their own world, which 
would easily solve (4), but (2) and (3) are still left "high and dry". 
An "unexported resources" area of a module could address (3), leaving 
(2), which could be solved by co-locating those resources with the 
classes themselves.  But by this point there is some additional 
complexity (maybe not *too* much, but definitely some).  Module imports 
would specify the module they want to import (possibly to package 
granularity), as well as the resource directories they want to import 
(which could be re-exported just like module packages could).  So from 
that perspective the additional complexity is manageable.  But when it 
comes to build and packaging of a module itself, users would now have 
face possibly fairly complex decision regarding which category a 
resource belongs to (not to mention that packaging tooling would have to 
support this decision), whereas in today's world they just "throw it in 
there" and hope for the best.

* Though I admit the idea of examining the call stack for this purpose 
hadn't occurred to me, but I'm not sure I could have done anything 
concrete with that idea anyway due to the myriad complex paths by which 
class loader methods are invoked (especially in a multiple-delegation 
situation like ours), not to mention the performance problems (such an 
approach would really need [1] to be solved performantly as a prerequisite).

[1] https://bugs.openjdk.java.net/browse/JDK-8043814
-- 
- DML