Non-java resources creating split-package problems?

[Stephan Herrmann]

Thanks Alan,

This looks like quite a party of stakeholders discussing what is or is not a 
package. Let me check if I get it right:

(1) JLS requires a package declaration in a compilation unit for a package to exist.

(2) a class file attribute in module-info.class may declare the set of packages.

(3) for automatic modules the rules are effectively similar to (1), provided 
that .class files have been placed in the proper folder in the jar.

(4) for explicit modules a new kind of package comes into the picture: packages 
created by non-java resources.


As you mention that ModuleFinder could say more about (4), I'd re-phrase it: 
currently nothing positively defines this concept.


If the above list is correct (complete?), some questions remain:

- What should happen when (2) declares a set of packages that differs from what 
scanning would result in? Can (2), e.g., be used to hide a package?

- Is it possible to export/open a package that has no .class in it? According to 
JLS: no!
   - does this imply non-java packages are always encapsulated?

- Can any package from (4) contribute to a conflict viz-a-viz JLS 7.4.3? Due to 
lack of export we may have to say: a non-java package in the current module vs. 
an exported java package from a read module, is that a conflict? Who should 
signal the conflict, if any?

- Is it true that a folder that is not a package never conflicts with a 
same-named folder in another module?

- What is the rationale for the fact that adding module-info may result in more 
packages (due to difference (3) vs. (4))?

- Where can we read the difference between what JLS allows (including concealed 
packages of the same name in different modules) vs. what the boot layer 
implementation accepts? Should we assume that the boot layer behaves as if 
created by ModuleLayer.defineModulesWithOneLoader()? The latter's javadoc is the 
only spec I could find mentioning the problem of "overlapping packages", but 
clearly that method cannot create the boot layer due to the restriction 
regarding java.base.

- javadoc of Module.getResourceAsStream() has a list of two bullets introduced 
by "Whether a resource can be located or not is determined as follows:". The two 
bullets may be at conflict, should we assume that the second bullet starts with 
"Otherwise"?

- the same javadoc speaks only of unnamed / named modules. But what happens in 
an automatic module? Isn't the javadoc saying that an automatic (=named) module 
may consider a non-java resource to be in a package, while according to (3) 
automatic modules have no non-java packages?


FWIW, the whole thing surfaced because adding module-info to some libraries 
triggered the mentioned LayerInstantiationException, the culprit being s.t. like 
about_files/EPL.txt appearing in each library. While the exception itself was a 
bit of a shock, I was then surprise to see that renaming about_files to 
about-files resolves the problem. If that is an intended solution, shouldn't it 
be much more advertised? Note that no individual library owner will see any 
problem, the problem only arises when trying to combine several unfortunate 
libraries.

Stephan

On 26.08.19 10:13, Alan Bateman wrote:
> On 24/08/2019 21:11, Stephan Herrmann wrote:
>> Hi,
>>
>> back then when working on our compiler I didn't pay much attention to non-Java 
>> resources, as JLS doesn't make any mention of them.
>>
>> Recently, however, we found that plain text files in a modular jar can well 
>> cause the VM to refuse starting, 
> I think you are mainly asking how to determine the set of packages in a (named) 
> module.
> 
> Automatic modules are straight-forward. The set of packages is determined from 
> the non-directory entries in the JAR file that end in ".class". The API docs for 
> ModuleFinder.of(Path...) have all the details.
> 
> Explicit modules have more to their story. They can be exploded on the file 
> system or may be packaged into modular JAR or other formats. What is the set of 
> packages in an explicit module? The Module class file attribute can help as it 
> contains the set of packages that are exported, open or contain service provider 
> implementations. However, it doesn't contain the set of packages that not 
> exported, not opened, or don't contain service provider implementations. To 
> determine the complete set of packages in a module will often require scanning 
> the contents of the module. This can mean scanning the file system (exploded 
> module) or scanning the contents of a JAR file (modular JAR). This is an area 
> where the ModuleFinder API docs could say more. When an exploded or modular JAR 
> is scanned, the entries are mapped to candidate package names. If an entry maps 
> to a legal package name then it will be added to set of packages in the module. 
> If you link this to the encapsulation specified by the Class/Module 
> getResourceXXX methods then it starts to become clear that non-class resources 
> will be encapsulated if the package isn't opened to other modules.
> 
> As an optimization (to avoid potentially expensive scanning) the module-info can 
> include a ModulePackages class file attribute that contains the set of packages 
> in the module. The `jar` tool adds (or updates) this attribute when creating (or 
> updating) a modular JAR. This means the set of packages in the module may be 
> determined at packaging time rather than other phases and the scanning done at 
> packaging time needs to exactly match the scanning that would be done when the 
> ModulePackages attribute is not present Again, this is an area where the 
> ModuleFinder API docs could say more.
> 
> -Alan