Why not use the Manifest?

[mark.reinhold at oracle.com]

2015/9/28 9:49 -0700, peter.kriens at aqute.biz:
> The encoding of the module data in a binary class file remains hard to
> understand and seems to go against the best practices we learned the
> hard way over the past decades. (Debugging!)
> 
> Especially since the current proposal leaves the heavy lifting to
> build tooling. In the proposal, build tools cannot use the
> module-info.java since it does not contain sufficient
> information. (Versions being the most glaring omission for a build
> too.) Nor is the format extensible to contain build tool specific
> information. Ergo, the build tool will have to generate the
> module-info.class file.
> 
> This make the class file makes these tools unnecessary hard and slower
> because it must either use the compiler as an intermediate step or use
> a library like ASM to create the unreadable version of its metadata.
> 
> This is the first time in a very long time that I see a regression to
> binary files for meta data. Especially because there is already a good
> place that all build tools are already using: the manifest.

The reasons for expressing module declarations in source files and
compiling them into class files are mentioned in SotMS, but here's a
longer take on just this topic.

The Java programming language, at present, provides for the definition
of three kinds of program components: classes, interfaces, and packages.
They are defined in Java source files, and compiled into class files.
Such files govern, among other things, the mechanisms of symbolic
resolution and access control implemented by every Java compiler and
JVM.  A developer need not reason about any other type of file in order
to understand the actions of these mechanisms.  A compiler or JVM need
not consume or produce any information other than that found in source
and class files in order to implement these mechanisms.

The module system extends the mechanism of symbolic resolution with the
concept of readability in order to provide reliable configuration.  It
extends the mechanism of access control, in part by relating it to
readability, in order to provide strong encapsulation.  To support these
extensions, every Java compiler and JVM must be able to locate and
interpret descriptions of modules which convey, at least, each module's
name, dependences, and exported packages.  Such descriptions, regardless
of their form, will govern symbolic resolution and access control in both
compilers and JVMs, so they will have to be specified in both the Java
Language Specification (JLS) and the Java Virtual Machine Specification
(JVMS).

Regardless of how they are described, modules are, fundamentally, a new
kind of Java program component.  The present proposal therefore treats
them as such, in both the language and the JVM: Module descriptions are
expressed and encoded in the same way as the other kinds of information
that define program components and govern symbolic resolution and access
control -- that is, as Java source and class files.

This approach is immediately familiar to developers, who already think
about program components, symbolic resolution, and access control in
terms of the Java programming language.  It is easy to retrofit into the
JLS and the JVMS, since those specifications are already centered upon
source and class files.  It is, finally, straightforward for existing
Java implementations and tool chains to support, since they need not be
revised to handle an entirely new type of file.

We could choose another format for module descriptions such as XML, JSON,
YAML, or JAR-file manifests.  Such formats may be more convenient for
tool maintainers but we'd have to bake that format into the JLS and the
JVMS, thereby increasing the complexity of those specifications and their
implementations.  If the format depends upon external standards (XML,
JSON, YAML) then these foundational specifications, and their
implementations, would become dependent upon those standards.  If the
format proves unsuitable over time then replacing it with something else
would require major revisions to these specifications, and to their
implementations.

Source and class files are fundamental to the Java platform, to the JLS
and the JVMS, and to their implementations and supporting tools.  Other
formats come and go; these will be with us forever.

                                  * * *

As to debugging, in the prototype we've already enhanced the jar tool
with an option to print the descriptor of a modular JAR file.  I expect
similar support to show up in related tools such as jmod and javap, and
eventually in IDEs and other tools outside the JDK.

As to ease of tooling, most tools will only need to read module
descriptors, not write them, and the java.lang.module.ModuleDescriptor
class already provides convenient static methods for doing that.  Adding
similar methods to write descriptors would be straightforward, though it
would complicate the API a bit.

As to extensibility, class files are already in a precisely-specified and
extensible format.  If a build tool needs to add information to a module
descriptor then it can do so via non-standard class-file attributes,
using an existing popular library such as ASM.

To make it easier for tools to manipulate module descriptors we could
consider enhancing the java.lang.module.ModuleDescriptor class, and
related tools, to read and write custom, non-standard class-file
attributes.  This might be a bit tricky depending on how the attributes
are defined, but for simple property-style attributes it's likely
straightforward and would obviate the need to use ASM directly.

- Mark