Why not use the Manifest?

[Peter Kriens]

> On 15 okt. 2015, at 16:26, forax at univ-mlv.fr wrote:
> I assume that some developpers will want to declare the module-info by hand.
Ok, but isn’t this then always redundant? 

Isn’t the build system forced to have the same (or at least closely corresponding) information with a version?

> Imagine that i want a build system that obey to semver and calculate the version of the modules i create by itself.
> The version is calculated by checking the bytecodes against the latest produced artefact, and the version is bumped automatically.
> If there is no conflict, the build tool only needs the major version of the required modules.
It sounds a bit like what bnd does but I am not completely sure I understand what you mean. Could you elaborate?

Kind regards,

	Peter Kriens

> 
> regards,
> Rémi
> 
> ----- Mail original -----
>> De: "Peter Kriens" <peter.kriens at aqute.biz>
>> À: forax at univ-mlv.fr
>> Cc: "Mark Reinhold" <mark.reinhold at oracle.com>, jpms-spec-experts at openjdk.java.net
>> Envoyé: Jeudi 15 Octobre 2015 08:53:28
>> Objet: Re: Why not use the Manifest?
>> 
>> I think our differences come from the fact that you assume that
>> module-info.java will be written by developers while I fail how to see that
>> can work realistically.
>> 
>> The build system must already have metadata that contains module name and
>> version (both for the current project and the dependencies), for example the
>> pom.xml in maven. You think it is realistic to expect people will write the
>> module name in two places?
>> 
>> For all practical cases, will the module info not be generated by the build
>> system?
>> 
>> Kind regards,
>> 
>> 	Peter Kriens
>> 
>> 
>> 
>> 
>>> On 14 okt. 2015, at 21:28, forax at univ-mlv.fr wrote:
>>> 
>>> ----- Mail original -----
>>>> De: "Peter Kriens" <peter.kriens at aqute.biz>
>>>> À: "Remi Forax" <forax at univ-mlv.fr>
>>>> Cc: "Mark Reinhold" <mark.reinhold at oracle.com>,
>>>> jpms-spec-experts at openjdk.java.net
>>>> Envoyé: Mercredi 14 Octobre 2015 17:00:50
>>>> Objet: Re: Why not use the Manifest?
>>>> 
>>>> But packages are sealed by a manifest header, as is the signing structure.
>>>> So
>>>> the JVM must already read the manifest today?
>>>> 
>>>> Kind regards,
>>>> 
>>>> 	Peter Kriens
>>> 
>>> Some JVM parse the manifest, but it's not part of the JVM spec.
>>> 
>>> But where are losing ourselves in the details.
>>> The compiler should check if the module-info is well formed so the error
>>> will be reported at compile time and not at runtime.
>>> Now, if the compiler checks the module-info file, the textual
>>> representation should be described in the JLS and
>>> binary representation described in the JVMS.
>>> 
>>> Using a text file in that context will be weird.
>>> 
>>> regards,
>>> Rémi
>>> 
>>>> 
>>>>> On 14 okt. 2015, at 15:41, Remi Forax <forax at univ-mlv.fr> wrote:
>>>>> 
>>>>> I don't think so, it's part of the tools descriptions,
>>>>> the manifest is described as part of the java/jar commands.
>>>>> 
>>>>> Anyway, it's important that the module descriptor is checked at compile
>>>>> time (by javac) and at runtime (by java),
>>>>> thus, it has to be a part of Java the language with a syntax
>>>>> understandable
>>>>> by javac.
>>>>> 
>>>>> Rémi
>>>>> 
>>>>> ----- Mail original -----
>>>>>> De: "Peter Kriens" <peter.kriens at aqute.biz>
>>>>>> À: "Mark Reinhold" <mark.reinhold at oracle.com>
>>>>>> Cc: jpms-spec-experts at openjdk.java.net
>>>>>> Envoyé: Lundi 12 Octobre 2015 18:51:52
>>>>>> Objet: Re: Why not use the Manifest?
>>>>>> 
>>>>>> Isn’t the manifest already part of the JVM specification?
>>>>>> 
>>>>>> Kind regards,
>>>>>> 
>>>>>> 	Peter Kriens
>>>>>> 
>>>>>>> On 5 okt. 2015, at 20:50, mark.reinhold at oracle.com wrote:
>>>>>>> 
>>>>>>> 2015/9/28 9:49 -0700, peter.kriens at aqute.biz:
>>>>>>>> The encoding of the module data in a binary class file remains hard to
>>>>>>>> understand and seems to go against the best practices we learned the
>>>>>>>> hard way over the past decades. (Debugging!)
>>>>>>>> 
>>>>>>>> Especially since the current proposal leaves the heavy lifting to
>>>>>>>> build tooling. In the proposal, build tools cannot use the
>>>>>>>> module-info.java since it does not contain sufficient
>>>>>>>> information. (Versions being the most glaring omission for a build
>>>>>>>> too.) Nor is the format extensible to contain build tool specific
>>>>>>>> information. Ergo, the build tool will have to generate the
>>>>>>>> module-info.class file.
>>>>>>>> 
>>>>>>>> This make the class file makes these tools unnecessary hard and slower
>>>>>>>> because it must either use the compiler as an intermediate step or use
>>>>>>>> a library like ASM to create the unreadable version of its metadata.
>>>>>>>> 
>>>>>>>> This is the first time in a very long time that I see a regression to
>>>>>>>> binary files for meta data. Especially because there is already a good
>>>>>>>> place that all build tools are already using: the manifest.
>>>>>>> 
>>>>>>> The reasons for expressing module declarations in source files and
>>>>>>> compiling them into class files are mentioned in SotMS, but here's a
>>>>>>> longer take on just this topic.
>>>>>>> 
>>>>>>> The Java programming language, at present, provides for the definition
>>>>>>> of three kinds of program components: classes, interfaces, and
>>>>>>> packages.
>>>>>>> They are defined in Java source files, and compiled into class files.
>>>>>>> Such files govern, among other things, the mechanisms of symbolic
>>>>>>> resolution and access control implemented by every Java compiler and
>>>>>>> JVM.  A developer need not reason about any other type of file in order
>>>>>>> to understand the actions of these mechanisms.  A compiler or JVM need
>>>>>>> not consume or produce any information other than that found in source
>>>>>>> and class files in order to implement these mechanisms.
>>>>>>> 
>>>>>>> The module system extends the mechanism of symbolic resolution with the
>>>>>>> concept of readability in order to provide reliable configuration.  It
>>>>>>> extends the mechanism of access control, in part by relating it to
>>>>>>> readability, in order to provide strong encapsulation.  To support
>>>>>>> these
>>>>>>> extensions, every Java compiler and JVM must be able to locate and
>>>>>>> interpret descriptions of modules which convey, at least, each module's
>>>>>>> name, dependences, and exported packages.  Such descriptions,
>>>>>>> regardless
>>>>>>> of their form, will govern symbolic resolution and access control in
>>>>>>> both
>>>>>>> compilers and JVMs, so they will have to be specified in both the Java
>>>>>>> Language Specification (JLS) and the Java Virtual Machine Specification
>>>>>>> (JVMS).
>>>>>>> 
>>>>>>> Regardless of how they are described, modules are, fundamentally, a new
>>>>>>> kind of Java program component.  The present proposal therefore treats
>>>>>>> them as such, in both the language and the JVM: Module descriptions are
>>>>>>> expressed and encoded in the same way as the other kinds of information
>>>>>>> that define program components and govern symbolic resolution and
>>>>>>> access
>>>>>>> control -- that is, as Java source and class files.
>>>>>>> 
>>>>>>> This approach is immediately familiar to developers, who already think
>>>>>>> about program components, symbolic resolution, and access control in
>>>>>>> terms of the Java programming language.  It is easy to retrofit into
>>>>>>> the
>>>>>>> JLS and the JVMS, since those specifications are already centered upon
>>>>>>> source and class files.  It is, finally, straightforward for existing
>>>>>>> Java implementations and tool chains to support, since they need not be
>>>>>>> revised to handle an entirely new type of file.
>>>>>>> 
>>>>>>> We could choose another format for module descriptions such as XML,
>>>>>>> JSON,
>>>>>>> YAML, or JAR-file manifests.  Such formats may be more convenient for
>>>>>>> tool maintainers but we'd have to bake that format into the JLS and the
>>>>>>> JVMS, thereby increasing the complexity of those specifications and
>>>>>>> their
>>>>>>> implementations.  If the format depends upon external standards (XML,
>>>>>>> JSON, YAML) then these foundational specifications, and their
>>>>>>> implementations, would become dependent upon those standards.  If the
>>>>>>> format proves unsuitable over time then replacing it with something
>>>>>>> else
>>>>>>> would require major revisions to these specifications, and to their
>>>>>>> implementations.
>>>>>>> 
>>>>>>> Source and class files are fundamental to the Java platform, to the JLS
>>>>>>> and the JVMS, and to their implementations and supporting tools.  Other
>>>>>>> formats come and go; these will be with us forever.
>>>>>>> 
>>>>>>>                               * * *
>>>>>>> 
>>>>>>> As to debugging, in the prototype we've already enhanced the jar tool
>>>>>>> with an option to print the descriptor of a modular JAR file.  I expect
>>>>>>> similar support to show up in related tools such as jmod and javap, and
>>>>>>> eventually in IDEs and other tools outside the JDK.
>>>>>>> 
>>>>>>> As to ease of tooling, most tools will only need to read module
>>>>>>> descriptors, not write them, and the java.lang.module.ModuleDescriptor
>>>>>>> class already provides convenient static methods for doing that.
>>>>>>> Adding
>>>>>>> similar methods to write descriptors would be straightforward, though
>>>>>>> it
>>>>>>> would complicate the API a bit.
>>>>>>> 
>>>>>>> As to extensibility, class files are already in a precisely-specified
>>>>>>> and
>>>>>>> extensible format.  If a build tool needs to add information to a
>>>>>>> module
>>>>>>> descriptor then it can do so via non-standard class-file attributes,
>>>>>>> using an existing popular library such as ASM.
>>>>>>> 
>>>>>>> To make it easier for tools to manipulate module descriptors we could
>>>>>>> consider enhancing the java.lang.module.ModuleDescriptor class, and
>>>>>>> related tools, to read and write custom, non-standard class-file
>>>>>>> attributes.  This might be a bit tricky depending on how the attributes
>>>>>>> are defined, but for simple property-style attributes it's likely
>>>>>>> straightforward and would obviate the need to use ASM directly.
>>>>>>> 
>>>>>>> - Mark
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>>