Explicit Layer Configuration [Was: Why not use the Manifest?]

Nicolai Parlog nipa at codefx.org
Thu Oct 22 05:48:03 UTC 2015 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

 Hi!

I think Ali's proposal is very interesting to think through. Giving
developers and build tools a way to explicitly create complex
configurations with multiple layers would be great! For one it would
close the chapter of JAR hell by allowing conflicting module versions.

Looking at the current goals, requirements and capabilities this fits
in quite nicely: It does not implement version selection and does not
require new module system capabilities.

It would be great to hear the EG's thoughts on this.

 so long ... Nicolai



On 15.10.2015 13:40, Ali Ebrahimi wrote:
> Hi, I don't think manipulating module-info by build system is good
> way. I think module-info should programmed by module owner and even
> versioned by him in source code form. This way code reviewer can
> find out what version of module he is exploring.
> 
> But, since in current design, dependency management is out of scope
> and this responsibility assigned to build systems so how this build
> systems can help module system? So there should be some way(or
> where) from that build systems can guide module system. Based on
> current prototype there is a concept named Configuration(can we 
> call this LayerConfiguration) and Layer that used for module graph
> building that allows for complex and multi level dependency graphs
> that currently can be only build programmatically. Why build
> systems (even in simple cases developer) can not build such graphs
> by some format that is known for all build systems and module
> system itself. So I propose new artifact by some standard name as 
> "module-configuraction.xml" that can be created by build systems or
> even by developer when not use any build system and this file
> passed or located by java launcher: java -c
> module-configuraction.xml
> 
> module-configuraction.xml can include any info that is needed for
> running application such as main module.
> 
> What do you thinks?
> 
> On Thu, Oct 15, 2015 at 10:23 AM, Peter Kriens
> <peter.kriens at aqute.biz> wrote:
> 
>> I think our differences come from the fact that you assume that 
>> module-info.java will be written by developers while I fail how
>> to see that can work realistically.
>> 
>> The build system must already have metadata that contains module
>> name and version (both for the current project and the
>> dependencies), for example the pom.xml in maven. You think it is
>> realistic to expect people will write the module name in two
>> places?
>> 
>> For all practical cases, will the module info not be generated by
>> the build system?
>> 
>> Kind regards,
>> 
>> Peter Kriens
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>>> On 14 okt. 2015, at 21:28, forax at univ-mlv.fr wrote:
>>> 
>>> ----- Mail original -----
>>>> De: "Peter Kriens" <peter.kriens at aqute.biz> À: "Remi Forax"
>>>> <forax at univ-mlv.fr> Cc: "Mark Reinhold"
>>>> <mark.reinhold at oracle.com>,
>> jpms-spec-experts at openjdk.java.net
>>>> Envoyé: Mercredi 14 Octobre 2015 17:00:50 Objet: Re: Why not
>>>> use the Manifest?
>>>> 
>>>> But packages are sealed by a manifest header, as is the
>>>> signing
>> structure. So
>>>> the JVM must already read the manifest today?
>>>> 
>>>> Kind regards,
>>>> 
>>>> Peter Kriens
>>> 
>>> Some JVM parse the manifest, but it's not part of the JVM
>>> spec.
>>> 
>>> But where are losing ourselves in the details. The compiler
>>> should check if the module-info is well formed so the error
>> will be reported at compile time and not at runtime.
>>> Now, if the compiler checks the module-info file, the textual
>> representation should be described in the JLS and
>>> binary representation described in the JVMS.
>>> 
>>> Using a text file in that context will be weird.
>>> 
>>> regards, Rémi
>>> 
>>>> 
>>>>> On 14 okt. 2015, at 15:41, Remi Forax <forax at univ-mlv.fr>
>>>>> wrote:
>>>>> 
>>>>> I don't think so, it's part of the tools descriptions, the
>>>>> manifest is described as part of the java/jar commands.
>>>>> 
>>>>> Anyway, it's important that the module descriptor is
>>>>> checked at compile time (by javac) and at runtime (by
>>>>> java), thus, it has to be a part of Java the language with
>>>>> a syntax
>> understandable
>>>>> by javac.
>>>>> 
>>>>> Rémi
>>>>> 
>>>>> ----- Mail original -----
>>>>>> De: "Peter Kriens" <peter.kriens at aqute.biz> À: "Mark
>>>>>> Reinhold" <mark.reinhold at oracle.com> Cc:
>>>>>> jpms-spec-experts at openjdk.java.net Envoyé: Lundi 12
>>>>>> Octobre 2015 18:51:52 Objet: Re: Why not use the
>>>>>> Manifest?
>>>>>> 
>>>>>> Isn’t the manifest already part of the JVM
>>>>>> specification?
>>>>>> 
>>>>>> Kind regards,
>>>>>> 
>>>>>> Peter Kriens
>>>>>> 
>>>>>>> On 5 okt. 2015, at 20:50, mark.reinhold at oracle.com
>>>>>>> wrote:
>>>>>>> 
>>>>>>> 2015/9/28 9:49 -0700, peter.kriens at aqute.biz:
>>>>>>>> The encoding of the module data in a binary class
>>>>>>>> file remains hard
>> to
>>>>>>>> understand and seems to go against the best practices
>>>>>>>> we learned the hard way over the past decades.
>>>>>>>> (Debugging!)
>>>>>>>> 
>>>>>>>> Especially since the current proposal leaves the
>>>>>>>> heavy lifting to build tooling. In the proposal,
>>>>>>>> build tools cannot use the module-info.java since it
>>>>>>>> does not contain sufficient information. (Versions
>>>>>>>> being the most glaring omission for a build too.) Nor
>>>>>>>> is the format extensible to contain build tool
>>>>>>>> specific information. Ergo, the build tool will have
>>>>>>>> to generate the module-info.class file.
>>>>>>>> 
>>>>>>>> This make the class file makes these tools
>>>>>>>> unnecessary hard and
>> slower
>>>>>>>> because it must either use the compiler as an
>>>>>>>> intermediate step or
>> use
>>>>>>>> a library like ASM to create the unreadable version
>>>>>>>> of its metadata.
>>>>>>>> 
>>>>>>>> This is the first time in a very long time that I see
>>>>>>>> a regression
>> to
>>>>>>>> binary files for meta data. Especially because there
>>>>>>>> is already a
>> good
>>>>>>>> place that all build tools are already using: the
>>>>>>>> manifest.
>>>>>>> 
>>>>>>> The reasons for expressing module declarations in
>>>>>>> source files and compiling them into class files are
>>>>>>> mentioned in SotMS, but here's a longer take on just
>>>>>>> this topic.
>>>>>>> 
>>>>>>> The Java programming language, at present, provides for
>>>>>>> the
>> definition
>>>>>>> of three kinds of program components: classes,
>>>>>>> interfaces, and
>> packages.
>>>>>>> They are defined in Java source files, and compiled
>>>>>>> into class files. Such files govern, among other
>>>>>>> things, the mechanisms of symbolic resolution and
>>>>>>> access control implemented by every Java compiler and 
>>>>>>> JVM.  A developer need not reason about any other type
>>>>>>> of file in
>> order
>>>>>>> to understand the actions of these mechanisms.  A
>>>>>>> compiler or JVM
>> need
>>>>>>> not consume or produce any information other than that
>>>>>>> found in
>> source
>>>>>>> and class files in order to implement these
>>>>>>> mechanisms.
>>>>>>> 
>>>>>>> The module system extends the mechanism of symbolic
>>>>>>> resolution with
>> the
>>>>>>> concept of readability in order to provide reliable
>>>>>>> configuration.
>> It
>>>>>>> extends the mechanism of access control, in part by
>>>>>>> relating it to readability, in order to provide strong
>>>>>>> encapsulation.  To support
>> these
>>>>>>> extensions, every Java compiler and JVM must be able to
>>>>>>> locate and interpret descriptions of modules which
>>>>>>> convey, at least, each
>> module's
>>>>>>> name, dependences, and exported packages.  Such
>>>>>>> descriptions,
>> regardless
>>>>>>> of their form, will govern symbolic resolution and
>>>>>>> access control in
>> both
>>>>>>> compilers and JVMs, so they will have to be specified
>>>>>>> in both the
>> Java
>>>>>>> Language Specification (JLS) and the Java Virtual
>>>>>>> Machine
>> Specification
>>>>>>> (JVMS).
>>>>>>> 
>>>>>>> Regardless of how they are described, modules are,
>>>>>>> fundamentally, a
>> new
>>>>>>> kind of Java program component.  The present proposal
>>>>>>> therefore
>> treats
>>>>>>> them as such, in both the language and the JVM: Module
>>>>>>> descriptions
>> are
>>>>>>> expressed and encoded in the same way as the other
>>>>>>> kinds of
>> information
>>>>>>> that define program components and govern symbolic
>>>>>>> resolution and
>> access
>>>>>>> control -- that is, as Java source and class files.
>>>>>>> 
>>>>>>> This approach is immediately familiar to developers,
>>>>>>> who already
>> think
>>>>>>> about program components, symbolic resolution, and
>>>>>>> access control in terms of the Java programming
>>>>>>> language.  It is easy to retrofit into
>> the
>>>>>>> JLS and the JVMS, since those specifications are
>>>>>>> already centered
>> upon
>>>>>>> source and class files.  It is, finally,
>>>>>>> straightforward for existing Java implementations and
>>>>>>> tool chains to support, since they need not
>> be
>>>>>>> revised to handle an entirely new type of file.
>>>>>>> 
>>>>>>> We could choose another format for module descriptions
>>>>>>> such as XML,
>> JSON,
>>>>>>> YAML, or JAR-file manifests.  Such formats may be more
>>>>>>> convenient for tool maintainers but we'd have to bake
>>>>>>> that format into the JLS and
>> the
>>>>>>> JVMS, thereby increasing the complexity of those
>>>>>>> specifications and
>> their
>>>>>>> implementations.  If the format depends upon external
>>>>>>> standards (XML, JSON, YAML) then these foundational
>>>>>>> specifications, and their implementations, would become
>>>>>>> dependent upon those standards.  If the format proves
>>>>>>> unsuitable over time then replacing it with something
>> else
>>>>>>> would require major revisions to these specifications,
>>>>>>> and to their implementations.
>>>>>>> 
>>>>>>> Source and class files are fundamental to the Java
>>>>>>> platform, to the
>> JLS
>>>>>>> and the JVMS, and to their implementations and
>>>>>>> supporting tools.
>> Other
>>>>>>> formats come and go; these will be with us forever.
>>>>>>> 
>>>>>>> * * *
>>>>>>> 
>>>>>>> As to debugging, in the prototype we've already
>>>>>>> enhanced the jar tool with an option to print the
>>>>>>> descriptor of a modular JAR file.  I
>> expect
>>>>>>> similar support to show up in related tools such as
>>>>>>> jmod and javap,
>> and
>>>>>>> eventually in IDEs and other tools outside the JDK.
>>>>>>> 
>>>>>>> As to ease of tooling, most tools will only need to
>>>>>>> read module descriptors, not write them, and the
>> java.lang.module.ModuleDescriptor
>>>>>>> class already provides convenient static methods for
>>>>>>> doing that.
>> Adding
>>>>>>> similar methods to write descriptors would be
>>>>>>> straightforward,
>> though it
>>>>>>> would complicate the API a bit.
>>>>>>> 
>>>>>>> As to extensibility, class files are already in a
>> precisely-specified and
>>>>>>> extensible format.  If a build tool needs to add
>>>>>>> information to a
>> module
>>>>>>> descriptor then it can do so via non-standard
>>>>>>> class-file attributes, using an existing popular
>>>>>>> library such as ASM.
>>>>>>> 
>>>>>>> To make it easier for tools to manipulate module
>>>>>>> descriptors we could consider enhancing the
>>>>>>> java.lang.module.ModuleDescriptor class, and related
>>>>>>> tools, to read and write custom, non-standard
>>>>>>> class-file attributes.  This might be a bit tricky
>>>>>>> depending on how the
>> attributes
>>>>>>> are defined, but for simple property-style attributes
>>>>>>> it's likely straightforward and would obviate the need
>>>>>>> to use ASM directly.
>>>>>>> 
>>>>>>> - Mark
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>> 
> 
> 

- -- 

PGP Key:
    http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
    http://codefx.org
        a blog about software development
    http://do-foss.de
        Free and Open Source Software for the City of Dortmund

Twitter:
    https://twitter.com/nipafx

Diaspora:
    nipa at pod.geraspora.de
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWKHiSAAoJEMo7rS6czNUJOAYP/Rw9D6H+yXtFDl0Q3HUGsj7D
YC+0f9bn0+5cvH9qIJE/VPrm7TMopKuCHeMivYgnkHSYLq6ybz3y0uudkUliVJ3h
4il6ZxLaj+JXGPFjQ0dgBjnqUzrIouYXSWgerEVc0rtD2HQ+2Cpzbt9tLYxhYOOF
8qRl0ibNu8CNFautZO6/yHr9AERvXJ5itxe2ak+6vKJxpBeg3kT0yTLNUlKIslch
UA6gO7CStLkbkMukkcmQxaOJPfdF8JEAVAn+LGpFnE3MDLLHUu+YL9w0//+h7NB1
p0f8nme2qOPVNSVnXNcVqlAQf1F5yzvZ99qFw1CM9/qJqeMt7KbOpH7ELizw2fFy
prmylEeLVEPr5HJmy3LxhZXey7rKLkEirb7e0HAnXjQS+Z6bLPAYpYfNKJykT+0Y
krK1twpAWBNvS9pH/DK0N3mHvDAktfq/iF+s/Bt/NcDJkrPF8Lltyt1FMeBYrwxE
gQHbMUDOxdodZen7e/sIiyfhp9OQY9A2H2zgoY5E9/+p8x41nCNZ83tiWDhE1P11
4CwewSJ2Wph1HXAG531oGTLuKy2dD1ZlqewC7JfFOk2DpwPb/LDLmRBdngbx7CuD
jNJ3eruTZtPJMaQa0vP0cwqIJIJudHtw797QjZ446DYQ95fWK1lqU1EZTkUtRXUD
7tv99CApHhBrbbkoLicg
=Av01
-----END PGP SIGNATURE-----

More information about the jpms-spec-observers mailing list