From forax at univ-mlv.fr Sun Dec 4 15:28:01 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Sun, 4 Dec 2016 16:28:01 +0100 (CET)
Subject: System Modules
Message-ID: <2032143511.2166652.1480865281798.JavaMail.zimbra@u-pem.fr>
I do not like the current definition of what a system module is, i.e. the modules returned by ModuleFinder.ofSystem().
Maybe i'm wrong but it's all modules available through jrt, so if one uses an image created by jlink, the application modules are considered as system modules (because there are also loaded by jrt) which seems wrong to me.
I think we should de-couple the notion of being in the jrt image and being a system module.
regards,
R?mi
From mark.reinhold at oracle.com Mon Dec 5 18:44:07 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 05 Dec 2016 10:44:07 -0800
Subject: Split packages
In-Reply-To: <871414197.2537969.1480174302378.JavaMail.zimbra@u-pem.fr>
References: <871414197.2537969.1480174302378.JavaMail.zimbra@u-pem.fr>
Message-ID: <20161205104407.235563@eggemoggin.niobe.net>
2016/11/26 7:31:42 -0800, forax at univ-mlv.fr:
> I'm trying to be able to see mavencentral as a provider of jigsaw
> modules (automatic for now, so as a flat hierarchy), and obviously,
> i've to solve the problem of split packages.
>
> There is an obvious way to solve the split packages issue, merge the
> modules that have split packages into one bigger module.
>
> Writing a ModuleFinder that virtually group two modules together is
> easy, so i wonder if there is a simple way to a user to specify a kind
> of automatic grouping module that group together several automatic
> modules ?
At the moment there is no such mechanism.
- Mark
From mark.reinhold at oracle.com Mon Dec 5 18:48:41 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 05 Dec 2016 10:48:41 -0800
Subject: Proposal: #ClassFileModuleName
In-Reply-To: <2034784044.1251203.1480547975649.JavaMail.zimbra@u-pem.fr>
References: <20161122164735.6899C1893F@eggemoggin.niobe.net>
<288915504.1641396.1479931405933.JavaMail.zimbra@u-pem.fr>
<2034784044.1251203.1480547975649.JavaMail.zimbra@u-pem.fr>
Message-ID: <20161205104841.830647548@eggemoggin.niobe.net>
2016/11/30 15:19:35 -0800, forax at univ-mlv.fr:
> I've just finished to update ASM6 to use the new classfile format.
>
> Practically, setting this_class to zero create several NPEs in the the
> visitors provided with ASM because a lot of them try to use indexOf on
> the name of the class (to find the package name).
>
> Given that the classfile for a package-info is named
> "foo/bar/package-info", i think it's a good idea to use "module-info"
> instead of null as class name for a module-info.class, it will help at
> least my users .
Okay, that's straightforward.
- Mark
From mark.reinhold at oracle.com Mon Dec 5 19:01:21 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 05 Dec 2016 11:01:21 -0800
Subject: System Modules
In-Reply-To: <2032143511.2166652.1480865281798.JavaMail.zimbra@u-pem.fr>
References: <2032143511.2166652.1480865281798.JavaMail.zimbra@u-pem.fr>
Message-ID: <20161205110121.610821898@eggemoggin.niobe.net>
2016/12/4 7:28:01 -0800, forax at univ-mlv.fr:
> I do not like the current definition of what a system module is,
> i.e. the modules returned by ModuleFinder.ofSystem().
>
> Maybe i'm wrong but it's all modules available through jrt, so if one
> uses an image created by jlink, the application modules are considered
> as system modules (because there are also loaded by jrt)
Correct.
> which seems
> wrong to me.
>
> I think we should de-couple the notion of being in the jrt image and
> being a system module.
The "system" modules are exactly those built-in to a run-time image,
though we should make that clearer in the API specification.
If you want a module finder for just the Java SE Platform modules,
or for all non-application system modules (i.e., SE + JDK), then
it's pretty easy to construct a filtering module finder to do that.
Do you think such finder(s) should be provided by the API?
- Mark
From mark.reinhold at oracle.com Mon Dec 5 23:05:56 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 5 Dec 2016 15:05:56 -0800 (PST)
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <19808667-7501-fc47-009d-b245e3a099ce@redhat.com>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<20161130000802.D65C81C6B0@eggemoggin.niobe.net>
<19808667-7501-fc47-009d-b245e3a099ce@redhat.com>
Message-ID: <20161205230556.5099720E1F@eggemoggin.niobe.net>
2016/11/29 18:47:33 -0800, david.lloyd at redhat.com:
> On 11/29/2016 06:08 PM, mark.reinhold at oracle.com wrote:
>> ...
>>
>> I'm concerned mostly about the complexity of the language specification,
>> which affects all users of this module system. If we change the language
>> of module declarations to allow arbitrary module names, possibly via some
>> sort of quoting scheme, then that's something that every developer would
>> have to understand even though it would most likely be of benefit to few.
>
> You don't need a quoting scheme to allow arbitrary names - you just
> allow them - and I don't see how this complicates the language
> specification in any way: in fact it should simplify it. There's just
> no reason to use "internal form".
At present the JLS does not allow identifiers to be composed of arbitrary
characters. If that's what you really want in the source language then
you'd have to introduce some sort of quoting scheme to allow whitespace,
all punctuation characters, and whatever else isn't already allowed by
`Character::isJavaIdentifierPart`. That would add significant complexity
to the JLS and, more importantly, would be something that every single
Java developer would have to learn.
Perhaps you don't really mean "arbitrary", however, just "something more
than is allowed in a Java qualified identifier".
>> I'm concerned also, though to a lesser degree, about the complexity of
>> the class-file specification (i.e., the JVMS), the long-term evolvability
>> of that specification, and the complexity of code that reads and writes
>> class files, though the latter is second-order since it mostly affects
>> maintainers of IDEs, compilers, and other kinds of tools rather than
>> developers in general. If there's a compelling reason to lift the usual
>> restrictions on the representation of qualified names in class files, at
>> least in the case of module names, then I'd like to hear it.
>
> I'm just really confused at this line of justification. Making module
> names be a qualified name is the very thing that I'm arguing against.
> Don't make any change to the restrictions on qualified names, and don't
> make module names be qualified names; that's all there is to it.
Okay, but in the language if an identifier is not a qualified name then
it cannot contain `.` characters, so using simple identifiers for module
names is a non-starter.
We could revise the JLS to introduce a brand-new kind of identifier,
just for module names, that allows `.` and maybe a few other additional
special characters, and encode these without using internal form in
`module-info.class` files. This new kind of identifier would, however,
be something that every Java developer must learn about.
Who, in practical terms, would benefit from this added complexity?
You've said that JBoss Modules allows arbitrary characters in module
names, but do you or your users actually use such names in practice?
If so, can you show us some examples?
>> ...
>>
>>
>> I don't expect Java EE modules to map directly to Java SE modules, nor
>> do any of the Java EE spec leads with whom I've discussed this issue.
>> EE modules and SE modules are completely different kinds of things.
>
> On March 11 of this year (and other occasions) I specifically asked
> exactly that, and you said [1] "Of course we have that expectation --
> that's why the requirements include an entire section on dynamic
> configuration". Did I misunderstand you then or now, or has something
> changed?
I don't think anything has changed.
I didn't mean for that statement to imply that I thought that every EE
module would map directly to an SE module, though I see how it can be
read that way. Apologies for the confusion.
- Mark
From mark.reinhold at oracle.com Mon Dec 5 23:06:56 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 5 Dec 2016 15:06:56 -0800 (PST)
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <69a96954-009a-125d-98dd-0f3cb341e59d@redhat.com>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<2034282380.1641047.1479931198355.JavaMail.zimbra@u-pem.fr>
<20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
<69a96954-009a-125d-98dd-0f3cb341e59d@redhat.com>
Message-ID: <20161205230656.541C020E24@eggemoggin.niobe.net>
2016/11/29 18:50:11 -0800, david.lloyd at redhat.com:
> On 11/29/2016 06:11 PM, mark.reinhold at oracle.com wrote:
>> ...
>>
>> As I wrote in my reply to David, I'm open to lifting the traditional
>> restrictions on the class-file representation of qualified names in the
>> case of module names. Given the weight of tradition and the past value
>> of the existing constraints, however, I'd like to have a more compelling
>> reason than "some future hypothetical module system might need this
>> flexibility".
>
> The key point is exactly that a present, concrete module system does
> need this flexibility; ours, specifically. And this is such an easy change.
No, it's not, as I've tried to explain nearby.
>> In trying to think about the future I do wonder if, today, we should
>> reserve a character or two just in case we discover five or ten years
>> from now that we need to add more structure to module names. Should
>> we set aside `:`, or perhaps some other character, just in case?
>
> We have had a module system with no character restrictions for many
> years now and never looked back. It adds flexibility as every module
> loader can establish acceptable name rules based on its policy, which
> afforded us a great deal of interoperability with all manner of system.
> You can still have reserved characters if that's what is appropriate to
> your module resolver; if a name is invalid for that resolver, the
> resolver can simply return no match.
I'm puzzled. The "Module names" page [1] in the documentation of JBoss
Modules says:
A module name is a dot-separated string which is used along with a
version slot identifier to uniquely identify a module within a module
loader. Names are typically organized along the same lines as package
names, however there is no specific relationship between a module name
and the packages it contains. These are all examples of typical module
names:
org.apache.commons.logging
org.jboss.remoting
cglib
javax.ejb.api
ch.qos.cal10n
...
Version slot identifiers are used when you wish to have more than one
instance of a module in a module loader under the same name. ...
When identifying a module in a string, the version slot identifier can
be appended to the module name, separated by a colon ":". For example,
the following two module identifier strings refer to the same module:
org.jboss.remoting:main
org.jboss.remoting
The following three module identifier strings refer to different
modules:
org.jboss.remoting:2
org.jboss.remoting:3
org.jboss.remoting
A related page, on "Module descriptors" [2], describes the algorithm for
locating `module.xml` files:
... its location on the filesystem is calculated by converting the
dot-separated segments of the module name to path elements, followed by
a path element which consists of the version slot for that module. This
path is then appended to each module path root in turn until a file
named module.xml is found within it.
Unless I'm misunderstanding something, or the documentation is incorrect
or incomplete, then module names in JBoss Modules cannot be arbitrary
strings. They cannot contain colons, since colons are used to separate
module names from version-slot identifiers, nor can they contain repeated
dots or slashes, since dots are transformed into slashes and repeated
slashes are, in most filesystems, equivalent to a single slash. (They
probably also can't contain characters that are illegal in the underlying
filesystem unless some sort of escaping is done, but the documentation
does not address this.)
More to the point, every single module name in the examples in the JBoss
Modules documentation would be considered a qualified identifier in Java.
When and in what circumstances are truly arbitrary module names used in
JBoss Modules? Or is this simply a theoretical concern on your part?
- Mark
[1] https://docs.jboss.org/author/display/MODULES/Module+names
[2] https://docs.jboss.org/author/display/MODULES/Module+descriptors
From mark.reinhold at oracle.com Mon Dec 5 23:07:56 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 5 Dec 2016 15:07:56 -0800 (PST)
Subject: Discussion: #MutableConfigurations,
#LazyConfigurationAndInstantiation, #CyclicDependences,
& #DiscardableModules
In-Reply-To: <7bb95e6d-a127-ea4f-7158-d9f062e8aed3@redhat.com>
References: <20161118163009.8C54A16AC5@eggemoggin.niobe.net>
<8e5bd410-9737-db44-6c2f-4289ad4c8db2@redhat.com>
<20161121135201.534880609@eggemoggin.niobe.net>
<e168d89d-7d4a-0492-d18b-dab94b42dd43@redhat.com>
<20161130000902.DB5A31C6B3@eggemoggin.niobe.net>
<7bb95e6d-a127-ea4f-7158-d9f062e8aed3@redhat.com>
Message-ID: <20161205230756.5A21A20E29@eggemoggin.niobe.net>
2016/11/29 18:47:59 -0800, david.lloyd at redhat.com:
> On 11/29/2016 06:09 PM, mark.reinhold at oracle.com wrote:
>> 2016/11/22 13:19:19 -0800, david.lloyd at redhat.com:
>>> On 11/21/2016 03:52 PM, mark.reinhold at oracle.com wrote:
>>>> ...
>>>>
>>>> If you're using an approach similar to what I suggested for OSGi [1]
>>>> then you should be able to implement cycles amongst JBoss modules by
>>>> inserting the necessary readability edges after each module is
>>>> instantiated in its own layer. This will be much easier with the new
>>>> API proposed for #ReadabilityAddedByLayerCreator [2].
>>>
>>> I asked earlier whether this allows the class resolver
>>
>> (What's a "class resolver" in this context?)
>
> The part of the class loader that resolves references between classes
> and performs linkage. There seems to be no non-overloaded term for
> this, anymore.
If that's what you mean by "class resolver" then you're really talking
about class loaders, not the module system, or at least not JPMS, since
JPMS doesn't change how class loading works.
>> ...
>>
>> If you want to allow cycles amongst your own modules then you can resolve
>> them yourself and add whatever cycle-inducing readability edges you need.
>> That is, as I understand it, how Watson's OSGi embedding works.
>
> But it fails when the actual classes have direct references to one
> another.
Can you be more specific? How does it fail? Is Watson's OSGi embedding
broken?
In addition to writing your own resolver you can -- and I've assumed that
you, like Watson, would -- write your own class loaders to resolve class
references in whatever way you require, even looking up classes in nearby
cyclically-related modules according to whatever module-graph data
structure you construct.
Unless that approach is broken then my original question remains [3]:
Is this approach workable for JBoss Modules as well? If so then I'd
like to close these issues out; if not then I'd like to understand if
there are additional, smaller changes that would make this approach
acceptable while avoiding the complexity of complete solutions to
these issues.
- Mark
[1] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-October/000410.html
[2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-November/000456.html
[3] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-November/000458.html
From mark.reinhold at oracle.com Mon Dec 5 23:08:56 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 5 Dec 2016 15:08:56 -0800 (PST)
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <2c7a0dbb-3666-dd7a-3260-e8bc4b8c10f2@redhat.com>
References: <20161031202238.D3E5DDB037@eggemoggin.niobe.net>
<0f23ba26-9e93-a62b-b537-be0d5a91b895@redhat.com>
<20161121135638.309938232@eggemoggin.niobe.net>
<0a1e41c6-9d36-ad5a-5ad0-7445e7ae3883@redhat.com>
<20161130001002.E23C91C6BC@eggemoggin.niobe.net>
<2c7a0dbb-3666-dd7a-3260-e8bc4b8c10f2@redhat.com>
Message-ID: <20161205230856.5E5A220E2E@eggemoggin.niobe.net>
2016/11/29 18:51:22 -0800, david.lloyd at redhat.com:
> On 11/29/2016 06:10 PM, mark.reinhold at oracle.com wrote:
>> 2016/11/22 13:23:34 -0800, david.lloyd at redhat.com:
>>> ...
>>>
>>> To degenerate the problem to the simplest case: I may have to layers, A
>>> and B, which are unrelated and both contain a module X. I may need the
>>> module X in one layer to have a dependency on the module X in another
>>> layer. Unless there's new code I haven't seen, each dependency is
>>> expressed by name only, therefore this is impossible. Whereas in our
>>> system, a dependency is a tuple of (module loader, module name),
>>> allowing arbitrary weaving of modules from different name spaces with no
>>> problem.
>>>
>>> I don't think we need it to work the exact same way, but we need some
>>> equivalent.
>>
>> If this is how you need resolution to work for your modules then can't
>> you write your own resolver to make it so, using whatever additional
>> (weak) data structures you need to associate names with layers, class
>> loaders, etc., in whatever manner you require?
>
> No, because the input to the resolver is just a name; there's no other
> context to say which layer the module comes from, without some kind of
> name mangling scheme which raises more problems (we ran into exactly
> this issue more than six years ago). In other words I can require a
> module named "X", but I can't say that I require the module "X" from
> this other layer/namespace; rather I have to flatten out all my
> namespaces in a way that can be mutually resolvable, which means that I
> must ultimately constrain user module names in some weird way. This is
> not so good when modules refer to one another by name programmatically,
> nor in diagnostic output.
So, what do you suggest we do?
- Mark
From david.lloyd at redhat.com Tue Dec 6 00:10:18 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Mon, 5 Dec 2016 18:10:18 -0600
Subject: Discussion: #MutableConfigurations,
#LazyConfigurationAndInstantiation, #CyclicDependences,
& #DiscardableModules
In-Reply-To: <20161205230756.5A21A20E29@eggemoggin.niobe.net>
References: <20161118163009.8C54A16AC5@eggemoggin.niobe.net>
<8e5bd410-9737-db44-6c2f-4289ad4c8db2@redhat.com>
<20161121135201.534880609@eggemoggin.niobe.net>
<e168d89d-7d4a-0492-d18b-dab94b42dd43@redhat.com>
<20161130000902.DB5A31C6B3@eggemoggin.niobe.net>
<7bb95e6d-a127-ea4f-7158-d9f062e8aed3@redhat.com>
<20161205230756.5A21A20E29@eggemoggin.niobe.net>
Message-ID: <dd19b793-c0fc-23d7-d76d-fd70832302d0@redhat.com>
On 12/05/2016 05:07 PM, mark.reinhold at oracle.com wrote:
> 2016/11/29 18:47:59 -0800, david.lloyd at redhat.com:
>> On 11/29/2016 06:09 PM, mark.reinhold at oracle.com wrote:
>>> 2016/11/22 13:19:19 -0800, david.lloyd at redhat.com:
>>>> On 11/21/2016 03:52 PM, mark.reinhold at oracle.com wrote:
>>>>> ...
>>>>>
>>>>> If you're using an approach similar to what I suggested for OSGi [1]
>>>>> then you should be able to implement cycles amongst JBoss modules by
>>>>> inserting the necessary readability edges after each module is
>>>>> instantiated in its own layer. This will be much easier with the new
>>>>> API proposed for #ReadabilityAddedByLayerCreator [2].
>>>>
>>>> I asked earlier whether this allows the class resolver
>>>
>>> (What's a "class resolver" in this context?)
>>
>> The part of the class loader that resolves references between classes
>> and performs linkage. There seems to be no non-overloaded term for
>> this, anymore.
>
> If that's what you mean by "class resolver" then you're really talking
> about class loaders, not the module system, or at least not JPMS, since
> JPMS doesn't change how class loading works.
>
>>> ...
>>>
>>> If you want to allow cycles amongst your own modules then you can resolve
>>> them yourself and add whatever cycle-inducing readability edges you need.
>>> That is, as I understand it, how Watson's OSGi embedding works.
>>
>> But it fails when the actual classes have direct references to one
>> another.
>
> Can you be more specific? How does it fail? Is Watson's OSGi embedding
> broken?
>
> In addition to writing your own resolver you can -- and I've assumed that
> you, like Watson, would -- write your own class loaders to resolve class
> references in whatever way you require, even looking up classes in nearby
> cyclically-related modules according to whatever module-graph data
> structure you construct.
Hmm, if this is true then maybe there isn't a problem anymore; I'll
update to the latest and see if I can make this work.
> Unless that approach is broken then my original question remains [3]:
>
> Is this approach workable for JBoss Modules as well? If so then I'd
> like to close these issues out; if not then I'd like to understand if
> there are additional, smaller changes that would make this approach
> acceptable while avoiding the complexity of complete solutions to
> these issues.
I'll try and produce a definitive answer as quickly as possible.
--
- DML
From david.lloyd at redhat.com Tue Dec 6 00:12:08 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Mon, 5 Dec 2016 18:12:08 -0600
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <20161205230656.541C020E24@eggemoggin.niobe.net>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<2034282380.1641047.1479931198355.JavaMail.zimbra@u-pem.fr>
<20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
<69a96954-009a-125d-98dd-0f3cb341e59d@redhat.com>
<20161205230656.541C020E24@eggemoggin.niobe.net>
Message-ID: <1108c38c-3449-915b-c6d4-29cae935a5ce@redhat.com>
On 12/05/2016 05:06 PM, mark.reinhold at oracle.com wrote:
> 2016/11/29 18:50:11 -0800, david.lloyd at redhat.com:
>> On 11/29/2016 06:11 PM, mark.reinhold at oracle.com wrote:
>>> ...
>>>
>>> As I wrote in my reply to David, I'm open to lifting the traditional
>>> restrictions on the class-file representation of qualified names in the
>>> case of module names. Given the weight of tradition and the past value
>>> of the existing constraints, however, I'd like to have a more compelling
>>> reason than "some future hypothetical module system might need this
>>> flexibility".
>>
>> The key point is exactly that a present, concrete module system does
>> need this flexibility; ours, specifically. And this is such an easy change.
>
> No, it's not, as I've tried to explain nearby.
>
>>> In trying to think about the future I do wonder if, today, we should
>>> reserve a character or two just in case we discover five or ten years
>>> from now that we need to add more structure to module names. Should
>>> we set aside `:`, or perhaps some other character, just in case?
>>
>> We have had a module system with no character restrictions for many
>> years now and never looked back. It adds flexibility as every module
>> loader can establish acceptable name rules based on its policy, which
>> afforded us a great deal of interoperability with all manner of system.
>> You can still have reserved characters if that's what is appropriate to
>> your module resolver; if a name is invalid for that resolver, the
>> resolver can simply return no match.
>
> I'm puzzled. The "Module names" page [1] in the documentation of JBoss
> Modules says:
>
> A module name is a dot-separated string which is used along with a
> version slot identifier to uniquely identify a module within a module
> loader. Names are typically organized along the same lines as package
> names, however there is no specific relationship between a module name
> and the packages it contains. These are all examples of typical module
> names:
Yes, this documentation covers defining modules statically for the
consumption of the filesystem module loader. Internally though, there
is no restriction for modules which are generated on the basis of other
module systems or class loading schemes. This situation is highly
analogous to what I've requested in my other email.
> Unless I'm misunderstanding something, or the documentation is incorrect
> or incomplete, then module names in JBoss Modules cannot be arbitrary
> strings. They cannot contain colons, since colons are used to separate
> module names from version-slot identifiers, nor can they contain repeated
> dots or slashes, since dots are transformed into slashes and repeated
> slashes are, in most filesystems, equivalent to a single slash. (They
> probably also can't contain characters that are illegal in the underlying
> filesystem unless some sort of escaping is done, but the documentation
> does not address this.)
Right, see above and my other email on this topic to understand why this is.
> More to the point, every single module name in the examples in the JBoss
> Modules documentation would be considered a qualified identifier in Java.
> When and in what circumstances are truly arbitrary module names used in
> JBoss Modules? Or is this simply a theoretical concern on your part?
Yes, our static filesystem modules should be well covered already.
--
- DML
From david.lloyd at redhat.com Tue Dec 6 00:12:06 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Mon, 5 Dec 2016 18:12:06 -0600
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <20161205230556.5099720E1F@eggemoggin.niobe.net>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<20161130000802.D65C81C6B0@eggemoggin.niobe.net>
<19808667-7501-fc47-009d-b245e3a099ce@redhat.com>
<20161205230556.5099720E1F@eggemoggin.niobe.net>
Message-ID: <7a4e8608-3d91-e5ef-d9f7-5577c9bf7303@redhat.com>
On 12/05/2016 05:05 PM, mark.reinhold at oracle.com wrote:
> 2016/11/29 18:47:33 -0800, david.lloyd at redhat.com:
>> On 11/29/2016 06:08 PM, mark.reinhold at oracle.com wrote:
>>> ...
>>>
>>> I'm concerned mostly about the complexity of the language specification,
>>> which affects all users of this module system. If we change the language
>>> of module declarations to allow arbitrary module names, possibly via some
>>> sort of quoting scheme, then that's something that every developer would
>>> have to understand even though it would most likely be of benefit to few.
>>
>> You don't need a quoting scheme to allow arbitrary names - you just
>> allow them - and I don't see how this complicates the language
>> specification in any way: in fact it should simplify it. There's just
>> no reason to use "internal form".
>
> At present the JLS does not allow identifiers to be composed of arbitrary
> characters. If that's what you really want in the source language then
> you'd have to introduce some sort of quoting scheme to allow whitespace,
> all punctuation characters, and whatever else isn't already allowed by
> `Character::isJavaIdentifierPart`. That would add significant complexity
> to the JLS and, more importantly, would be something that every single
> Java developer would have to learn.
>
> Perhaps you don't really mean "arbitrary", however, just "something more
> than is allowed in a Java qualified identifier".
See below...
>>> I'm concerned also, though to a lesser degree, about the complexity of
>>> the class-file specification (i.e., the JVMS), the long-term evolvability
>>> of that specification, and the complexity of code that reads and writes
>>> class files, though the latter is second-order since it mostly affects
>>> maintainers of IDEs, compilers, and other kinds of tools rather than
>>> developers in general. If there's a compelling reason to lift the usual
>>> restrictions on the representation of qualified names in class files, at
>>> least in the case of module names, then I'd like to hear it.
>>
>> I'm just really confused at this line of justification. Making module
>> names be a qualified name is the very thing that I'm arguing against.
>> Don't make any change to the restrictions on qualified names, and don't
>> make module names be qualified names; that's all there is to it.
>
> Okay, but in the language if an identifier is not a qualified name then
> it cannot contain `.` characters, so using simple identifiers for module
> names is a non-starter.
>
> We could revise the JLS to introduce a brand-new kind of identifier,
> just for module names, that allows `.` and maybe a few other additional
> special characters, and encode these without using internal form in
> `module-info.class` files. This new kind of identifier would, however,
> be something that every Java developer must learn about.
OK I think I see the disconnect here. Here's what I'm reading from what
you're saying: The JLS says that names have to be one of: a qualified
names, an identifier, or some new invented thing that would have to be
added to the JLS, and if it's the last, that's a problem for new
developers and probably for a few other reasons as well.
I have no disagreement with this point at all. I'm only requesting that
the backing representation in the JVM/class file spec and/or
ModuleDescriptor.Builder side have no particular restrictions. Can
these two concepts be decoupled somehow? I recognize that this means
that javac-generated descriptors can never reference modules with
"invalid" (according to the JLS) names, but I don't see that as a
problem: this is solely for modules which are generated and referenced
by our container code from some other existing module scheme.
The reason I am opposed to using an obfuscation scheme in this case is
simple: it makes the diagnostic output and calls to get the name of your
current module have a confusing result, and this mechanism is one of
only two value propositions of Jigsaw that cannot currently be otherwise
achieved without special JDK support (the other being the new
security/encapsulation feature). The .-to-/ mapping of internal names
is particularly irksome.
> Who, in practical terms, would benefit from this added complexity?
> You've said that JBoss Modules allows arbitrary characters in module
> names, but do you or your users actually use such names in practice?
> If so, can you show us some examples?
The names can be mapped (exactly) from another module system or
structure, including names that have either different limitations or no
limitations in practice like Java EE names, old-style Extension-list
names, JAR file base names, Maven artifacts, etc.
We generally apply name restrictions on a per-loader basis, i.e. it's up
to the loader to understand names, and if you have (for example) a
filesystem-backed module loader and you look in it for a module with a
name that cannot be mapped to the filesystem, you're not going to find
anything as the invalid name will be ignored (but otherwise there's no
harm from trying it).
So in practice our filesystem names tend to be package-ish (though we
allow '-' in this case as well), and that loader does in fact use a
.-to-/ mapping as well as forbidding potential filesystem separators
(because they are specifically relevant to how the module is located on
the filesystem). But our container module names are considerably looser
and could contain any valid UTF-8 text, and in fact in some cases
consist of a literal filesystem path name (which in turn generally has
few restrictions).
>>> I don't expect Java EE modules to map directly to Java SE modules, nor
>>> do any of the Java EE spec leads with whom I've discussed this issue.
>>> EE modules and SE modules are completely different kinds of things.
>>
>> On March 11 of this year (and other occasions) I specifically asked
>> exactly that, and you said [1] "Of course we have that expectation --
>> that's why the requirements include an entire section on dynamic
>> configuration". Did I misunderstand you then or now, or has something
>> changed?
>
> I don't think anything has changed.
>
> I didn't mean for that statement to imply that I thought that every EE
> module would map directly to an SE module, though I see how it can be
> read that way. Apologies for the confusion.
OK, that's definitely a relief... but I do think we need to be very
clear about exactly what the expectations are in terms of EE 9 support
for modules, with the participation of that expert group, so that we
don't get blindsided if/when that spec is being prepared. Is there some
way we can (reasonably quickly) come to a public agreement with the Java
EE 9 expert group on this topic? I realize this is rapidly becoming a
tangent, so maybe this should be spun off into a separate discussion thread.
--
- DML
From david.lloyd at redhat.com Tue Dec 6 00:21:19 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Mon, 5 Dec 2016 18:21:19 -0600
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <20161205230856.5E5A220E2E@eggemoggin.niobe.net>
References: <20161031202238.D3E5DDB037@eggemoggin.niobe.net>
<0f23ba26-9e93-a62b-b537-be0d5a91b895@redhat.com>
<20161121135638.309938232@eggemoggin.niobe.net>
<0a1e41c6-9d36-ad5a-5ad0-7445e7ae3883@redhat.com>
<20161130001002.E23C91C6BC@eggemoggin.niobe.net>
<2c7a0dbb-3666-dd7a-3260-e8bc4b8c10f2@redhat.com>
<20161205230856.5E5A220E2E@eggemoggin.niobe.net>
Message-ID: <ce1856d9-0432-3eb1-c68d-282566a9cc73@redhat.com>
On 12/05/2016 05:08 PM, mark.reinhold at oracle.com wrote:
> 2016/11/29 18:51:22 -0800, david.lloyd at redhat.com:
>> On 11/29/2016 06:10 PM, mark.reinhold at oracle.com wrote:
>>> 2016/11/22 13:23:34 -0800, david.lloyd at redhat.com:
>>>> ...
>>>>
>>>> To degenerate the problem to the simplest case: I may have to layers, A
>>>> and B, which are unrelated and both contain a module X. I may need the
>>>> module X in one layer to have a dependency on the module X in another
>>>> layer. Unless there's new code I haven't seen, each dependency is
>>>> expressed by name only, therefore this is impossible. Whereas in our
>>>> system, a dependency is a tuple of (module loader, module name),
>>>> allowing arbitrary weaving of modules from different name spaces with no
>>>> problem.
>>>>
>>>> I don't think we need it to work the exact same way, but we need some
>>>> equivalent.
>>>
>>> If this is how you need resolution to work for your modules then can't
>>> you write your own resolver to make it so, using whatever additional
>>> (weak) data structures you need to associate names with layers, class
>>> loaders, etc., in whatever manner you require?
>>
>> No, because the input to the resolver is just a name; there's no other
>> context to say which layer the module comes from, without some kind of
>> name mangling scheme which raises more problems (we ran into exactly
>> this issue more than six years ago). In other words I can require a
>> module named "X", but I can't say that I require the module "X" from
>> this other layer/namespace; rather I have to flatten out all my
>> namespaces in a way that can be mutually resolvable, which means that I
>> must ultimately constrain user module names in some weird way. This is
>> not so good when modules refer to one another by name programmatically,
>> nor in diagnostic output.
>
> So, what do you suggest we do?
Is it possible to add overloads in ModuleDescriptor.Builder like this:
// Export to the named module in the given layer
ModuleDescriptor.Builder exports(String pn, String target, Layer
targetLayer);
// Add dependence on module with the given name in the given layer
ModuleDescriptor.Builder requires(String mn, Layer layer);
ModuleDescriptor.Builder requires(Set<Modifier> mods, String mn,
Layer layer);
I think that's all that's necessary.
--
- DML
From forax at univ-mlv.fr Tue Dec 6 08:08:58 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Tue, 6 Dec 2016 09:08:58 +0100 (CET)
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<2034282380.1641047.1479931198355.JavaMail.zimbra@u-pem.fr>
<20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
Message-ID: <947350194.519683.1481011738220.JavaMail.zimbra@u-pem.fr>
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "David M. Lloyd" <david.lloyd at redhat.com>, jpms-spec-experts at openjdk.java.net
> Envoy?: Mercredi 30 Novembre 2016 01:11:02
> Objet: Re: Proposal: #ModuleNameCharacters
> 2016/11/23 11:59:58 -0800, forax at univ-mlv.fr:
>> I agree with David, module names are just names so there should be
>> encoded as 'UTF8' kind of constant with no restriction.
>>
>> Java (the language) module name format has more restrictions because
>> javac has to compile module-info.java files, but in the bytecode there
>> is no need to encode module name as 'internal name'.
>>
>> When doing the JSR 292 spec, we (the EG) removed all the constraints
>> on class name, method name, etc that could be removed, in order to
>> ease the mapping of any languages on top of the VM.
>
> Most, if not all, such constraints were removed long before JSR 292.
> For those that remain, John Rose proposed a fairly straightforward
> name-mangling scheme [1].
>
>> If we want ease
>> the mapping between any existing and future module system to JPMS,
>> module name inside the class file format should be plain string
>> constant.
>
> As I wrote in my reply to David, I'm open to lifting the traditional
> restrictions on the class-file representation of qualified names in the
> case of module names.
Ok, cool.
> Given the weight of tradition and the past value
> of the existing constraints, however, I'd like to have a more compelling
> reason than "some future hypothetical module system might need this
> flexibility".
Existing constraints exist because a package name is a part of a qualified class name. There is no tradition for module names. Module names in the class file are not mixed with other constrained names, so i see no compelling reason to add arbitrary rules to try to restrict module names.
Note that, JLS module names have to be parsed by the compiler, so for JLS module names, having the same constraints as any other qualified identifiers make sense, but here, we're talking about module names in the JVM spec, not in the JLS.
Now, the constant pool is typed and structured, if we want to have constraints on module names, in my opinion, we should introduce a new constant pool item to make it clear that module names are not plain names but specific names exactly like there is a Class constant pool item.
And with my ASM hat, having to add replace('.', '/') and replace('/', '.') at the right places is error prone, if we can avoid that is a big win in term of usability.
>
> In trying to think about the future I do wonder if, today, we should
> reserve a character or two just in case we discover five or ten years
> from now that we need to add more structure to module names. Should
> we set aside `:`, or perhaps some other character, just in case?
if we want structure, we will add another constant pool item. It's what valhalla does for parameterized types.
>
> - Mark
>
>
> [1] https://blogs.oracle.com/jrose/entry/symbolic_freedom_in_the_vm
R?mi
From jason at takari.io Mon Dec 5 23:31:16 2016
From: jason at takari.io (Jason van Zyl)
Date: Mon, 5 Dec 2016 18:31:16 -0500
Subject: Resignation from group
Message-ID: <BC58612F-C75D-40D3-AFF2-31F08F8ADF43@takari.io>
Hi,
I?d like to resign from the group, but I suggest allowing Robert Scholte from the Maven project to replace me.
Thanks,
Jason van Zyl
From david.lloyd at redhat.com Tue Dec 6 23:08:27 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Tue, 6 Dec 2016 17:08:27 -0600
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <ce1856d9-0432-3eb1-c68d-282566a9cc73@redhat.com>
References: <20161031202238.D3E5DDB037@eggemoggin.niobe.net>
<0f23ba26-9e93-a62b-b537-be0d5a91b895@redhat.com>
<20161121135638.309938232@eggemoggin.niobe.net>
<0a1e41c6-9d36-ad5a-5ad0-7445e7ae3883@redhat.com>
<20161130001002.E23C91C6BC@eggemoggin.niobe.net>
<2c7a0dbb-3666-dd7a-3260-e8bc4b8c10f2@redhat.com>
<20161205230856.5E5A220E2E@eggemoggin.niobe.net>
<ce1856d9-0432-3eb1-c68d-282566a9cc73@redhat.com>
Message-ID: <72fbdcd9-a0c9-e84e-25fe-868399cded24@redhat.com>
On 12/05/2016 06:21 PM, David M. Lloyd wrote:
> On 12/05/2016 05:08 PM, mark.reinhold at oracle.com wrote:
>> 2016/11/29 18:51:22 -0800, david.lloyd at redhat.com:
>>> On 11/29/2016 06:10 PM, mark.reinhold at oracle.com wrote:
>>>> 2016/11/22 13:23:34 -0800, david.lloyd at redhat.com:
>>>>> ...
>>>>>
>>>>> To degenerate the problem to the simplest case: I may have to
>>>>> layers, A
>>>>> and B, which are unrelated and both contain a module X. I may need
>>>>> the
>>>>> module X in one layer to have a dependency on the module X in another
>>>>> layer. Unless there's new code I haven't seen, each dependency is
>>>>> expressed by name only, therefore this is impossible. Whereas in our
>>>>> system, a dependency is a tuple of (module loader, module name),
>>>>> allowing arbitrary weaving of modules from different name spaces
>>>>> with no
>>>>> problem.
>>>>>
>>>>> I don't think we need it to work the exact same way, but we need some
>>>>> equivalent.
>>>>
>>>> If this is how you need resolution to work for your modules then can't
>>>> you write your own resolver to make it so, using whatever additional
>>>> (weak) data structures you need to associate names with layers, class
>>>> loaders, etc., in whatever manner you require?
>>>
>>> No, because the input to the resolver is just a name; there's no other
>>> context to say which layer the module comes from, without some kind of
>>> name mangling scheme which raises more problems (we ran into exactly
>>> this issue more than six years ago). In other words I can require a
>>> module named "X", but I can't say that I require the module "X" from
>>> this other layer/namespace; rather I have to flatten out all my
>>> namespaces in a way that can be mutually resolvable, which means that I
>>> must ultimately constrain user module names in some weird way. This is
>>> not so good when modules refer to one another by name programmatically,
>>> nor in diagnostic output.
>>
>> So, what do you suggest we do?
>
> Is it possible to add overloads in ModuleDescriptor.Builder like this:
>
> // Export to the named module in the given layer
> ModuleDescriptor.Builder exports(String pn, String target, Layer
> targetLayer);
>
> // Add dependence on module with the given name in the given layer
> ModuleDescriptor.Builder requires(String mn, Layer layer);
> ModuleDescriptor.Builder requires(Set<Modifier> mods, String mn, Layer
> layer);
>
> I think that's all that's necessary.
I'm currently looking into defining modules with no dependency
information, and adding all the exports after the fact: that might be a
solution to this.
--
- DML
From david.lloyd at redhat.com Wed Dec 7 04:43:00 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Tue, 6 Dec 2016 22:43:00 -0600
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <72fbdcd9-a0c9-e84e-25fe-868399cded24@redhat.com>
References: <20161031202238.D3E5DDB037@eggemoggin.niobe.net>
<0f23ba26-9e93-a62b-b537-be0d5a91b895@redhat.com>
<20161121135638.309938232@eggemoggin.niobe.net>
<0a1e41c6-9d36-ad5a-5ad0-7445e7ae3883@redhat.com>
<20161130001002.E23C91C6BC@eggemoggin.niobe.net>
<2c7a0dbb-3666-dd7a-3260-e8bc4b8c10f2@redhat.com>
<20161205230856.5E5A220E2E@eggemoggin.niobe.net>
<ce1856d9-0432-3eb1-c68d-282566a9cc73@redhat.com>
<72fbdcd9-a0c9-e84e-25fe-868399cded24@redhat.com>
Message-ID: <f52f515c-0ea0-31f3-2e39-5deab4342434@redhat.com>
On 12/06/2016 05:08 PM, David M. Lloyd wrote:
> On 12/05/2016 06:21 PM, David M. Lloyd wrote:
>> On 12/05/2016 05:08 PM, mark.reinhold at oracle.com wrote:
>>> 2016/11/29 18:51:22 -0800, david.lloyd at redhat.com:
>>>> On 11/29/2016 06:10 PM, mark.reinhold at oracle.com wrote:
>>>>> 2016/11/22 13:23:34 -0800, david.lloyd at redhat.com:
>>>>>> ...
>>>>>>
>>>>>> To degenerate the problem to the simplest case: I may have to
>>>>>> layers, A
>>>>>> and B, which are unrelated and both contain a module X. I may need
>>>>>> the
>>>>>> module X in one layer to have a dependency on the module X in another
>>>>>> layer. Unless there's new code I haven't seen, each dependency is
>>>>>> expressed by name only, therefore this is impossible. Whereas in our
>>>>>> system, a dependency is a tuple of (module loader, module name),
>>>>>> allowing arbitrary weaving of modules from different name spaces
>>>>>> with no
>>>>>> problem.
>>>>>>
>>>>>> I don't think we need it to work the exact same way, but we need some
>>>>>> equivalent.
>>>>>
>>>>> If this is how you need resolution to work for your modules then can't
>>>>> you write your own resolver to make it so, using whatever additional
>>>>> (weak) data structures you need to associate names with layers, class
>>>>> loaders, etc., in whatever manner you require?
>>>>
>>>> No, because the input to the resolver is just a name; there's no other
>>>> context to say which layer the module comes from, without some kind of
>>>> name mangling scheme which raises more problems (we ran into exactly
>>>> this issue more than six years ago). In other words I can require a
>>>> module named "X", but I can't say that I require the module "X" from
>>>> this other layer/namespace; rather I have to flatten out all my
>>>> namespaces in a way that can be mutually resolvable, which means that I
>>>> must ultimately constrain user module names in some weird way. This is
>>>> not so good when modules refer to one another by name programmatically,
>>>> nor in diagnostic output.
>>>
>>> So, what do you suggest we do?
>>
>> Is it possible to add overloads in ModuleDescriptor.Builder like this:
>>
>> // Export to the named module in the given layer
>> ModuleDescriptor.Builder exports(String pn, String target, Layer
>> targetLayer);
>>
>> // Add dependence on module with the given name in the given layer
>> ModuleDescriptor.Builder requires(String mn, Layer layer);
>> ModuleDescriptor.Builder requires(Set<Modifier> mods, String mn, Layer
>> layer);
>>
>> I think that's all that's necessary.
>
> I'm currently looking into defining modules with no dependency
> information, and adding all the exports after the fact: that might be a
> solution to this.
OK so my initial experiments indicate that I'd need at least a few more
methods added to Layer.Controller to make this work. I did need a
small-ish patch to let me get some tests working, which can be viewed at
https://gist.github.com/dmlloyd/e3c114f0ac7595a93ecb63b1dbed5e00 and
taken from there if these changes make sense (or I can bring this up on
jigsaw-dev if that's where this discussion should take place).
I could probably get away without add*ToAllUnnamed (which are just there
to allow for a few more interop options) if there's a security or other
risk involved in that construct (I couldn't think of anything in
particular though).
I'm still trying to devise tests that would show if this is actually
working in nontrivial cases, but the fact that I've gotten it to run
through at least the existing basic JBoss Modules unit tests is a good sign.
I think I'm also running across the same issue that Tom Watson raised
last Tuesday about not knowing in advance what services are being used
by a module. I seem to recall that we discussed allowing the Layer to
mediate service load requests made by modules within that layer; is that
something I've recalled correctly, and if so, is that option still on
the table?
--
- DML
From tjwatson at us.ibm.com Wed Dec 7 13:57:08 2016
From: tjwatson at us.ibm.com (Thomas Watson)
Date: Wed, 7 Dec 2016 07:57:08 -0600
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <f52f515c-0ea0-31f3-2e39-5deab4342434@redhat.com>
References: <20161031202238.D3E5DDB037@eggemoggin.niobe.net><0f23ba26-9e93-a62b-b537-be0d5a91b895@redhat.com><20161121135638.309938232@eggemoggin.niobe.net><0a1e41c6-9d36-ad5a-5ad0-7445e7ae3883@redhat.com><20161130001002.E23C91C6BC@eggemoggin.niobe.net><2c7a0dbb-3666-dd7a-3260-e8bc4b8c10f2@redhat.com><20161205230856.5E5A220E2E@eggemoggin.niobe.net><ce1856d9-0432-3eb1-c68d-282566a9cc73@redhat.com><72fbdcd9-a0c9-e84e-25fe-868399cded24@redhat.com>
<f52f515c-0ea0-31f3-2e39-5deab4342434@redhat.com>
Message-ID: <OF378BFC21.13E6E022-ON00258082.004A5D54-86258082.004CA7AB@notes.na.collabserv.com>
>
> I think I'm also running across the same issue that Tom Watson raised
> last Tuesday about not knowing in advance what services are being used
> by a module. I seem to recall that we discussed allowing the Layer to
> mediate service load requests made by modules within that layer; is that
> something I've recalled correctly, and if so, is that option still on
> the table?
>
> --
> - DML
>
The way I got around this for services provided by boot layer was to
discover ALL the services provided by the boot layer which come from
unqualified exported packages. I then define ModuleDescriptors for each
OSGi bundle that uses each and every one of the 'public' services provided
by the boot layer. This allowed my to effectively state that a module for
an OSGi bundle uses * services from the boot layer. But this also
required me to make each OSGi bundle's ModuleDescription require any
module from boot which exports a packages that contains the contract API
being loaded by the ServiceLoader. Otherwise a failure will occur
creating the configuration because JPMS thinks the module using the
service does not have access to the service contract.
I would be interested to hear what ideas there are around allowing a Layer
Controller to influence this behavior such that we can detect
ServiceLoader usage by modules in our layers and addUses at runtime as
appropriate for our module systems. Perhaps a more simple option is to
simply allow any attempt at ServiceLoader.load, without checking for
service uses, from modules that are contained in a Layer that was resolved
with with the 'resolveRequires' vs. the 'resolveRequiresAndUses' method.
Tom
From mark.reinhold at oracle.com Wed Dec 7 23:38:16 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 7 Dec 2016 15:38:16 -0800 (PST)
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <7a4e8608-3d91-e5ef-d9f7-5577c9bf7303@redhat.com>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<20161130000802.D65C81C6B0@eggemoggin.niobe.net>
<19808667-7501-fc47-009d-b245e3a099ce@redhat.com>
<20161205230556.5099720E1F@eggemoggin.niobe.net>
<7a4e8608-3d91-e5ef-d9f7-5577c9bf7303@redhat.com>
Message-ID: <20161207233816.EDE19213E4@eggemoggin.niobe.net>
2016/12/5 16:12:06 -0800, david.lloyd at redhat.com:
> On 12/05/2016 05:05 PM, mark.reinhold at oracle.com wrote:
>> 2016/11/29 18:47:33 -0800, david.lloyd at redhat.com:
>>> ...
>>>
>>> I'm just really confused at this line of justification. Making module
>>> names be a qualified name is the very thing that I'm arguing against.
>>> Don't make any change to the restrictions on qualified names, and don't
>>> make module names be qualified names; that's all there is to it.
>>
>> Okay, but in the language if an identifier is not a qualified name then
>> it cannot contain `.` characters, so using simple identifiers for module
>> names is a non-starter.
>>
>> We could revise the JLS to introduce a brand-new kind of identifier,
>> just for module names, that allows `.` and maybe a few other additional
>> special characters, and encode these without using internal form in
>> `module-info.class` files. This new kind of identifier would, however,
>> be something that every Java developer must learn about.
>
> OK I think I see the disconnect here. Here's what I'm reading from what
> you're saying: The JLS says that names have to be one of: a qualified
> names, an identifier, or some new invented thing that would have to be
> added to the JLS, and if it's the last, that's a problem for new
> developers and probably for a few other reasons as well.
Right. Just about the last thing we want is additional complexity in the
source language.
> I have no disagreement with this point at all.
Okay, good.
> I'm only requesting that
> the backing representation in the JVM/class file spec and/or
> ModuleDescriptor.Builder side have no particular restrictions. Can
> these two concepts be decoupled somehow? I recognize that this means
> that javac-generated descriptors can never reference modules with
> "invalid" (according to the JLS) names, but I don't see that as a
> problem: this is solely for modules which are generated and referenced
> by our container code from some other existing module scheme.
If you can live with that then yes, I think we can make something work.
> The reason I am opposed to using an obfuscation scheme in this case is
> simple: it makes the diagnostic output and calls to get the name of your
> current module have a confusing result, and this mechanism is one of
> only two value propositions of Jigsaw that cannot currently be otherwise
> achieved without special JDK support (the other being the new
> security/encapsulation feature). The .-to-/ mapping of internal names
> is particularly irksome.
Understood. I do think, though, that we should reserve a few characters
for future use. I'll make a specific suggestion in my reply to R??mi in a
moment, since it's also related to an issue he raised about the structure
of class files.
- Mark
From mark.reinhold at oracle.com Wed Dec 7 23:39:16 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 7 Dec 2016 15:39:16 -0800 (PST)
Subject: Java SE modules in Java EE
Message-ID: <20161207233917.07E31213E7@eggemoggin.niobe.net>
2016/12/5 16:12:06 -0800, david.lloyd at redhat.com:
> On 12/05/2016 05:05 PM, mark.reinhold at oracle.com wrote:
>> 2016/11/29 18:47:33 -0800, david.lloyd at redhat.com:
>>> On 11/29/2016 06:08 PM, mark.reinhold at oracle.com wrote:
>>>> ...
>>>>
>>>> I don't expect Java EE modules to map directly to Java SE modules, nor
>>>> do any of the Java EE spec leads with whom I've discussed this issue.
>>>> EE modules and SE modules are completely different kinds of things.
>>>
>>> On March 11 of this year (and other occasions) I specifically asked
>>> exactly that, and you said [1] "Of course we have that expectation --
>>> that's why the requirements include an entire section on dynamic
>>> configuration". Did I misunderstand you then or now, or has something
>>> changed?
>>
>> I don't think anything has changed.
>>
>> I didn't mean for that statement to imply that I thought that every EE
>> module would map directly to an SE module, though I see how it can be
>> read that way. Apologies for the confusion.
>
> OK, that's definitely a relief... but I do think we need to be very
> clear about exactly what the expectations are in terms of EE 9 support
> for modules, with the participation of that expert group, so that we
> don't get blindsided if/when that spec is being prepared. Is there some
> way we can (reasonably quickly) come to a public agreement with the Java
> EE 9 expert group on this topic? I realize this is rapidly becoming a
> tangent, so maybe this should be spun off into a separate discussion thread.
This is indeed a tangent, so I've started a new thread.
In an ideal universe we'd all be working on Java SE 9 and Java EE 9 at
the same time, and all these specifications would be perfectly in sync
and in mutual support of each other.
That is not, sadly, the universe in which we live. The Java EE 9 EG
doesn't even exist yet, and so far as I know the EE 8 EG (rightly) has
no intent to adopt Java SE 9 features in that release, so there isn't
really an opportunity for a formal working relationship between two EGs.
I have reviewed the JPMS design with Bill Shannon and other Java EE spec
leads on a regular basis over the past few years, and taken account of
their feedback as appropriate, but realistically I think it will be a
while before they collectively have the bandwidth to think deeply about
how, and even the degree to which, EE 9 will incorporate SE 9 modules.
I'll touch base with Bill again and see if perhaps he and his fellow spec
leads would like to interact with this EG more formally.
Any EE spec lead (or anyone else, for that matter!) is, of course, always
welcome to send feedback directly to this EG via the jpms-spec-comments
list.
- Mark
From mark.reinhold at oracle.com Wed Dec 7 23:40:17 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 7 Dec 2016 15:40:17 -0800 (PST)
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <947350194.519683.1481011738220.JavaMail.zimbra@u-pem.fr>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<2034282380.1641047.1479931198355.JavaMail.zimbra@u-pem.fr>
<20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
<947350194.519683.1481011738220.JavaMail.zimbra@u-pem.fr>
Message-ID: <20161207234017.1761F213EC@eggemoggin.niobe.net>
2016/12/6 0:08:58 -0800, forax at univ-mlv.fr:
> 2016/11/29 16:11:02 -0800, mark.reinhold at oracle.com:
>> ...
>>
>> As I wrote in my reply to David, I'm open to lifting the traditional
>> restrictions on the class-file representation of qualified names in the
>> case of module names.
>
> Ok, cool.
>
>> Given the weight of tradition and the past value
>> of the existing constraints, however, I'd like to have a more compelling
>> reason than "some future hypothetical module system might need this
>> flexibility".
>
> Existing constraints exist because a package name is a part of a
> qualified class name. There is no tradition for module names. Module
> names in the class file are not mixed with other constrained names, so
> i see no compelling reason to add arbitrary rules to try to restrict
> module names.
Okay, okay ... taken together with David's examples, I get the point.
(Personally I've always considered the whole `.`-to-`/` mapping kind
of archaic anyway.)
> Note that, JLS module names have to be parsed by the compiler, so for
> JLS module names, having the same constraints as any other qualified
> identifiers make sense, but here, we're talking about module names in
> the JVM spec, not in the JLS.
Correct.
> Now, the constant pool is typed and structured, if we want to have
> constraints on module names, in my opinion, we should introduce a new
> constant pool item to make it clear that module names are not plain
> names but specific names exactly like there is a Class constant pool
> item.
Agreed. This is, in fact, an inconsistency in the present proposal,
since it imposes constraints on otherwise untagged CONSTANT_Utf8
structures. If we're going to impose constraints on free-standing
module and package names then we should introduce the obvious new
`CONSTANT_Module_info` and `CONSTANT_Package_info` structures.
> And with my ASM hat, having to add replace('.', '/') and replace('/',
> '.') at the right places is error prone, if we can avoid that is a big
> win in term of usability.
Yep.
>> In trying to think about the future I do wonder if, today, we should
>> reserve a character or two just in case we discover five or ten years
>> from now that we need to add more structure to module names. Should
>> we set aside `:`, or perhaps some other character, just in case?
>
> if we want structure, we will add another constant pool item. It's
> what valhalla does for parameterized types.
So the question is, then: Which, if any, characters should we reserve?
Peering into the myriad alternate visions swirling around in my cloudy
crystal ball, I can see:
- A structured namespace of modules. `:` is a logical separator here,
even in the source language if need be, so let's reserve it now in
class files.
- Module names encoded in class files together with specific version
strings, to form compound module identifiers. We already use `@` to
separate module names from version strings in the module-system API
(e.g., the result of `ModuleDescriptor::toString`), so let's reserve
that in class files now.
(This is just my imagination, not specific suggestions for the future!)
Additionally we should reserve the universal escape character (`\`) and
for sanity also forbid any character whose code point is less than 0x20
(` `). (Ideally we'd forbid all Unicode non-printing characters, but
it's best not to have the JVMS depend upon the Unicode specification.)
To sum up: Reserve `:`, `@`, and `\` for future use, and forbid the ASCII
non-printing characters (< 0x20).
David -- Are these restrictions acceptable in your use cases, or if not
then at least tolerable? I'm pretty sure I've never seen any of these
characters in Java EE module names, JAR file base names, Maven group or
artifact names, or the other examples you mentioned.
- Mark
From mark.reinhold at oracle.com Wed Dec 7 23:41:17 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 7 Dec 2016 15:41:17 -0800 (PST)
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <f52f515c-0ea0-31f3-2e39-5deab4342434@redhat.com>
Message-ID: <20161207234117.1E5F1213F1@eggemoggin.niobe.net>
2016/12/6 20:43:00 -0800, david.lloyd at redhat.com:
> ...
>
> OK so my initial experiments indicate that I'd need at least a few more
> methods added to Layer.Controller to make this work. I did need a
> small-ish patch to let me get some tests working, which can be viewed at
> https://gist.github.com/dmlloyd/e3c114f0ac7595a93ecb63b1dbed5e00 and
> taken from there if these changes make sense (or I can bring this up on
> jigsaw-dev if that's where this discussion should take place).
Could you please describe here, in plain text, what further methods you
think you need? For IP clarity any material you want this EG to consider
must be posted to this list or to related OpenJDK infrastructure.
It'd be fine to discuss specific code patches on jigsaw-dev, but again
you'd need to include it directly rather than point to GitHub. This is
a design issue, though, so it's probably best to discuss it here first.
- Mark
From forax at univ-mlv.fr Thu Dec 8 10:26:12 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Thu, 8 Dec 2016 11:26:12 +0100 (CET)
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <20161207234017.1761F213EC@eggemoggin.niobe.net>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<2034282380.1641047.1479931198355.JavaMail.zimbra@u-pem.fr>
<20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
<947350194.519683.1481011738220.JavaMail.zimbra@u-pem.fr>
<20161207234017.1761F213EC@eggemoggin.niobe.net>
Message-ID: <564127947.1475086.1481192772861.JavaMail.zimbra@u-pem.fr>
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: forax at univ-mlv.fr
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Jeudi 8 D?cembre 2016 00:40:17
> Objet: Re: Proposal: #ModuleNameCharacters
> 2016/12/6 0:08:58 -0800, forax at univ-mlv.fr:
>> 2016/11/29 16:11:02 -0800, mark.reinhold at oracle.com:
>>> ...
>>>
>>> As I wrote in my reply to David, I'm open to lifting the traditional
>>> restrictions on the class-file representation of qualified names in the
>>> case of module names.
>>
>> Ok, cool.
>>
>>> Given the weight of tradition and the past value
>>> of the existing constraints, however, I'd like to have a more compelling
>>> reason than "some future hypothetical module system might need this
>>> flexibility".
>>
>> Existing constraints exist because a package name is a part of a
>> qualified class name. There is no tradition for module names. Module
>> names in the class file are not mixed with other constrained names, so
>> i see no compelling reason to add arbitrary rules to try to restrict
>> module names.
>
> Okay, okay ... taken together with David's examples, I get the point.
>
> (Personally I've always considered the whole `.`-to-`/` mapping kind
> of archaic anyway.)
>
>> Note that, JLS module names have to be parsed by the compiler, so for
>> JLS module names, having the same constraints as any other qualified
>> identifiers make sense, but here, we're talking about module names in
>> the JVM spec, not in the JLS.
>
> Correct.
>
>> Now, the constant pool is typed and structured, if we want to have
>> constraints on module names, in my opinion, we should introduce a new
>> constant pool item to make it clear that module names are not plain
>> names but specific names exactly like there is a Class constant pool
>> item.
>
> Agreed. This is, in fact, an inconsistency in the present proposal,
> since it imposes constraints on otherwise untagged CONSTANT_Utf8
> structures. If we're going to impose constraints on free-standing
> module and package names then we should introduce the obvious new
> `CONSTANT_Module_info` and `CONSTANT_Package_info` structures.
>
>> And with my ASM hat, having to add replace('.', '/') and replace('/',
>> '.') at the right places is error prone, if we can avoid that is a big
>> win in term of usability.
>
> Yep.
>
>>> In trying to think about the future I do wonder if, today, we should
>>> reserve a character or two just in case we discover five or ten years
>>> from now that we need to add more structure to module names. Should
>>> we set aside `:`, or perhaps some other character, just in case?
>>
>> if we want structure, we will add another constant pool item. It's
>> what valhalla does for parameterized types.
>
> So the question is, then: Which, if any, characters should we reserve?
>
> Peering into the myriad alternate visions swirling around in my cloudy
> crystal ball, I can see:
>
> - A structured namespace of modules. `:` is a logical separator here,
> even in the source language if need be, so let's reserve it now in
> class files.
>
> - Module names encoded in class files together with specific version
> strings, to form compound module identifiers. We already use `@` to
> separate module names from version strings in the module-system API
> (e.g., the result of `ModuleDescriptor::toString`), so let's reserve
> that in class files now.
>
> (This is just my imagination, not specific suggestions for the future!)
>
> Additionally we should reserve the universal escape character (`\`) and
> for sanity also forbid any character whose code point is less than 0x20
> (` `). (Ideally we'd forbid all Unicode non-printing characters, but
> it's best not to have the JVMS depend upon the Unicode specification.)
>
> To sum up: Reserve `:`, `@`, and `\` for future use, and forbid the ASCII
> non-printing characters (< 0x20).
You also need to reserve '/' because the java launcher (-m) use '/' to separate between the module name and the main class.
R?mi
>
> David -- Are these restrictions acceptable in your use cases, or if not
> then at least tolerable? I'm pretty sure I've never seen any of these
> characters in Java EE module names, JAR file base names, Maven group or
> artifact names, or the other examples you mentioned.
>
> - Mark
From david.lloyd at redhat.com Thu Dec 8 13:46:24 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 8 Dec 2016 07:46:24 -0600
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <20161207234117.1E5F1213F1@eggemoggin.niobe.net>
References: <20161207234117.1E5F1213F1@eggemoggin.niobe.net>
Message-ID: <1de06198-930b-4b71-fe5d-1477d9cace80@redhat.com>
On 12/07/2016 05:41 PM, mark.reinhold at oracle.com wrote:
> 2016/12/6 20:43:00 -0800, david.lloyd at redhat.com:
>> ...
>>
>> OK so my initial experiments indicate that I'd need at least a few more
>> methods added to Layer.Controller to make this work. I did need a
>> small-ish patch to let me get some tests working, which can be viewed at
>> https://gist.github.com/dmlloyd/e3c114f0ac7595a93ecb63b1dbed5e00 and
>> taken from there if these changes make sense (or I can bring this up on
>> jigsaw-dev if that's where this discussion should take place).
>
> Could you please describe here, in plain text, what further methods you
> think you need? For IP clarity any material you want this EG to consider
> must be posted to this list or to related OpenJDK infrastructure.
>
> It'd be fine to discuss specific code patches on jigsaw-dev, but again
> you'd need to include it directly rather than point to GitHub. This is
> a design issue, though, so it's probably best to discuss it here first.
Sure. I've added the following methods to Layer.Controller:
public Controller addPackage(Module source, String pn) { ... }
public Controller addOpens(Module source, String pn, Module target) { ... }
public Controller addOpensToAll(Module source, String pn) { ... }
public Controller addOpensToAllUnnamed(Module source, String pn) { ... }
public Controller addExports(Module source, String pn, Module target) {
... }
public Controller addExportsToAll(Module source, String pn) { ... }
public Controller addExportsToAllUnnamed(Module source, String pn) { ... }
public Controller addUses(Module source, Class<?> service) { ... }
--
- DML
From forax at univ-mlv.fr Thu Dec 8 14:22:27 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Thu, 8 Dec 2016 15:22:27 +0100 (CET)
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <1de06198-930b-4b71-fe5d-1477d9cace80@redhat.com>
References: <20161207234117.1E5F1213F1@eggemoggin.niobe.net>
<1de06198-930b-4b71-fe5d-1477d9cace80@redhat.com>
Message-ID: <103086633.1596649.1481206947913.JavaMail.zimbra@u-pem.fr>
I fully agree with David here,
All these methods should be added.
Currently, if you want to implement in Java the equivalent of the Java launcher you need to add the module of your launcher in the exported list of the module that contains the main method.
You can do that by changing the export directive by writing a ModuleFinder that will acts as proxy and add the directive you want because if you are the creator of the Layer, you are controlling the ModuleDescriptor of all modules in the Layer too.
So adding these methods is just a convenience for module runtime developers, it doesn't add new capability compared to what can be done now
(that's not fully true because you can not represent the unnamed module in a module-info, BTW).
R?mi
----- Mail original -----
> De: "David M. Lloyd" <david.lloyd at redhat.com>
> ?: "mark reinhold" <mark.reinhold at oracle.com>
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Jeudi 8 D?cembre 2016 14:46:24
> Objet: Re: Proposal: #NonHierarchicalLayers
> On 12/07/2016 05:41 PM, mark.reinhold at oracle.com wrote:
>> 2016/12/6 20:43:00 -0800, david.lloyd at redhat.com:
>>> ...
>>>
>>> OK so my initial experiments indicate that I'd need at least a few more
>>> methods added to Layer.Controller to make this work. I did need a
>>> small-ish patch to let me get some tests working, which can be viewed at
>>> https://gist.github.com/dmlloyd/e3c114f0ac7595a93ecb63b1dbed5e00 and
>>> taken from there if these changes make sense (or I can bring this up on
>>> jigsaw-dev if that's where this discussion should take place).
>>
>> Could you please describe here, in plain text, what further methods you
>> think you need? For IP clarity any material you want this EG to consider
>> must be posted to this list or to related OpenJDK infrastructure.
>>
>> It'd be fine to discuss specific code patches on jigsaw-dev, but again
>> you'd need to include it directly rather than point to GitHub. This is
>> a design issue, though, so it's probably best to discuss it here first.
>
> Sure. I've added the following methods to Layer.Controller:
>
> public Controller addPackage(Module source, String pn) { ... }
> public Controller addOpens(Module source, String pn, Module target) { ... }
> public Controller addOpensToAll(Module source, String pn) { ... }
> public Controller addOpensToAllUnnamed(Module source, String pn) { ... }
> public Controller addExports(Module source, String pn, Module target) {
> ... }
> public Controller addExportsToAll(Module source, String pn) { ... }
> public Controller addExportsToAllUnnamed(Module source, String pn) { ... }
> public Controller addUses(Module source, Class<?> service) { ... }
>
> --
> - DML
From mark.reinhold at oracle.com Thu Dec 8 16:14:06 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu, 08 Dec 2016 08:14:06 -0800
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <564127947.1475086.1481192772861.JavaMail.zimbra@u-pem.fr>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<2034282380.1641047.1479931198355.JavaMail.zimbra@u-pem.fr>
<20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
<947350194.519683.1481011738220.JavaMail.zimbra@u-pem.fr>
<20161207234017.1761F213EC@eggemoggin.niobe.net>
<564127947.1475086.1481192772861.JavaMail.zimbra@u-pem.fr>
Message-ID: <20161208081406.291112732@eggemoggin.niobe.net>
2016/12/8 2:26:12 -0800, forax at univ-mlv.fr:
> 2016/12/7 15:40:17 -0800, mark.reinhold at oracle.com:
>> ...
>>
>> To sum up: Reserve `:`, `@`, and `\` for future use, and forbid the ASCII
>> non-printing characters (< 0x20).
>
> You also need to reserve '/' because the java launcher (-m) use '/' to
> separate between the module name and the main class.
No, that's just a launcher detail. If this becomes a real problem
then we can work out a `/`-quoting mechanism for the launcher.
- Mark
From david.lloyd at redhat.com Thu Dec 8 16:26:40 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 8 Dec 2016 10:26:40 -0600
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <564127947.1475086.1481192772861.JavaMail.zimbra@u-pem.fr>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<2034282380.1641047.1479931198355.JavaMail.zimbra@u-pem.fr>
<20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
<947350194.519683.1481011738220.JavaMail.zimbra@u-pem.fr>
<20161207234017.1761F213EC@eggemoggin.niobe.net>
<564127947.1475086.1481192772861.JavaMail.zimbra@u-pem.fr>
Message-ID: <0cff8b9f-5c78-61cc-8468-40969748a103@redhat.com>
On 12/08/2016 04:26 AM, forax at univ-mlv.fr wrote:
> ----- Mail original -----
>> De: "mark reinhold" <mark.reinhold at oracle.com>
>> ?: forax at univ-mlv.fr
>> Cc: jpms-spec-experts at openjdk.java.net
>> Envoy?: Jeudi 8 D?cembre 2016 00:40:17
>> Objet: Re: Proposal: #ModuleNameCharacters
>
>> 2016/12/6 0:08:58 -0800, forax at univ-mlv.fr:
>>> 2016/11/29 16:11:02 -0800, mark.reinhold at oracle.com:
>>>> ...
>>>>
>>>> As I wrote in my reply to David, I'm open to lifting the traditional
>>>> restrictions on the class-file representation of qualified names in the
>>>> case of module names.
>>>
>>> Ok, cool.
>>>
>>>> Given the weight of tradition and the past value
>>>> of the existing constraints, however, I'd like to have a more compelling
>>>> reason than "some future hypothetical module system might need this
>>>> flexibility".
>>>
>>> Existing constraints exist because a package name is a part of a
>>> qualified class name. There is no tradition for module names. Module
>>> names in the class file are not mixed with other constrained names, so
>>> i see no compelling reason to add arbitrary rules to try to restrict
>>> module names.
>>
>> Okay, okay ... taken together with David's examples, I get the point.
>>
>> (Personally I've always considered the whole `.`-to-`/` mapping kind
>> of archaic anyway.)
>>
>>> Note that, JLS module names have to be parsed by the compiler, so for
>>> JLS module names, having the same constraints as any other qualified
>>> identifiers make sense, but here, we're talking about module names in
>>> the JVM spec, not in the JLS.
>>
>> Correct.
>>
>>> Now, the constant pool is typed and structured, if we want to have
>>> constraints on module names, in my opinion, we should introduce a new
>>> constant pool item to make it clear that module names are not plain
>>> names but specific names exactly like there is a Class constant pool
>>> item.
>>
>> Agreed. This is, in fact, an inconsistency in the present proposal,
>> since it imposes constraints on otherwise untagged CONSTANT_Utf8
>> structures. If we're going to impose constraints on free-standing
>> module and package names then we should introduce the obvious new
>> `CONSTANT_Module_info` and `CONSTANT_Package_info` structures.
>>
>>> And with my ASM hat, having to add replace('.', '/') and replace('/',
>>> '.') at the right places is error prone, if we can avoid that is a big
>>> win in term of usability.
>>
>> Yep.
>>
>>>> In trying to think about the future I do wonder if, today, we should
>>>> reserve a character or two just in case we discover five or ten years
>>>> from now that we need to add more structure to module names. Should
>>>> we set aside `:`, or perhaps some other character, just in case?
>>>
>>> if we want structure, we will add another constant pool item. It's
>>> what valhalla does for parameterized types.
>>
>> So the question is, then: Which, if any, characters should we reserve?
>>
>> Peering into the myriad alternate visions swirling around in my cloudy
>> crystal ball, I can see:
>>
>> - A structured namespace of modules. `:` is a logical separator here,
>> even in the source language if need be, so let's reserve it now in
>> class files.
>>
>> - Module names encoded in class files together with specific version
>> strings, to form compound module identifiers. We already use `@` to
>> separate module names from version strings in the module-system API
>> (e.g., the result of `ModuleDescriptor::toString`), so let's reserve
>> that in class files now.
>>
>> (This is just my imagination, not specific suggestions for the future!)
>>
>> Additionally we should reserve the universal escape character (`\`) and
>> for sanity also forbid any character whose code point is less than 0x20
>> (` `). (Ideally we'd forbid all Unicode non-printing characters, but
>> it's best not to have the JVMS depend upon the Unicode specification.)
>>
>> To sum up: Reserve `:`, `@`, and `\` for future use, and forbid the ASCII
>> non-printing characters (< 0x20).
>
> You also need to reserve '/' because the java launcher (-m) use '/' to separate between the module name and the main class.
>
> R?mi
>
>>
>> David -- Are these restrictions acceptable in your use cases, or if not
>> then at least tolerable? I'm pretty sure I've never seen any of these
>> characters in Java EE module names, JAR file base names, Maven group or
>> artifact names, or the other examples you mentioned.
Breaking it down one by one...
':' is going to break two things that I know of: modules generated from
Maven coordinates which have the syntax
<groupId>:<artifactId>:<classifier>, and modules in the JBoss Modules
static loader which have a slot component, using the syntax
"module-name:slot".
'@' might be OK to reserve; I can't think of any specific conflicts,
though we have allowed this character in the past.
'\' is a problem because in JBoss Modules uses that character to escape
':' (particularly in the Maven coordinates case) to avoid mixing up the
slot name with the module name. For a module named `foo\:bar:5`, the
static module loader would treat the name component as "foo:bar" and the
slot as "5", and locate the module accordingly, however the core system
does not treat '\' specially: the proper name of this module would be
`foo\:bar:5` according to the system, and that's the string you would
have to use to load the module by name.
'/' also may be a problem because within our container, we use file
names from the file system as the name of modules that come from the
file system. This also causes a problem for '\' on Windows. We could
possibly work out some kind of alternative in this case, with some
creative thinking.
Definitely in favor of forbidding non-printing code points 0x00 through
0x1F, and probably also 0x80 through 0x9F (we probably don't want to go
any further down the Unicode rabbit hole than that though - at least,
not in the JVM - if we want to get out of here this side of 2020).
--
- DML
From mark.reinhold at oracle.com Fri Dec 9 02:00:08 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu, 08 Dec 2016 18:00:08 -0800
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <0cff8b9f-5c78-61cc-8468-40969748a103@redhat.com>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<2034282380.1641047.1479931198355.JavaMail.zimbra@u-pem.fr>
<20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
<947350194.519683.1481011738220.JavaMail.zimbra@u-pem.fr>
<20161207234017.1761F213EC@eggemoggin.niobe.net>
<564127947.1475086.1481192772861.JavaMail.zimbra@u-pem.fr>
<0cff8b9f-5c78-61cc-8468-40969748a103@redhat.com>
Message-ID: <20161208180008.393555671@eggemoggin.niobe.net>
2016/12/8 8:26:40 -0800, david.lloyd at redhat.com:
> 2016/12/7 15:40:17 -0800, mark.reinhold at oracle.com:
>> David -- Are these restrictions acceptable in your use cases, or if not
>> then at least tolerable? I'm pretty sure I've never seen any of these
>> characters in Java EE module names, JAR file base names, Maven group or
>> artifact names, or the other examples you mentioned.
>
> Breaking it down one by one...
>
> ':' is going to break two things that I know of: modules generated from
> Maven coordinates which have the syntax
> <groupId>:<artifactId>:<classifier>, and modules in the JBoss Modules
> static loader which have a slot component, using the syntax
> "module-name:slot".
>
> '@' might be OK to reserve; I can't think of any specific conflicts,
> though we have allowed this character in the past.
>
> '\' is a problem because in JBoss Modules uses that character to escape
> ':' (particularly in the Maven coordinates case) to avoid mixing up the
> slot name with the module name. For a module named `foo\:bar:5`, the
> static module loader would treat the name component as "foo:bar" and the
> slot as "5", and locate the module accordingly, however the core system
> does not treat '\' specially: the proper name of this module would be
> `foo\:bar:5` according to the system, and that's the string you would
> have to use to load the module by name.
Okay, so the obvious thing to do is define `\` as a quotation character
for itself, for `:`, and for `@`. Then you can simply quote/dequote
wherever you interface with JPMS.
> '/' also may be a problem because within our container, we use file
> names from the file system as the name of modules that come from the
> file system. This also causes a problem for '\' on Windows. We could
> possibly work out some kind of alternative in this case, with some
> creative thinking.
I don't think we need to reserve `/`, as I said to R?mi.
As for `\`, I'm pretty sure that every version of Windows shipped this
century will accept `/` in pathnames as an alternative to `\`.
> Definitely in favor of forbidding non-printing code points 0x00 through
> 0x1F, and probably also 0x80 through 0x9F (we probably don't want to go
> any further down the Unicode rabbit hole than that though - at least,
> not in the JVM - if we want to get out of here this side of 2020).
Validating any code points past 0x7f is problematic at the JVM level
since it's UTF-8, so you'd have to decode it.
- Mark
From david.lloyd at redhat.com Fri Dec 9 02:12:10 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 8 Dec 2016 20:12:10 -0600
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <20161208180008.393555671@eggemoggin.niobe.net>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<2034282380.1641047.1479931198355.JavaMail.zimbra@u-pem.fr>
<20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
<947350194.519683.1481011738220.JavaMail.zimbra@u-pem.fr>
<20161207234017.1761F213EC@eggemoggin.niobe.net>
<564127947.1475086.1481192772861.JavaMail.zimbra@u-pem.fr>
<0cff8b9f-5c78-61cc-8468-40969748a103@redhat.com>
<20161208180008.393555671@eggemoggin.niobe.net>
Message-ID: <8112fb95-a545-baa9-fe97-9a6d79228036@redhat.com>
On 12/08/2016 08:00 PM, mark.reinhold at oracle.com wrote:
> 2016/12/8 8:26:40 -0800, david.lloyd at redhat.com:
>> 2016/12/7 15:40:17 -0800, mark.reinhold at oracle.com:
>>> David -- Are these restrictions acceptable in your use cases, or if not
>>> then at least tolerable? I'm pretty sure I've never seen any of these
>>> characters in Java EE module names, JAR file base names, Maven group or
>>> artifact names, or the other examples you mentioned.
>>
>> Breaking it down one by one...
>>
>> ':' is going to break two things that I know of: modules generated from
>> Maven coordinates which have the syntax
>> <groupId>:<artifactId>:<classifier>, and modules in the JBoss Modules
>> static loader which have a slot component, using the syntax
>> "module-name:slot".
>>
>> '@' might be OK to reserve; I can't think of any specific conflicts,
>> though we have allowed this character in the past.
>>
>> '\' is a problem because in JBoss Modules uses that character to escape
>> ':' (particularly in the Maven coordinates case) to avoid mixing up the
>> slot name with the module name. For a module named `foo\:bar:5`, the
>> static module loader would treat the name component as "foo:bar" and the
>> slot as "5", and locate the module accordingly, however the core system
>> does not treat '\' specially: the proper name of this module would be
>> `foo\:bar:5` according to the system, and that's the string you would
>> have to use to load the module by name.
>
> Okay, so the obvious thing to do is define `\` as a quotation character
> for itself, for `:`, and for `@`. Then you can simply quote/dequote
> wherever you interface with JPMS.
That should be fine, I think.
>> '/' also may be a problem because within our container, we use file
>> names from the file system as the name of modules that come from the
>> file system. This also causes a problem for '\' on Windows. We could
>> possibly work out some kind of alternative in this case, with some
>> creative thinking.
>
> I don't think we need to reserve `/`, as I said to R?mi.
>
> As for `\`, I'm pretty sure that every version of Windows shipped this
> century will accept `/` in pathnames as an alternative to `\`.
Yeah, but in any event allowing \ as an escape solves everything, and
limiting the number of specials prevents it from becoming too unwieldy,
so I'm satisfied with that.
>> Definitely in favor of forbidding non-printing code points 0x00 through
>> 0x1F, and probably also 0x80 through 0x9F (we probably don't want to go
>> any further down the Unicode rabbit hole than that though - at least,
>> not in the JVM - if we want to get out of here this side of 2020).
>
> Validating any code points past 0x7f is problematic at the JVM level
> since it's UTF-8, so you'd have to decode it.
Fair enough.
Thanks!
--
- DML
From david.lloyd at redhat.com Fri Dec 9 14:59:57 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 9 Dec 2016 08:59:57 -0600
Subject: Restrictions on service loading
Message-ID: <4d44823a-bf79-5ecf-281f-d9cf8777d341@redhat.com>
It doesn't seem clear whether this is a spec or implementation issue, so
I'm posting here to start.
The gist of the problem is, code running in a module cannot use
ServiceLoader to find things that it hasn't been declared to "use" even
if a target class loader is specified. This doesn't really make a lot
of sense because in the case where you're specifying a class loader, you
are pretty clearly establishing that you want to search the class
loader, not a module space.
This can be worked around by adding a "uses" at run time, but it
shouldn't be necessary for the explicit class loader case.
--
- DML
From mark.reinhold at oracle.com Fri Dec 9 16:39:16 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Fri, 09 Dec 2016 08:39:16 -0800
Subject: Proposal: #ModuleNameCharacters
In-Reply-To: <8112fb95-a545-baa9-fe97-9a6d79228036@redhat.com>
References: <20161122164935.7633C18948@eggemoggin.niobe.net>
<3c1891c8-b8c5-c1ba-ec87-34b7824fd7f7@redhat.com>
<2034282380.1641047.1479931198355.JavaMail.zimbra@u-pem.fr>
<20161130001102.E7C5C1C6C1@eggemoggin.niobe.net>
<947350194.519683.1481011738220.JavaMail.zimbra@u-pem.fr>
<20161207234017.1761F213EC@eggemoggin.niobe.net>
<564127947.1475086.1481192772861.JavaMail.zimbra@u-pem.fr>
<0cff8b9f-5c78-61cc-8468-40969748a103@redhat.com>
<20161208180008.393555671@eggemoggin.niobe.net>
<8112fb95-a545-baa9-fe97-9a6d79228036@redhat.com>
Message-ID: <20161209083916.612643363@eggemoggin.niobe.net>
2016/12/8 18:12:10 -0800, david.lloyd at redhat.com:
> On 12/08/2016 08:00 PM, mark.reinhold at oracle.com wrote:
>> ...
>>
>> Okay, so the obvious thing to do is define `\` as a quotation character
>> for itself, for `:`, and for `@`. Then you can simply quote/dequote
>> wherever you interface with JPMS.
>
> That should be fine, I think.
>
>>> '/' also may be a problem because within our container, we use file
>>> names from the file system as the name of modules that come from the
>>> file system. This also causes a problem for '\' on Windows. We could
>>> possibly work out some kind of alternative in this case, with some
>>> creative thinking.
>>
>> I don't think we need to reserve `/`, as I said to R?mi.
>>
>> As for `\`, I'm pretty sure that every version of Windows shipped this
>> century will accept `/` in pathnames as an alternative to `\`.
>
> Yeah, but in any event allowing \ as an escape solves everything, and
> limiting the number of specials prevents it from becoming too unwieldy,
> so I'm satisfied with that.
Okay, good. I'll send a revised proposal for #ModuleNameCharacters
shortly.
- Mark
From mark.reinhold at oracle.com Fri Dec 9 21:45:46 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Fri, 9 Dec 2016 13:45:46 -0800 (PST)
Subject: Proposal: #ModuleNameCharacters (revised)
Message-ID: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
Issue summary
-------------
#ModuleNameCharacters --- Module names are presently constrained to
be Java identifiers. Some existing module systems allow additional
characters in module names, such as hyphens and slashes. Should this
restriction be lifted or, perhaps, should it somehow be made
layer-specific? [1]
Proposal
--------
Do not change the treatment of module names in source code; they will
remain qualified names. Revise the encoding of module names in compiled
module-declaration class files to lift the current constraints but adopt
new, less onerous constraints that still provide for the future evolution
of the platform. Revise the format of class files to structure module
and package names in a manner consistent with that already used for other
kinds of constrained names.
* * *
Modules are a new construct of the Java programming language in the
present design. In the source language they are hence identified by
qualified names [2] in the same manner as the existing structural
constructs, i.e., packages and classes. As such these names do allow
some unusual characters, though not hyphens or slashes [3].
In the very long term a future version of the language may well support
not just the declaration of modules, and of relationships between them,
but also the expression of operations upon them as is possible in, e.g.,
Standard ML [4], or qualified references in code to a type in some other
named module, or yet some other kind of use that we do not imagine today.
It would hence be unwise at this point to allow module names in source
code to be any different in nature than the other kinds of qualified
names already in the language.
We will therefore retain the present constraints on module names in the
source language and also continue to enforce those constraints in the
`ModuleDescriptor.Builder` API, which is intended to be consistent with
the language. (The `ModuleDescriptor` API will continue to be able to
read class files that contain module names not expressible in the source
language.)
* * *
Module names in compiled module-declaration class files are presently
encoded in the internal form traditionally used for qualified names:
Periods (`.`) are replaced with forward slashes (`/`), and periods,
semicolons (`;`), and left square brackets (`[`) are forbidden [5].
This encoding is inconvenient for other module systems that may
interoperate with JPMS, so we will abandon it for module names despite
the fact that doing so will increase the complexity of any code that
parses class files.
To allow for the future evolution of the platform we propose a different,
less onerous encoding of module names in class files:
- If at some future point we find that we need to add structure to
module names, or combine module names with qualified type names,
then the `:` character would be a good candidate, even in the
source language if need be, so we reserve that character now.
- We presently use `@` in the API to separate module names from
version strings, where available, so it is prudent to reserve
that character in module names in class files also, just in case
we someday decide to introduce compound module identifiers that
combine module names with version strings.
- In further support of interoperation we will reserve the universal
escape character (`\`) and define the sequences `\\`, `\:`, and
`\@` to stand for `\`, `:`, and `@`, respectively.
- We will finally, for sanity, forbid any character whose Unicode code
point is less than 0x20 (` `). (Ideally we'd forbid all Unicode
non-printing characters, but it's best not to have the JVMS depend
too deeply upon details of the Unicode specification.)
To sum up: In module names in class files reserve `:` and `@` for future
use; reserve `\` as an escape character and use it to quote itself, `:`,
and `@`; and forbid the non-printing ASCII characters (< 0x20).
* * *
The first version of this proposal [6] claimed that the present design is
consistent with the existing treatment of qualified names in class files.
That is, in fact, not true, since qualified names in class files today
are always wrapped in tagged constant-pool structures rather than simple
`CONSTANT_Utf8_info` structures. Class names, e.g., are wrapped in
`CONSTANT_Class_info` structures, which in turn reference the `Utf8`
structures that represent the actual class names [7].
To address this inconsistency, and particularly in light of the new
encoding of module names described above, we propose to use consistent
kinds of class-file structures for module and package names.
Module names in a compiled module-declaration class file will be encoded
as above and wrapped in tagged `CONSTANT_Module_info` structures:
CONSTANT_Module_info {
u1 tag; // == CONSTANT_Module == 19
u2 name_index; // Index of a CONSTANT_Utf8_info
}
Package names in class files will be encoded in the traditional internal
form and wrapped in tagged `CONSTANT_Package_info` structures:
CONSTANT_Package_info {
u1 tag; // == CONSTANT_Package == 20
u2 name_index; // Index of a CONSTANT_Utf8_info
}
Existing references in the class-file format to module and package names
will be adjusted to refer to these new kinds of tagged structures.
[1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameCharacters
[2] http://docs.oracle.com/javase/specs/jls/se8/html/jls-6.html#jls-6.2
[3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.8
[4] https://en.wikipedia.org/wiki/Standard_ML#Module_system
[5] http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.2.1
[6] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-November/000468.html
[7] http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4.1
From mark.reinhold at oracle.com Fri Dec 9 21:46:46 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Fri, 9 Dec 2016 13:46:46 -0800 (PST)
Subject: Proposal: #VersionedDependences
Message-ID: <20161209214646.D915621F70@eggemoggin.niobe.net>
Issue summary
-------------
#VersionedDependences --- Consider allowing specific version strings,
or perhaps version constraints, as an optional element of `requires`
clauses in module declarations. Failing that, consider allowing
specific version strings, or perhaps version constraints, to be added
to the dependences recorded in a compiled module descriptor; this
would, e.g., allow a compiler or build system to record the versions
of the modules against which a particular module was compiled, for
use by other tools. In either case, if such version information is
merely informative then it will still honor the version selection
non-requirement; if such version information is interpreted by the
module system then that requirement may come into question. [1]
Proposal
--------
When compiling a module that depends on some other modules, record the
version strings of those modules, if available, in the resulting module
descriptor. These version strings will not be interpreted, in any phase;
they are provided solely for diagnostic purposes and for use by tools,
frameworks, and containers that assist in the selection and validation
of module versions. This proposal, therefore, continues to honor the
version-selection non-requirement [2].
The specific changes to implement this feature include:
- Extend module descriptors to allow the inclusion of version strings
in the `requires` table of the `Module` attribute by introducing a
`u2 requires_version_index` field after the `requires_flags` field
of each entry of that table. The value of this new field is either
zero, to indicate that no version string was recorded, or else the
constant-pool index of a `CONSTANT_Utf8_info` structure whose value
is the version string.
- Extend the `java.lang.module.ModuleDescriptor.Requires` API with a
single new method:
Optional<ModuleDescriptor.Version> compiledVersion();
This method will return an empty `Optional` object if no version
string was recorded for the corresponding dependence, or else an
`Optional` that contains the recorded version string.
- Recommend that Java language compilers provide a means by which a
version string can be indicated for any modules being compiled, and
that they record such strings in the resulting module descriptors.
(The RI's `javac` compiler will support this via a new command-line
option, `--module-version`.)
- If a compiler can record the version string of the modules being
compiled in the resulting compiled module descriptors then it would
be simpler for those version strings to be placed in the `Module`
attribute rather than the ancillary `ModuleVersion` attribute. We'll
therefore drop the latter and add a `u2 module_version_index` field
to the `Module` attribute, immediately following the `module_flags`
field.
Now that compile-time versions can be recorded in module descriptors
there is even less need to tolerate version information in module names,
a bad practice that we'd like to discourage at the outset. We therefore
further propose to:
- Revise the accepted proposal for #VersionsInModuleNames [3] to state
that a module name appearing anywhere in a source-form module
declaration must both start and end with "Java letters" [4]. This
includes the name of the module being declared and also the names of
other modules mentioned in `requires` directives or in qualified
`exports` or `opens` directives. This restriction will be enforced
by the `ModuleDescriptor.Builder` API, which is intended to be
consistent with the source language. The `ModuleDescriptor` API
itself will continue to be able to read class files that contain
module names that violate this constraint.
[1] http://openjdk.java.net/projects/jigsaw/spec/issues/#VersionedDependences
[2] http://openjdk.java.net/projects/jigsaw/spec/reqs/#version-selection
[3] http://openjdk.java.net/projects/jigsaw/spec/issues/#VersionsInModuleNames
[4] http://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-JavaLetter
From david.lloyd at redhat.com Fri Dec 9 21:48:54 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 9 Dec 2016 15:48:54 -0600
Subject: Proposal: #ModuleNameCharacters (revised)
In-Reply-To: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
References: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
Message-ID: <16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
+1 here
On 12/09/2016 03:45 PM, mark.reinhold at oracle.com wrote:
> Issue summary
> -------------
>
> #ModuleNameCharacters --- Module names are presently constrained to
> be Java identifiers. Some existing module systems allow additional
> characters in module names, such as hyphens and slashes. Should this
> restriction be lifted or, perhaps, should it somehow be made
> layer-specific? [1]
>
> Proposal
> --------
>
> Do not change the treatment of module names in source code; they will
> remain qualified names. Revise the encoding of module names in compiled
> module-declaration class files to lift the current constraints but adopt
> new, less onerous constraints that still provide for the future evolution
> of the platform. Revise the format of class files to structure module
> and package names in a manner consistent with that already used for other
> kinds of constrained names.
>
> * * *
>
> Modules are a new construct of the Java programming language in the
> present design. In the source language they are hence identified by
> qualified names [2] in the same manner as the existing structural
> constructs, i.e., packages and classes. As such these names do allow
> some unusual characters, though not hyphens or slashes [3].
>
> In the very long term a future version of the language may well support
> not just the declaration of modules, and of relationships between them,
> but also the expression of operations upon them as is possible in, e.g.,
> Standard ML [4], or qualified references in code to a type in some other
> named module, or yet some other kind of use that we do not imagine today.
> It would hence be unwise at this point to allow module names in source
> code to be any different in nature than the other kinds of qualified
> names already in the language.
>
> We will therefore retain the present constraints on module names in the
> source language and also continue to enforce those constraints in the
> `ModuleDescriptor.Builder` API, which is intended to be consistent with
> the language. (The `ModuleDescriptor` API will continue to be able to
> read class files that contain module names not expressible in the source
> language.)
>
> * * *
>
> Module names in compiled module-declaration class files are presently
> encoded in the internal form traditionally used for qualified names:
> Periods (`.`) are replaced with forward slashes (`/`), and periods,
> semicolons (`;`), and left square brackets (`[`) are forbidden [5].
> This encoding is inconvenient for other module systems that may
> interoperate with JPMS, so we will abandon it for module names despite
> the fact that doing so will increase the complexity of any code that
> parses class files.
>
> To allow for the future evolution of the platform we propose a different,
> less onerous encoding of module names in class files:
>
> - If at some future point we find that we need to add structure to
> module names, or combine module names with qualified type names,
> then the `:` character would be a good candidate, even in the
> source language if need be, so we reserve that character now.
>
> - We presently use `@` in the API to separate module names from
> version strings, where available, so it is prudent to reserve
> that character in module names in class files also, just in case
> we someday decide to introduce compound module identifiers that
> combine module names with version strings.
>
> - In further support of interoperation we will reserve the universal
> escape character (`\`) and define the sequences `\\`, `\:`, and
> `\@` to stand for `\`, `:`, and `@`, respectively.
>
> - We will finally, for sanity, forbid any character whose Unicode code
> point is less than 0x20 (` `). (Ideally we'd forbid all Unicode
> non-printing characters, but it's best not to have the JVMS depend
> too deeply upon details of the Unicode specification.)
>
> To sum up: In module names in class files reserve `:` and `@` for future
> use; reserve `\` as an escape character and use it to quote itself, `:`,
> and `@`; and forbid the non-printing ASCII characters (< 0x20).
>
> * * *
>
> The first version of this proposal [6] claimed that the present design is
> consistent with the existing treatment of qualified names in class files.
> That is, in fact, not true, since qualified names in class files today
> are always wrapped in tagged constant-pool structures rather than simple
> `CONSTANT_Utf8_info` structures. Class names, e.g., are wrapped in
> `CONSTANT_Class_info` structures, which in turn reference the `Utf8`
> structures that represent the actual class names [7].
>
> To address this inconsistency, and particularly in light of the new
> encoding of module names described above, we propose to use consistent
> kinds of class-file structures for module and package names.
>
> Module names in a compiled module-declaration class file will be encoded
> as above and wrapped in tagged `CONSTANT_Module_info` structures:
>
> CONSTANT_Module_info {
> u1 tag; // == CONSTANT_Module == 19
> u2 name_index; // Index of a CONSTANT_Utf8_info
> }
>
> Package names in class files will be encoded in the traditional internal
> form and wrapped in tagged `CONSTANT_Package_info` structures:
>
> CONSTANT_Package_info {
> u1 tag; // == CONSTANT_Package == 20
> u2 name_index; // Index of a CONSTANT_Utf8_info
> }
>
> Existing references in the class-file format to module and package names
> will be adjusted to refer to these new kinds of tagged structures.
>
>
> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameCharacters
> [2] http://docs.oracle.com/javase/specs/jls/se8/html/jls-6.html#jls-6.2
> [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.8
> [4] https://en.wikipedia.org/wiki/Standard_ML#Module_system
> [5] http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.2.1
> [6] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-November/000468.html
> [7] http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4.1
>
--
- DML
From david.lloyd at redhat.com Fri Dec 9 22:05:02 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 9 Dec 2016 16:05:02 -0600
Subject: Proposal: #ModuleNameCharacters (revised)
In-Reply-To: <16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
References: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
<16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
Message-ID: <6c9bbb7f-133c-0600-d13a-1154098532de@redhat.com>
Whoops, hang on... one problem I didn't spot on my first read-through:
>> We will therefore retain the present constraints on module names in the
>> source language and also continue to enforce those constraints in the
>> `ModuleDescriptor.Builder` API, which is intended to be consistent with
>> the language. (The `ModuleDescriptor` API will continue to be able to
>> read class files that contain module names not expressible in the source
>> language.)
So... essentially a custom module system has to generate binary
descriptors? That's going to be a real pain.
On 12/09/2016 03:48 PM, David M. Lloyd wrote:
> +1 here
>
> On 12/09/2016 03:45 PM, mark.reinhold at oracle.com wrote:
>> Issue summary
>> -------------
>>
>> #ModuleNameCharacters --- Module names are presently constrained to
>> be Java identifiers. Some existing module systems allow additional
>> characters in module names, such as hyphens and slashes. Should this
>> restriction be lifted or, perhaps, should it somehow be made
>> layer-specific? [1]
>>
>> Proposal
>> --------
>>
>> Do not change the treatment of module names in source code; they will
>> remain qualified names. Revise the encoding of module names in compiled
>> module-declaration class files to lift the current constraints but adopt
>> new, less onerous constraints that still provide for the future evolution
>> of the platform. Revise the format of class files to structure module
>> and package names in a manner consistent with that already used for other
>> kinds of constrained names.
>>
>> * * *
>>
>> Modules are a new construct of the Java programming language in the
>> present design. In the source language they are hence identified by
>> qualified names [2] in the same manner as the existing structural
>> constructs, i.e., packages and classes. As such these names do allow
>> some unusual characters, though not hyphens or slashes [3].
>>
>> In the very long term a future version of the language may well support
>> not just the declaration of modules, and of relationships between them,
>> but also the expression of operations upon them as is possible in, e.g.,
>> Standard ML [4], or qualified references in code to a type in some other
>> named module, or yet some other kind of use that we do not imagine today.
>> It would hence be unwise at this point to allow module names in source
>> code to be any different in nature than the other kinds of qualified
>> names already in the language.
>>
>> We will therefore retain the present constraints on module names in the
>> source language and also continue to enforce those constraints in the
>> `ModuleDescriptor.Builder` API, which is intended to be consistent with
>> the language. (The `ModuleDescriptor` API will continue to be able to
>> read class files that contain module names not expressible in the source
>> language.)
>>
>> * * *
>>
>> Module names in compiled module-declaration class files are presently
>> encoded in the internal form traditionally used for qualified names:
>> Periods (`.`) are replaced with forward slashes (`/`), and periods,
>> semicolons (`;`), and left square brackets (`[`) are forbidden [5].
>> This encoding is inconvenient for other module systems that may
>> interoperate with JPMS, so we will abandon it for module names despite
>> the fact that doing so will increase the complexity of any code that
>> parses class files.
>>
>> To allow for the future evolution of the platform we propose a different,
>> less onerous encoding of module names in class files:
>>
>> - If at some future point we find that we need to add structure to
>> module names, or combine module names with qualified type names,
>> then the `:` character would be a good candidate, even in the
>> source language if need be, so we reserve that character now.
>>
>> - We presently use `@` in the API to separate module names from
>> version strings, where available, so it is prudent to reserve
>> that character in module names in class files also, just in case
>> we someday decide to introduce compound module identifiers that
>> combine module names with version strings.
>>
>> - In further support of interoperation we will reserve the universal
>> escape character (`\`) and define the sequences `\\`, `\:`, and
>> `\@` to stand for `\`, `:`, and `@`, respectively.
>>
>> - We will finally, for sanity, forbid any character whose Unicode code
>> point is less than 0x20 (` `). (Ideally we'd forbid all Unicode
>> non-printing characters, but it's best not to have the JVMS depend
>> too deeply upon details of the Unicode specification.)
>>
>> To sum up: In module names in class files reserve `:` and `@` for future
>> use; reserve `\` as an escape character and use it to quote itself, `:`,
>> and `@`; and forbid the non-printing ASCII characters (< 0x20).
>>
>> * * *
>>
>> The first version of this proposal [6] claimed that the present design is
>> consistent with the existing treatment of qualified names in class files.
>> That is, in fact, not true, since qualified names in class files today
>> are always wrapped in tagged constant-pool structures rather than simple
>> `CONSTANT_Utf8_info` structures. Class names, e.g., are wrapped in
>> `CONSTANT_Class_info` structures, which in turn reference the `Utf8`
>> structures that represent the actual class names [7].
>>
>> To address this inconsistency, and particularly in light of the new
>> encoding of module names described above, we propose to use consistent
>> kinds of class-file structures for module and package names.
>>
>> Module names in a compiled module-declaration class file will be encoded
>> as above and wrapped in tagged `CONSTANT_Module_info` structures:
>>
>> CONSTANT_Module_info {
>> u1 tag; // == CONSTANT_Module == 19
>> u2 name_index; // Index of a CONSTANT_Utf8_info
>> }
>>
>> Package names in class files will be encoded in the traditional internal
>> form and wrapped in tagged `CONSTANT_Package_info` structures:
>>
>> CONSTANT_Package_info {
>> u1 tag; // == CONSTANT_Package == 20
>> u2 name_index; // Index of a CONSTANT_Utf8_info
>> }
>>
>> Existing references in the class-file format to module and package names
>> will be adjusted to refer to these new kinds of tagged structures.
>>
>>
>> [1]
>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameCharacters
>> [2] http://docs.oracle.com/javase/specs/jls/se8/html/jls-6.html#jls-6.2
>> [3] http://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.8
>> [4] https://en.wikipedia.org/wiki/Standard_ML#Module_system
>> [5]
>> http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.2.1
>> [6]
>> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-November/000468.html
>>
>> [7]
>> http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4.1
>>
>
--
- DML
From david.lloyd at redhat.com Fri Dec 9 22:10:43 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 9 Dec 2016 16:10:43 -0600
Subject: Proposal: #VersionedDependences
In-Reply-To: <20161209214646.D915621F70@eggemoggin.niobe.net>
References: <20161209214646.D915621F70@eggemoggin.niobe.net>
Message-ID: <c9e30e57-c832-a01b-23c9-532c08cd840d@redhat.com>
Inline...
On 12/09/2016 03:46 PM, mark.reinhold at oracle.com wrote:
> Issue summary
> -------------
> [...]
>
> Proposal
> --------
>
> When compiling a module that depends on some other modules, record the
> version strings of those modules, if available, in the resulting module
> descriptor. These version strings will not be interpreted, in any phase;
> they are provided solely for diagnostic purposes and for use by tools,
> frameworks, and containers that assist in the selection and validation
> of module versions. This proposal, therefore, continues to honor the
> version-selection non-requirement [2].
I can see this being very useful.
> The specific changes to implement this feature include:
>
> - Extend module descriptors to allow the inclusion of version strings
> [...]
> - Extend the `java.lang.module.ModuleDescriptor.Requires` API with a
> single new method:
> [...]
> - Recommend that Java language compilers provide a means by which a
> version string can be indicated for any modules being compiled, and
> that they record such strings in the resulting module descriptors.
Great, this will be very useful.
> - If a compiler can record the version string of the modules being
> compiled in the resulting compiled module descriptors then it would
> be simpler for those version strings to be placed in the `Module`
> attribute rather than the ancillary `ModuleVersion` attribute. We'll
> therefore drop the latter and add a `u2 module_version_index` field
> to the `Module` attribute, immediately following the `module_flags`
> field.
OK
> Now that compile-time versions can be recorded in module descriptors
> there is even less need to tolerate version information in module names,
> a bad practice that we'd like to discourage at the outset. We therefore
> further propose to:
>
> - Revise the accepted proposal for #VersionsInModuleNames [3] to state
> that a module name appearing anywhere in a source-form module
> declaration must both start and end with "Java letters" [4].
Can we just drop this part? I really am not a fan of the
social-enforcement-in-technical-code thing, and I can already think of
one existing project off the top of my head that will suffer
collaterally from this: Fabric8. Also any other project to which the
"vanity license plate effect" would apply. And I can think of many ways
to circumvent this rule, including, but not limited to, bracketing with
letters "v5slot", roman numerals, etc., so really all it does is add an
annoying "big brother" effect without practical benefit.
Anyway I disagree pretty strongly that versions (or more specifically,
version segments) in module names are that really that strong of an
anti-pattern. Sure having whole version numbers in is a pretty fragile
technique, but it's very useful to have (for example) multiple major
versions of a library on a large distribution to ease migration,
especially when you're talking hundreds or thousands of modules.
> This
> includes the name of the module being declared and also the names of
> other modules mentioned in `requires` directives or in qualified
> `exports` or `opens` directives. This restriction will be enforced
> by the `ModuleDescriptor.Builder` API, which is intended to be
> consistent with the source language.
See other email ("whoops...") on this bit. It would be good to have
*some* way to programmatically establish modules without having to
generate bytecode.
--
- DML
From forax at univ-mlv.fr Sat Dec 10 07:50:39 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Sat, 10 Dec 2016 07:50:39 +0000
Subject: Proposal: #ModuleNameCharacters (revised)
In-Reply-To: <6c9bbb7f-133c-0600-d13a-1154098532de@redhat.com>
References: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
<16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
<6c9bbb7f-133c-0600-d13a-1154098532de@redhat.com>
Message-ID: <610BAFF3-B5AB-467D-A4F6-F51050274344@univ-mlv.fr>
No, i hope it's more that ModuleDescriptor will be an interface.
So we can have our own module descriptor builder.
R?mi
On December 9, 2016 11:05:02 PM GMT+01:00, "David M. Lloyd" <david.lloyd at redhat.com> wrote:
>Whoops, hang on... one problem I didn't spot on my first read-through:
>
>>> We will therefore retain the present constraints on module names in
>the
>>> source language and also continue to enforce those constraints in
>the
>>> `ModuleDescriptor.Builder` API, which is intended to be consistent
>with
>>> the language. (The `ModuleDescriptor` API will continue to be able
>to
>>> read class files that contain module names not expressible in the
>source
>>> language.)
>
>So... essentially a custom module system has to generate binary
>descriptors? That's going to be a real pain.
>
>
>On 12/09/2016 03:48 PM, David M. Lloyd wrote:
>> +1 here
>>
>> On 12/09/2016 03:45 PM, mark.reinhold at oracle.com wrote:
>>> Issue summary
>>> -------------
>>>
>>> #ModuleNameCharacters --- Module names are presently constrained
>to
>>> be Java identifiers. Some existing module systems allow
>additional
>>> characters in module names, such as hyphens and slashes. Should
>this
>>> restriction be lifted or, perhaps, should it somehow be made
>>> layer-specific? [1]
>>>
>>> Proposal
>>> --------
>>>
>>> Do not change the treatment of module names in source code; they
>will
>>> remain qualified names. Revise the encoding of module names in
>compiled
>>> module-declaration class files to lift the current constraints but
>adopt
>>> new, less onerous constraints that still provide for the future
>evolution
>>> of the platform. Revise the format of class files to structure
>module
>>> and package names in a manner consistent with that already used for
>other
>>> kinds of constrained names.
>>>
>>> * * *
>>>
>>> Modules are a new construct of the Java programming language in the
>>> present design. In the source language they are hence identified by
>>> qualified names [2] in the same manner as the existing structural
>>> constructs, i.e., packages and classes. As such these names do
>allow
>>> some unusual characters, though not hyphens or slashes [3].
>>>
>>> In the very long term a future version of the language may well
>support
>>> not just the declaration of modules, and of relationships between
>them,
>>> but also the expression of operations upon them as is possible in,
>e.g.,
>>> Standard ML [4], or qualified references in code to a type in some
>other
>>> named module, or yet some other kind of use that we do not imagine
>today.
>>> It would hence be unwise at this point to allow module names in
>source
>>> code to be any different in nature than the other kinds of qualified
>>> names already in the language.
>>>
>>> We will therefore retain the present constraints on module names in
>the
>>> source language and also continue to enforce those constraints in
>the
>>> `ModuleDescriptor.Builder` API, which is intended to be consistent
>with
>>> the language. (The `ModuleDescriptor` API will continue to be able
>to
>>> read class files that contain module names not expressible in the
>source
>>> language.)
>>>
>>> * * *
>>>
>>> Module names in compiled module-declaration class files are
>presently
>>> encoded in the internal form traditionally used for qualified names:
>>> Periods (`.`) are replaced with forward slashes (`/`), and periods,
>>> semicolons (`;`), and left square brackets (`[`) are forbidden [5].
>>> This encoding is inconvenient for other module systems that may
>>> interoperate with JPMS, so we will abandon it for module names
>despite
>>> the fact that doing so will increase the complexity of any code that
>>> parses class files.
>>>
>>> To allow for the future evolution of the platform we propose a
>different,
>>> less onerous encoding of module names in class files:
>>>
>>> - If at some future point we find that we need to add structure to
>>> module names, or combine module names with qualified type names,
>>> then the `:` character would be a good candidate, even in the
>>> source language if need be, so we reserve that character now.
>>>
>>> - We presently use `@` in the API to separate module names from
>>> version strings, where available, so it is prudent to reserve
>>> that character in module names in class files also, just in case
>>> we someday decide to introduce compound module identifiers that
>>> combine module names with version strings.
>>>
>>> - In further support of interoperation we will reserve the
>universal
>>> escape character (`\`) and define the sequences `\\`, `\:`, and
>>> `\@` to stand for `\`, `:`, and `@`, respectively.
>>>
>>> - We will finally, for sanity, forbid any character whose Unicode
>code
>>> point is less than 0x20 (` `). (Ideally we'd forbid all Unicode
>>> non-printing characters, but it's best not to have the JVMS
>depend
>>> too deeply upon details of the Unicode specification.)
>>>
>>> To sum up: In module names in class files reserve `:` and `@` for
>future
>>> use; reserve `\` as an escape character and use it to quote itself,
>`:`,
>>> and `@`; and forbid the non-printing ASCII characters (< 0x20).
>>>
>>> * * *
>>>
>>> The first version of this proposal [6] claimed that the present
>design is
>>> consistent with the existing treatment of qualified names in class
>files.
>>> That is, in fact, not true, since qualified names in class files
>today
>>> are always wrapped in tagged constant-pool structures rather than
>simple
>>> `CONSTANT_Utf8_info` structures. Class names, e.g., are wrapped in
>>> `CONSTANT_Class_info` structures, which in turn reference the `Utf8`
>>> structures that represent the actual class names [7].
>>>
>>> To address this inconsistency, and particularly in light of the new
>>> encoding of module names described above, we propose to use
>consistent
>>> kinds of class-file structures for module and package names.
>>>
>>> Module names in a compiled module-declaration class file will be
>encoded
>>> as above and wrapped in tagged `CONSTANT_Module_info` structures:
>>>
>>> CONSTANT_Module_info {
>>> u1 tag; // == CONSTANT_Module == 19
>>> u2 name_index; // Index of a CONSTANT_Utf8_info
>>> }
>>>
>>> Package names in class files will be encoded in the traditional
>internal
>>> form and wrapped in tagged `CONSTANT_Package_info` structures:
>>>
>>> CONSTANT_Package_info {
>>> u1 tag; // == CONSTANT_Package == 20
>>> u2 name_index; // Index of a CONSTANT_Utf8_info
>>> }
>>>
>>> Existing references in the class-file format to module and package
>names
>>> will be adjusted to refer to these new kinds of tagged structures.
>>>
>>>
>>> [1]
>>>
>http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameCharacters
>>> [2]
>http://docs.oracle.com/javase/specs/jls/se8/html/jls-6.html#jls-6.2
>>> [3]
>http://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.8
>>> [4] https://en.wikipedia.org/wiki/Standard_ML#Module_system
>>> [5]
>>>
>http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.2.1
>>> [6]
>>>
>http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-November/000468.html
>>>
>>> [7]
>>>
>http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4.1
>>>
>>
>
>--
>- DML
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
From forax at univ-mlv.fr Sat Dec 10 07:51:09 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Sat, 10 Dec 2016 07:51:09 +0000
Subject: Proposal: #ModuleNameCharacters (revised)
In-Reply-To: <16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
References: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
<16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
Message-ID: <39014A1A-CF81-4485-955B-EE974955A86A@univ-mlv.fr>
Same here :)
R?mi
On December 9, 2016 10:48:54 PM GMT+01:00, "David M. Lloyd" <david.lloyd at redhat.com> wrote:
>+1 here
>
>On 12/09/2016 03:45 PM, mark.reinhold at oracle.com wrote:
>> Issue summary
>> -------------
>>
>> #ModuleNameCharacters --- Module names are presently constrained to
>> be Java identifiers. Some existing module systems allow additional
>> characters in module names, such as hyphens and slashes. Should
>this
>> restriction be lifted or, perhaps, should it somehow be made
>> layer-specific? [1]
>>
>> Proposal
>> --------
>>
>> Do not change the treatment of module names in source code; they will
>> remain qualified names. Revise the encoding of module names in
>compiled
>> module-declaration class files to lift the current constraints but
>adopt
>> new, less onerous constraints that still provide for the future
>evolution
>> of the platform. Revise the format of class files to structure
>module
>> and package names in a manner consistent with that already used for
>other
>> kinds of constrained names.
>>
>> * * *
>>
>> Modules are a new construct of the Java programming language in the
>> present design. In the source language they are hence identified by
>> qualified names [2] in the same manner as the existing structural
>> constructs, i.e., packages and classes. As such these names do allow
>> some unusual characters, though not hyphens or slashes [3].
>>
>> In the very long term a future version of the language may well
>support
>> not just the declaration of modules, and of relationships between
>them,
>> but also the expression of operations upon them as is possible in,
>e.g.,
>> Standard ML [4], or qualified references in code to a type in some
>other
>> named module, or yet some other kind of use that we do not imagine
>today.
>> It would hence be unwise at this point to allow module names in
>source
>> code to be any different in nature than the other kinds of qualified
>> names already in the language.
>>
>> We will therefore retain the present constraints on module names in
>the
>> source language and also continue to enforce those constraints in the
>> `ModuleDescriptor.Builder` API, which is intended to be consistent
>with
>> the language. (The `ModuleDescriptor` API will continue to be able
>to
>> read class files that contain module names not expressible in the
>source
>> language.)
>>
>> * * *
>>
>> Module names in compiled module-declaration class files are presently
>> encoded in the internal form traditionally used for qualified names:
>> Periods (`.`) are replaced with forward slashes (`/`), and periods,
>> semicolons (`;`), and left square brackets (`[`) are forbidden [5].
>> This encoding is inconvenient for other module systems that may
>> interoperate with JPMS, so we will abandon it for module names
>despite
>> the fact that doing so will increase the complexity of any code that
>> parses class files.
>>
>> To allow for the future evolution of the platform we propose a
>different,
>> less onerous encoding of module names in class files:
>>
>> - If at some future point we find that we need to add structure to
>> module names, or combine module names with qualified type names,
>> then the `:` character would be a good candidate, even in the
>> source language if need be, so we reserve that character now.
>>
>> - We presently use `@` in the API to separate module names from
>> version strings, where available, so it is prudent to reserve
>> that character in module names in class files also, just in case
>> we someday decide to introduce compound module identifiers that
>> combine module names with version strings.
>>
>> - In further support of interoperation we will reserve the
>universal
>> escape character (`\`) and define the sequences `\\`, `\:`, and
>> `\@` to stand for `\`, `:`, and `@`, respectively.
>>
>> - We will finally, for sanity, forbid any character whose Unicode
>code
>> point is less than 0x20 (` `). (Ideally we'd forbid all Unicode
>> non-printing characters, but it's best not to have the JVMS
>depend
>> too deeply upon details of the Unicode specification.)
>>
>> To sum up: In module names in class files reserve `:` and `@` for
>future
>> use; reserve `\` as an escape character and use it to quote itself,
>`:`,
>> and `@`; and forbid the non-printing ASCII characters (< 0x20).
>>
>> * * *
>>
>> The first version of this proposal [6] claimed that the present
>design is
>> consistent with the existing treatment of qualified names in class
>files.
>> That is, in fact, not true, since qualified names in class files
>today
>> are always wrapped in tagged constant-pool structures rather than
>simple
>> `CONSTANT_Utf8_info` structures. Class names, e.g., are wrapped in
>> `CONSTANT_Class_info` structures, which in turn reference the `Utf8`
>> structures that represent the actual class names [7].
>>
>> To address this inconsistency, and particularly in light of the new
>> encoding of module names described above, we propose to use
>consistent
>> kinds of class-file structures for module and package names.
>>
>> Module names in a compiled module-declaration class file will be
>encoded
>> as above and wrapped in tagged `CONSTANT_Module_info` structures:
>>
>> CONSTANT_Module_info {
>> u1 tag; // == CONSTANT_Module == 19
>> u2 name_index; // Index of a CONSTANT_Utf8_info
>> }
>>
>> Package names in class files will be encoded in the traditional
>internal
>> form and wrapped in tagged `CONSTANT_Package_info` structures:
>>
>> CONSTANT_Package_info {
>> u1 tag; // == CONSTANT_Package == 20
>> u2 name_index; // Index of a CONSTANT_Utf8_info
>> }
>>
>> Existing references in the class-file format to module and package
>names
>> will be adjusted to refer to these new kinds of tagged structures.
>>
>>
>> [1]
>http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameCharacters
>> [2]
>http://docs.oracle.com/javase/specs/jls/se8/html/jls-6.html#jls-6.2
>> [3]
>http://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.8
>> [4] https://en.wikipedia.org/wiki/Standard_ML#Module_system
>> [5]
>http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.2.1
>> [6]
>http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-November/000468.html
>> [7]
>http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4.1
>>
>
>--
>- DML
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
From ali.ebrahimi1781 at gmail.com Sat Dec 10 07:55:46 2016
From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi)
Date: Sat, 10 Dec 2016 11:25:46 +0330
Subject: Proposal: #VersionedDependences
In-Reply-To: <20161209214646.D915621F70@eggemoggin.niobe.net>
References: <20161209214646.D915621F70@eggemoggin.niobe.net>
Message-ID: <CAA0cW5BB16e7Q1XpVvNydzHOHBA9HX9EnWXJN+WFU=YD8n8HUA@mail.gmail.com>
Hi Mark,
All good, but why we don't support source code support for version strings
in module declarations.
module foo at 1.0 {
requires bar at 1.0;
}
Using command line options is pain when compiling multi-module code bases.
Also existence of version strings in source code of module declaration
helps in code reviews(Ah!, I seeing version 1.0 of module foo, latest is
2.0 )
On Sat, Dec 10, 2016 at 1:16 AM, <mark.reinhold at oracle.com> wrote:
> Issue summary
> -------------
>
> #VersionedDependences --- Consider allowing specific version strings,
> or perhaps version constraints, as an optional element of `requires`
> clauses in module declarations. Failing that, consider allowing
> specific version strings, or perhaps version constraints, to be added
> to the dependences recorded in a compiled module descriptor; this
> would, e.g., allow a compiler or build system to record the versions
> of the modules against which a particular module was compiled, for
> use by other tools. In either case, if such version information is
> merely informative then it will still honor the version selection
> non-requirement; if such version information is interpreted by the
> module system then that requirement may come into question. [1]
>
> Proposal
> --------
>
> When compiling a module that depends on some other modules, record the
> version strings of those modules, if available, in the resulting module
> descriptor. These version strings will not be interpreted, in any phase;
> they are provided solely for diagnostic purposes and for use by tools,
> frameworks, and containers that assist in the selection and validation
> of module versions. This proposal, therefore, continues to honor the
> version-selection non-requirement [2].
>
> The specific changes to implement this feature include:
>
> - Extend module descriptors to allow the inclusion of version strings
> in the `requires` table of the `Module` attribute by introducing a
> `u2 requires_version_index` field after the `requires_flags` field
> of each entry of that table. The value of this new field is either
> zero, to indicate that no version string was recorded, or else the
> constant-pool index of a `CONSTANT_Utf8_info` structure whose value
> is the version string.
>
> - Extend the `java.lang.module.ModuleDescriptor.Requires` API with a
> single new method:
>
> Optional<ModuleDescriptor.Version> compiledVersion();
>
> This method will return an empty `Optional` object if no version
> string was recorded for the corresponding dependence, or else an
> `Optional` that contains the recorded version string.
>
> - Recommend that Java language compilers provide a means by which a
> version string can be indicated for any modules being compiled, and
> that they record such strings in the resulting module descriptors.
> (The RI's `javac` compiler will support this via a new command-line
> option, `--module-version`.)
>
> - If a compiler can record the version string of the modules being
> compiled in the resulting compiled module descriptors then it would
> be simpler for those version strings to be placed in the `Module`
> attribute rather than the ancillary `ModuleVersion` attribute. We'll
> therefore drop the latter and add a `u2 module_version_index` field
> to the `Module` attribute, immediately following the `module_flags`
> field.
>
> Now that compile-time versions can be recorded in module descriptors
> there is even less need to tolerate version information in module names,
> a bad practice that we'd like to discourage at the outset. We therefore
> further propose to:
>
> - Revise the accepted proposal for #VersionsInModuleNames [3] to state
> that a module name appearing anywhere in a source-form module
> declaration must both start and end with "Java letters" [4]. This
> includes the name of the module being declared and also the names of
> other modules mentioned in `requires` directives or in qualified
> `exports` or `opens` directives. This restriction will be enforced
> by the `ModuleDescriptor.Builder` API, which is intended to be
> consistent with the source language. The `ModuleDescriptor` API
> itself will continue to be able to read class files that contain
> module names that violate this constraint.
>
>
> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#
> VersionedDependences
> [2] http://openjdk.java.net/projects/jigsaw/spec/reqs/#version-selection
> [3] http://openjdk.java.net/projects/jigsaw/spec/issues/#
> VersionsInModuleNames
> [4] http://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#
> jls-JavaLetter
>
--
Best Regards,
Ali Ebrahimi
From david.lloyd at redhat.com Sat Dec 10 15:24:18 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Sat, 10 Dec 2016 09:24:18 -0600
Subject: Proposal: #ModuleNameCharacters (revised)
In-Reply-To: <610BAFF3-B5AB-467D-A4F6-F51050274344@univ-mlv.fr>
References: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
<16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
<6c9bbb7f-133c-0600-d13a-1154098532de@redhat.com>
<610BAFF3-B5AB-467D-A4F6-F51050274344@univ-mlv.fr>
Message-ID: <dacc0d8c-e2a9-d1bd-d2d9-c52fa08c7083@redhat.com>
Well, maybe we can build on this idea: what about making
ModuleDescriptor.Builder non-final, so we can push validation logic into
separate overridable methods? This way the default behavior is
consistent, and it's much harder to accidentally bypass the logic, but
it's also still easy to provide a customized scheme.
On 12/10/2016 01:50 AM, Remi Forax wrote:
> No, i hope it's more that ModuleDescriptor will be an interface.
> So we can have our own module descriptor builder.
>
> R?mi
>
>
>
> On December 9, 2016 11:05:02 PM GMT+01:00, "David M. Lloyd"
> <david.lloyd at redhat.com> wrote:
>
> Whoops, hang on... one problem I didn't spot on my first read-through:
>
> We will therefore retain the present constraints on module
> names in the
> source language and also continue to enforce those
> constraints in the
> `ModuleDescriptor.Builder` API, which is intended to be
> consistent with
> the language. (The `ModuleDescriptor` API will continue to
> be able to
> read class files that contain module names not expressible
> in the source
> language.)
>
>
> So... essentially a custom module system has to generate binary
> descriptors? That's going to be a real pain.
>
>
> On 12/09/2016 03:48 PM, David M. Lloyd wrote:
>
> +1 here
>
> On 12/09/2016 03:45 PM, mark.reinhold at oracle.com wrote:
>
> Issue summary
> -------------
>
> #ModuleNameCharacters --- Module names are presently
> constrained to
> be Java identifiers. Some existing module systems allow
> additional
> characters in module names, such as hyphens and slashes.
> Should this
> restriction be lifted or, perhaps, should it somehow be made
> layer-specific? [1]
>
> Proposal
> --------
>
> Do not change the treatment of module names in source code;
> they will
> remain qualified names. Revise the encoding of module names
> in compiled
> module-declaration class files to lift the current
> constraints but adopt
> new, less onerous constraints that still provide for ! the
> future evolution
> of the platform. Revise the format of class files to
> structure module
> and package names in a manner consistent with that already
> used for other
> kinds of constrained names.
>
> * * *
>
> Modules are a new construct of the Java programming language
> in the
> present design. In the source language they are hence
> identified by
> qualified names [2] in the same manner as the existing
> structural
> constructs, i.e., packages and classes. As such these names
> do allow
> some unusual characters, though not hyphens or slashes [3].
>
> In the very long term a future version of the language may
> well support
> not just the declaration of modules, and of relationships
> between them,
> but also the expression of operations upon them as is
> possible in, e.g.,
> Standard ML [4], or qualified references in code to a type
> in some other
> named module, or yet some other ki! nd of use that we do not
> imagine today.
> It would hence be unwise at this point to allow module names
> in source
> code to be any different in nature than the other kinds of
> qualified
> names already in the language.
>
> We will therefore retain the present constraints on module
> names in the
> source language and also continue to enforce those
> constraints in the
> `ModuleDescriptor.Builder` API, which is intended to be
> consistent with
> the language. (The `ModuleDescriptor` API will continue to
> be able to
> read class files that contain module names not expressible
> in the source
> language.)
>
> * * *
>
> Module names in compiled module-declaration class files are
> presently
> encoded in the internal form traditionally used for
> qualified names:
> Periods (`.`) are replaced with forward slashes (`/`), and
> periods,
> semicolons (`;`), and left square brackets (`[`) are
> forbidden [5]! .
> This encoding is inconvenient for other module systems that may
> interoperate with JPMS, so we will abandon it for module
> names despite
> the fact that doing so will increase the complexity of any
> code that
> parses class files.
>
> To allow for the future evolution of the platform we propose
> a different,
> less onerous encoding of module names in class files:
>
> - If at some future point we find that we need to add
> structure to
> module names, or combine module names with qualified type names,
> then the `:` character would be a good candidate, even in the
> source language if need be, so we reserve that character now.
>
> - We presently use `@` in the API to separate module names from
> version strings, where available, so it is prudent to reserve
> that character in module names in class files also, just in case
> we someday decide to introduce compound module identifiers
> thatcombine module names with version strings.
>
> - In further support of interoperation we will reserve the
> universal
> escape character (`\`) and define the sequences `\\`, `\:`, and
> `\@` to stand for `\`, `:`, and `@`, respectively.
>
> - We will finally, for sanity, forbid any character whose
> Unicode code
> point is less than 0x20 (` `). (Ideally we'd forbid all Unicode
> non-printing characters, but it's best not to have the JVMS
> depend
> too deeply upon details of the Unicode specification.)
>
> To sum up: In module names in class files reserve `:` and
> `@` for future
> use; reserve `\` as an escape character and use it to quote
> itself, `:`,
> and `@`; and forbid the non-printing ASCII characters (< 0x20).
>
> * * *
>
> The first version of this proposal [6] claimed that the
> present design is
> consistent with the existing treatment of qua! lified names
> in class files.
> That is, in fact, not true, since qualified names in class
> files today
> are always wrapped in tagged constant-pool structures rather
> than simple
> `CONSTANT_Utf8_info` structures. Class names, e.g., are
> wrapped in
> `CONSTANT_Class_info` structures, which in turn reference
> the `Utf8`
> structures that represent the actual class names [7].
>
> To address this inconsistency, and particularly in light of
> the new
> encoding of module names described above, we propose to use
> consistent
> kinds of class-file structures for module and package names.
>
> Module names in a compiled module-declaration class file
> will be encoded
> as above and wrapped in tagged `CONSTANT_Module_info`
> structures:
>
> CONSTANT_Module_info {
> u1 tag; // == CONSTANT_Module == 19
> u2 name_index; // Index of a CONSTANT_Utf8_info
> }
>
> Package names in clas! s files will be encoded in the
> traditional internal
> form and wrapped in tagged `CONSTANT_Package_info` structures:
>
> CONSTANT_Package_info {
> u1 tag; // == CONSTANT_Package == 20
> u2 name_index; // Index of a CONSTANT_Utf8_info
> }
>
> Existing references in the class-file format to module and
> package names
> will be adjusted to refer to these new kinds of tagged
> structures.
>
>
> [1]
> http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameCharacters
> [2]
> http://docs.oracle.com/javase/specs/jls/se8/html/jls-6.html#jls-6.2
> [3]
> http://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.8
> [4] https://en.wikipedia.org/wiki/Standard_ML#Module_system
> [5]
> http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.2.1
> [6]
> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-November/000468.html
>
> [7]
> http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4.1
>
>
>
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
--
- DML
From stef at epardaud.fr Mon Dec 12 09:21:23 2016
From: stef at epardaud.fr (Stephane Epardaud)
Date: Mon, 12 Dec 2016 10:21:23 +0100
Subject: Proposal: #ModuleNameCharacters (revised)
In-Reply-To: <6c9bbb7f-133c-0600-d13a-1154098532de@redhat.com>
References: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
<16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
<6c9bbb7f-133c-0600-d13a-1154098532de@redhat.com>
Message-ID: <584E6C13.6070308@epardaud.fr>
On 09/12/16 23:05, David M. Lloyd wrote:
> So... essentially a custom module system has to generate binary
> descriptors? That's going to be a real pain.
Can't you use the ClassWriter API of the javac tool? That's
JDK-specific, not JRE, though.
From mark.reinhold at oracle.com Mon Dec 12 23:23:35 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 12 Dec 2016 15:23:35 -0800 (PST)
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <1de06198-930b-4b71-fe5d-1477d9cace80@redhat.com>
References: <20161207234117.1E5F1213F1@eggemoggin.niobe.net>
<1de06198-930b-4b71-fe5d-1477d9cace80@redhat.com>
Message-ID: <20161212232335.A9BFC24B27@eggemoggin.niobe.net>
2016/12/8 5:46:24 -0800, david.lloyd at redhat.com:
> On 12/07/2016 05:41 PM, mark.reinhold at oracle.com wrote:
>> ...
>>
>> Could you please describe here, in plain text, what further methods you
>> think you need? For IP clarity any material you want this EG to consider
>> must be posted to this list or to related OpenJDK infrastructure.
>>
>> It'd be fine to discuss specific code patches on jigsaw-dev, but again
>> you'd need to include it directly rather than point to GitHub. This is
>> a design issue, though, so it's probably best to discuss it here first.
>
> Sure. I've added the following methods to Layer.Controller:
>
> public Controller addPackage(Module source, String pn) { ... }
> public Controller addOpens(Module source, String pn, Module target) { ... }
> public Controller addOpensToAll(Module source, String pn) { ... }
> public Controller addOpensToAllUnnamed(Module source, String pn) { ... }
> public Controller addExports(Module source, String pn, Module target) {
> ... }
> public Controller addExportsToAll(Module source, String pn) { ... }
> public Controller addExportsToAllUnnamed(Module source, String pn) { ... }
> public Controller addUses(Module source, Class<?> service) { ... }
Can you explain exactly why you need all these methods?
I can see why you might need the qualified `addExports` method, akin
to the existing `addOpens` method, if you're doing some form of module
resolution on your own that's somehow taking named layers, or whatever,
into account.
The `add{Opens,Exports}ToAll` variants shouldn't be needed since you can
just include unqualified `open` and `exports` directives in the module
descriptor that you're going to build anyway. That has the additional
benefit of making the exports apparent to the JPMS resolver so that JPMS
modules can resolve against your modules, whereas invoking these methods
wouldn't do that.
The `addPackage` method is problematic, in part since it's quite slow
(at least in HotSpot, where we've optimized for the common case of the
set of packages in a module never changing). Can't you compute the set
of packages in a module at the time you create the descriptor?
- Mark
From mark.reinhold at oracle.com Mon Dec 12 23:24:35 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 12 Dec 2016 15:24:35 -0800 (PST)
Subject: Restrictions on service loading
In-Reply-To: <4d44823a-bf79-5ecf-281f-d9cf8777d341@redhat.com>
References: <4d44823a-bf79-5ecf-281f-d9cf8777d341@redhat.com>
Message-ID: <20161212232435.AE70924B2C@eggemoggin.niobe.net>
2016/12/9 6:59:57 -0800, david.lloyd at redhat.com:
> It doesn't seem clear whether this is a spec or implementation issue, so
> I'm posting here to start.
>
> The gist of the problem is, code running in a module cannot use
> ServiceLoader to find things that it hasn't been declared to "use" even
> if a target class loader is specified. This doesn't really make a lot
> of sense because in the case where you're specifying a class loader, you
> are pretty clearly establishing that you want to search the class
> loader, not a module space.
That's true prior to SE 9, though even if you specify a class loader
when trying to locate a service you could get a provider from some
other loader that's related to that loader via parent delegation.
Once modules are part of the picture then if you specify a class loader
when trying to locate a service you could, additionally, get a provider
from some other loader in the same non-boot layer as that loader, or in
the same non-boot layer as one of that loader's parents. The motivation
for this is to make the "right thing" happen in module-per-loader
scenarios, i.e., in layers where every module has its own class loader.
These details are laid out in the "Locating providers" section of the
`ServiceLoader` API specification [1].
> This can be worked around by adding a "uses" at run time, but it
> shouldn't be necessary for the explicit class loader case.
Service-use declarations have both benefits and, naturally, costs.
The benefit of static `uses` declarations is that they make it easier to
configure complex systems. The resolver will locate providers on the
module path for you, without any further instruction, and resolve them
recursively as needed. If you forget to include a `uses` declaration for
a service that you actually use, and you invoke a `ServiceLoader.load`
method to load it, then you get an immediate error that tells you that
you forgot the declaration, so that you can add it. This is preferable
to failing silently because no providers were discovered because you
forgot the `uses` declaration. It's also preferable to not supporting
`uses` declarations in the module sysem at all, in which case the only
way to configure providers would be to request them explicitly via
run-time options such as `--add-modules`, in the case of the boot layer,
or by specifying them explicitly when creating other layers via the API.
A cost of static `uses` declarations is that, sometimes, sophisticated
users of services must do a little bit of work to get past the above
check. If, in particular, one module uses `ServiceLoader` to locate
providers on behalf of a second module then the first module will have to
invoke the `Module::addUses` method, passing in the service type obtained
from the second, in order to enable the run-time `uses` check to succeed.
Having to invoke `addUses` can be an annoyance, but we expect very few
developers to have to do it. Many developers, and in particular many
non-expert developers, will benefit from the sanity checking that static
`uses` declarations enable, hence the cost is justified.
- Mark
[1] http://download.java.net/java/jigsaw/docs/api/java/util/ServiceLoader.html
From mark.reinhold at oracle.com Mon Dec 12 23:47:30 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Mon, 12 Dec 2016 15:47:30 -0800
Subject: Proposal: #ModuleNameCharacters (revised)
In-Reply-To: <dacc0d8c-e2a9-d1bd-d2d9-c52fa08c7083@redhat.com>
References: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
<16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
<6c9bbb7f-133c-0600-d13a-1154098532de@redhat.com>
<610BAFF3-B5AB-467D-A4F6-F51050274344@univ-mlv.fr>
<dacc0d8c-e2a9-d1bd-d2d9-c52fa08c7083@redhat.com>
Message-ID: <20161212154730.907442096@eggemoggin.niobe.net>
2016/12/10 7:24:18 -0800, david.lloyd at redhat.com:
> On 12/10/2016 01:50 AM, Remi Forax wrote:
>> On December 9, 2016 11:05:02 PM GMT+01:00, david.lloyd at redhat.com wrote:
>>> ...
>>>
>>> So... essentially a custom module system has to generate binary
>>> descriptors? That's going to be a real pain.
>> ...
>>
>> No, i hope it's more that ModuleDescriptor will be an interface.
>> So we can have our own module descriptor builder.
>
> Well, maybe we can build on this idea: what about making
> ModuleDescriptor.Builder non-final, so we can push validation logic into
> separate overridable methods? This way the default behavior is
> consistent, and it's much harder to accidentally bypass the logic, but
> it's also still easy to provide a customized scheme.
This is easier said than done, and I'm skeptical that the result would
be of use to more than an extraordinarily small number of developers.
I don't think it's unreasonable to ask the very, very few people who
will want unusual module names to roll their own binary descriptors.
- Mark
From forax at univ-mlv.fr Tue Dec 13 08:12:42 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Tue, 13 Dec 2016 09:12:42 +0100 (CET)
Subject: Proposal: #ModuleNameCharacters (revised)
In-Reply-To: <20161212154730.907442096@eggemoggin.niobe.net>
References: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
<16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
<6c9bbb7f-133c-0600-d13a-1154098532de@redhat.com>
<610BAFF3-B5AB-467D-A4F6-F51050274344@univ-mlv.fr>
<dacc0d8c-e2a9-d1bd-d2d9-c52fa08c7083@redhat.com>
<20161212154730.907442096@eggemoggin.niobe.net>
Message-ID: <1619498537.509143.1481616761921.JavaMail.zimbra@u-pem.fr>
I think part of the problem is that currently the builder enforces some rules and it should not,
the fact that a ModuleDescriptor is well formed or not should be checked by the configuration and not by the builder itself.
Checks that need to be done:
- no repetition, the ModuleDecriptor interface uses Set so this is enforced by design
- all referenced packages should be present in packages,
currently, the builder see that the other way around, i think it's a mistake.
This parts should be moved when resolving a configuation
- referenced modules exist, this part is already checked when resolving the configuration
- metadata associated with the module descriptor, version, main-class, hashes, all of these should be specified through a kind of Map API,
so build tool/language developers, JDK maintainers could add any metadata when they want.
Actions:
- the builder should only build a descriptor and not check if it is well formed or not
- when resolving the configuration, additional checks should be done to verify that a module descriptor is well formed
- add a way to specify additional metadata to the module descriptor.
cheers,
R?mi
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: "David M. Lloyd" <david.lloyd at redhat.com>
> Cc: "Remi Forax" <forax at univ-mlv.fr>, jpms-spec-experts at openjdk.java.net
> Envoy?: Mardi 13 D?cembre 2016 00:47:30
> Objet: Re: Proposal: #ModuleNameCharacters (revised)
> 2016/12/10 7:24:18 -0800, david.lloyd at redhat.com:
>> On 12/10/2016 01:50 AM, Remi Forax wrote:
>>> On December 9, 2016 11:05:02 PM GMT+01:00, david.lloyd at redhat.com wrote:
>>>> ...
>>>>
>>>> So... essentially a custom module system has to generate binary
>>>> descriptors? That's going to be a real pain.
>>> ...
>>>
>>> No, i hope it's more that ModuleDescriptor will be an interface.
>>> So we can have our own module descriptor builder.
>>
>> Well, maybe we can build on this idea: what about making
>> ModuleDescriptor.Builder non-final, so we can push validation logic into
>> separate overridable methods? This way the default behavior is
>> consistent, and it's much harder to accidentally bypass the logic, but
>> it's also still easy to provide a customized scheme.
>
> This is easier said than done, and I'm skeptical that the result would
> be of use to more than an extraordinarily small number of developers.
> I don't think it's unreasonable to ask the very, very few people who
> will want unusual module names to roll their own binary descriptors.
>
> - Mark
From david.lloyd at redhat.com Tue Dec 13 14:02:17 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Tue, 13 Dec 2016 08:02:17 -0600
Subject: Proposal: #ModuleNameCharacters (revised)
In-Reply-To: <1619498537.509143.1481616761921.JavaMail.zimbra@u-pem.fr>
References: <20161209214546.D23A721F6C@eggemoggin.niobe.net>
<16e76b92-b6c1-6ac0-0e18-14f888dcf35c@redhat.com>
<6c9bbb7f-133c-0600-d13a-1154098532de@redhat.com>
<610BAFF3-B5AB-467D-A4F6-F51050274344@univ-mlv.fr>
<dacc0d8c-e2a9-d1bd-d2d9-c52fa08c7083@redhat.com>
<20161212154730.907442096@eggemoggin.niobe.net>
<1619498537.509143.1481616761921.JavaMail.zimbra@u-pem.fr>
Message-ID: <e063a4b6-19bc-5f40-e73a-7e7cacb44197@redhat.com>
This would be much better from my perspective, assuming that we let the
Layer (or something) do the validation so that it can be overridden...
Note that this may also have application for #VersionSyntax.
On 12/13/2016 02:12 AM, forax at univ-mlv.fr wrote:
> I think part of the problem is that currently the builder enforces some rules and it should not,
> the fact that a ModuleDescriptor is well formed or not should be checked by the configuration and not by the builder itself.
>
> Checks that need to be done:
> - no repetition, the ModuleDecriptor interface uses Set so this is enforced by design
> - all referenced packages should be present in packages,
> currently, the builder see that the other way around, i think it's a mistake.
> This parts should be moved when resolving a configuation
> - referenced modules exist, this part is already checked when resolving the configuration
> - metadata associated with the module descriptor, version, main-class, hashes, all of these should be specified through a kind of Map API,
> so build tool/language developers, JDK maintainers could add any metadata when they want.
>
> Actions:
> - the builder should only build a descriptor and not check if it is well formed or not
> - when resolving the configuration, additional checks should be done to verify that a module descriptor is well formed
> - add a way to specify additional metadata to the module descriptor.
>
> cheers,
> R?mi
>
> ----- Mail original -----
>> De: "mark reinhold" <mark.reinhold at oracle.com>
>> ?: "David M. Lloyd" <david.lloyd at redhat.com>
>> Cc: "Remi Forax" <forax at univ-mlv.fr>, jpms-spec-experts at openjdk.java.net
>> Envoy?: Mardi 13 D?cembre 2016 00:47:30
>> Objet: Re: Proposal: #ModuleNameCharacters (revised)
>
>> 2016/12/10 7:24:18 -0800, david.lloyd at redhat.com:
>>> On 12/10/2016 01:50 AM, Remi Forax wrote:
>>>> On December 9, 2016 11:05:02 PM GMT+01:00, david.lloyd at redhat.com wrote:
>>>>> ...
>>>>>
>>>>> So... essentially a custom module system has to generate binary
>>>>> descriptors? That's going to be a real pain.
>>>> ...
>>>>
>>>> No, i hope it's more that ModuleDescriptor will be an interface.
>>>> So we can have our own module descriptor builder.
>>>
>>> Well, maybe we can build on this idea: what about making
>>> ModuleDescriptor.Builder non-final, so we can push validation logic into
>>> separate overridable methods? This way the default behavior is
>>> consistent, and it's much harder to accidentally bypass the logic, but
>>> it's also still easy to provide a customized scheme.
>>
>> This is easier said than done, and I'm skeptical that the result would
>> be of use to more than an extraordinarily small number of developers.
>> I don't think it's unreasonable to ask the very, very few people who
>> will want unusual module names to roll their own binary descriptors.
>>
>> - Mark
--
- DML
From david.lloyd at redhat.com Tue Dec 13 14:47:53 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Tue, 13 Dec 2016 08:47:53 -0600
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <20161212232335.A9BFC24B27@eggemoggin.niobe.net>
References: <20161207234117.1E5F1213F1@eggemoggin.niobe.net>
<1de06198-930b-4b71-fe5d-1477d9cace80@redhat.com>
<20161212232335.A9BFC24B27@eggemoggin.niobe.net>
Message-ID: <9b66aa52-877a-e4b2-1458-9ff47dfb690b@redhat.com>
On 12/12/2016 05:23 PM, mark.reinhold at oracle.com wrote:
> 2016/12/8 5:46:24 -0800, david.lloyd at redhat.com:
>> On 12/07/2016 05:41 PM, mark.reinhold at oracle.com wrote:
>>> ...
>>>
>>> Could you please describe here, in plain text, what further methods you
>>> think you need? For IP clarity any material you want this EG to consider
>>> must be posted to this list or to related OpenJDK infrastructure.
>>>
>>> It'd be fine to discuss specific code patches on jigsaw-dev, but again
>>> you'd need to include it directly rather than point to GitHub. This is
>>> a design issue, though, so it's probably best to discuss it here first.
>>
>> Sure. I've added the following methods to Layer.Controller:
>>
>> public Controller addPackage(Module source, String pn) { ... }
>> public Controller addOpens(Module source, String pn, Module target) { ... }
>> public Controller addOpensToAll(Module source, String pn) { ... }
>> public Controller addOpensToAllUnnamed(Module source, String pn) { ... }
>> public Controller addExports(Module source, String pn, Module target) {
>> ... }
>> public Controller addExportsToAll(Module source, String pn) { ... }
>> public Controller addExportsToAllUnnamed(Module source, String pn) { ... }
>> public Controller addUses(Module source, Class<?> service) { ... }
>
> Can you explain exactly why you need all these methods?
>
> I can see why you might need the qualified `addExports` method, akin
> to the existing `addOpens` method, if you're doing some form of module
> resolution on your own that's somehow taking named layers, or whatever,
> into account.
Yeah we're assembling the module structure in a multi-stage lazy
resolution process, thus we don't know exactly what we're opening or
exporting until after all contents and dependencies are defined (and
this can change over time).
> The `add{Opens,Exports}ToAll` variants shouldn't be needed since you can
> just include unqualified `open` and `exports` directives in the module
> descriptor that you're going to build anyway. That has the additional
> benefit of making the exports apparent to the JPMS resolver so that JPMS
> modules can resolve against your modules, whereas invoking these methods
> wouldn't do that.
The problem is that when I first build the module, I don't have all the
dependency information available, and also some of the dependencies will
include modules which are not visible to the layer (in fact right now
I'm putting each module into a separate layer), and some of the
dependencies are on non-Jigsaw entities which I also can't know initially.
> The `addPackage` method is problematic, in part since it's quite slow
> (at least in HotSpot, where we've optimized for the common case of the
> set of packages in a module never changing). Can't you compute the set
> of packages in a module at the time you create the descriptor?
Two things prevent this: firstly, we don't have the module contents
until after we've constructed the module class loader, which requires
the layer controller in order to process dependencies, so as of now,
adding contents depends on having the module established. Secondly, it
is allowed to add content to a JBoss module after it has been
established. The first might be fixable, but I can't think of a way
around the second.
--
- DML
From blackdrag at gmx.org Tue Dec 13 17:38:52 2016
From: blackdrag at gmx.org (Jochen Theodorou)
Date: Tue, 13 Dec 2016 18:38:52 +0100
Subject: Proposal: #NonHierarchicalLayers
In-Reply-To: <20161212232335.A9BFC24B27@eggemoggin.niobe.net>
References: <20161207234117.1E5F1213F1@eggemoggin.niobe.net>
<1de06198-930b-4b71-fe5d-1477d9cace80@redhat.com>
<20161212232335.A9BFC24B27@eggemoggin.niobe.net>
Message-ID: <568a94e0-afc8-09d2-be92-7cb06cd62de1@gmx.org>
On 13.12.2016 00:23, mark.reinhold at oracle.com wrote:
[...]
> The `add{Opens,Exports}ToAll` variants shouldn't be needed since you can
> just include unqualified `open` and `exports` directives in the module
> descriptor that you're going to build anyway. That has the additional
> benefit of making the exports apparent to the JPMS resolver so that JPMS
> modules can resolve against your modules, whereas invoking these methods
> wouldn't do that.
what if I need to do those things after the module had been loaded? And
not sure if you did mean the runtime case, but I do not want to produce
a module descriptor class file at runtime.
> The `addPackage` method is problematic, in part since it's quite slow
> (at least in HotSpot, where we've optimized for the common case of the
> set of packages in a module never changing). Can't you compute the set
> of packages in a module at the time you create the descriptor?
which means I cannot add a class to a module in a "new" package at runtime?
bye Jochen
From mark.reinhold at oracle.com Wed Dec 14 19:20:48 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 14 Dec 2016 11:20:48 -0800
Subject: Resignation from group
In-Reply-To: <BC58612F-C75D-40D3-AFF2-31F08F8ADF43@takari.io>
References: <BC58612F-C75D-40D3-AFF2-31F08F8ADF43@takari.io>
Message-ID: <20161214112048.953801172@eggemoggin.niobe.net>
2016/12/5 15:31:16 -0800, Jason van Zyl <jason at takari.io>:
> I?d like to resign from the group, but I suggest allowing Robert
> Scholte from the Maven project to replace me.
Thanks, Jason. I look forward to welcoming Robert to this EG
once the necessary JCP paperwork is completed.
- Mark
From forax at univ-mlv.fr Thu Dec 15 17:20:51 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Thu, 15 Dec 2016 18:20:51 +0100 (CET)
Subject: Proposal: #VersionedDependences
In-Reply-To: <20161209214646.D915621F70@eggemoggin.niobe.net>
References: <20161209214646.D915621F70@eggemoggin.niobe.net>
Message-ID: <1717092100.1687029.1481822451766.JavaMail.zimbra@u-pem.fr>
I do not like this proposal for several (good) reasons.
The issue asks to be able to store a version strings or a constraints.
It can be interpreted as two different things.
- the configuration used by example by Maven which uses constraints that will be resolved,
- the other is the effectively resolved versions which is what this proposal do.
In my opinion, storing the former info maybe more interesting for a language (the actual configuration) than storing the later.
We already agree that we support annotations (and obviously classfile attributes) so any languages are free to store this kind of information in it's own annotation/attribute, which is more flexible.
If we still want to store the version of each requires, i think it's better to store it in a side attribute (an array of versions should be enough) than in the Module attribute, it's more aligned with the way we encode ModuleVersion currently.
The flag --module-version in javac is useless:
- it's an optional feature that few will use, so runtimes can not use it reliably, so it's as useful as -parameters,
- you often need to compile several modules together, if a module A requires a module B and the module B uses a service from module A, but --module-version can only specify one version for all the modules compiled together.
To summarize, i believe that as specified, this feature not useful.
regards,
R?mi
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: jpms-spec-experts at openjdk.java.net
> Envoy?: Vendredi 9 D?cembre 2016 22:46:46
> Objet: Proposal: #VersionedDependences
> Issue summary
> -------------
>
> #VersionedDependences --- Consider allowing specific version strings,
> or perhaps version constraints, as an optional element of `requires`
> clauses in module declarations. Failing that, consider allowing
> specific version strings, or perhaps version constraints, to be added
> to the dependences recorded in a compiled module descriptor; this
> would, e.g., allow a compiler or build system to record the versions
> of the modules against which a particular module was compiled, for
> use by other tools. In either case, if such version information is
> merely informative then it will still honor the version selection
> non-requirement; if such version information is interpreted by the
> module system then that requirement may come into question. [1]
>
> Proposal
> --------
>
> When compiling a module that depends on some other modules, record the
> version strings of those modules, if available, in the resulting module
> descriptor. These version strings will not be interpreted, in any phase;
> they are provided solely for diagnostic purposes and for use by tools,
> frameworks, and containers that assist in the selection and validation
> of module versions. This proposal, therefore, continues to honor the
> version-selection non-requirement [2].
>
> The specific changes to implement this feature include:
>
> - Extend module descriptors to allow the inclusion of version strings
> in the `requires` table of the `Module` attribute by introducing a
> `u2 requires_version_index` field after the `requires_flags` field
> of each entry of that table. The value of this new field is either
> zero, to indicate that no version string was recorded, or else the
> constant-pool index of a `CONSTANT_Utf8_info` structure whose value
> is the version string.
>
> - Extend the `java.lang.module.ModuleDescriptor.Requires` API with a
> single new method:
>
> Optional<ModuleDescriptor.Version> compiledVersion();
>
> This method will return an empty `Optional` object if no version
> string was recorded for the corresponding dependence, or else an
> `Optional` that contains the recorded version string.
>
> - Recommend that Java language compilers provide a means by which a
> version string can be indicated for any modules being compiled, and
> that they record such strings in the resulting module descriptors.
> (The RI's `javac` compiler will support this via a new command-line
> option, `--module-version`.)
>
> - If a compiler can record the version string of the modules being
> compiled in the resulting compiled module descriptors then it would
> be simpler for those version strings to be placed in the `Module`
> attribute rather than the ancillary `ModuleVersion` attribute. We'll
> therefore drop the latter and add a `u2 module_version_index` field
> to the `Module` attribute, immediately following the `module_flags`
> field.
>
> Now that compile-time versions can be recorded in module descriptors
> there is even less need to tolerate version information in module names,
> a bad practice that we'd like to discourage at the outset. We therefore
> further propose to:
>
> - Revise the accepted proposal for #VersionsInModuleNames [3] to state
> that a module name appearing anywhere in a source-form module
> declaration must both start and end with "Java letters" [4]. This
> includes the name of the module being declared and also the names of
> other modules mentioned in `requires` directives or in qualified
> `exports` or `opens` directives. This restriction will be enforced
> by the `ModuleDescriptor.Builder` API, which is intended to be
> consistent with the source language. The `ModuleDescriptor` API
> itself will continue to be able to read class files that contain
> module names that violate this constraint.
>
>
> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#VersionedDependences
> [2] http://openjdk.java.net/projects/jigsaw/spec/reqs/#version-selection
> [3] http://openjdk.java.net/projects/jigsaw/spec/issues/#VersionsInModuleNames
> [4] http://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-JavaLetter
From stef at epardaud.fr Fri Dec 16 10:55:23 2016
From: stef at epardaud.fr (Stephane Epardaud)
Date: Fri, 16 Dec 2016 11:55:23 +0100
Subject: Proposal: #VersionedDependences
In-Reply-To: <1717092100.1687029.1481822451766.JavaMail.zimbra@u-pem.fr>
References: <20161209214646.D915621F70@eggemoggin.niobe.net>
<1717092100.1687029.1481822451766.JavaMail.zimbra@u-pem.fr>
Message-ID: <5853C81B.9010102@epardaud.fr>
On 15/12/16 18:20, Remi Forax wrote:
> We already agree that we support annotations (and obviously classfile attributes) so any languages are free to store this kind of information in it's own annotation/attribute, which is more flexible.
I don't think annotations on module imports were agreed to, yet? Were they?
> - it's an optional feature that few will use, so runtimes can not use it reliably, so it's as useful as -parameters,
I still regret that this was not the default :(
From mark.reinhold at oracle.com Wed Dec 21 21:12:16 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 21 Dec 2016 13:12:16 -0800 (PST)
Subject: Draft JPMS EDR specification
Message-ID: <20161221211216.71D3D27C41@eggemoggin.niobe.net>
I've posted a draft EDR specification for your review, here:
http://cr.openjdk.java.net/~mr/jigsaw/spec/
This document gathers the collected changes proposed for the JLS, the
JVMS, the Java SE API, and various low-level interfaces (JNI, JVM TI,
and JDWP).
The intent of the EDR is to provide a comprehensive snapshot of our work
to JCP participants who do not follow our activity closely. It by no
means implies that we are finished -- we'll continue to work through the
issues already identified, and on issues raised by feedback yet to come.
I plan to submit this to the PMO as the Early Draft Review Specification
on 10 January. Please let me know by 17:00 UTC on Monday, 9 January if
you think any changes are required, or if you need more time for review.
- Mark
From forax at univ-mlv.fr Sat Dec 24 12:53:22 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Sat, 24 Dec 2016 13:53:22 +0100 (CET)
Subject: Draft JPMS EDR specification
In-Reply-To: <20161221211216.71D3D27C41@eggemoggin.niobe.net>
References: <20161221211216.71D3D27C41@eggemoggin.niobe.net>
Message-ID: <854291806.1578063.1482584002002.JavaMail.zimbra@u-pem.fr>
Minor nit,
in the VM spec part, section 2.1, the attribute ModuleVersion is mentioned while it has disappeared
(the module version is now a field of the Module attribute).
I will modify ASM next week to be in sync with the spec, i do not expect any problems.
I still think that encoding a version for the requires inside the Module attribute is now a good idea (cf my previous message).
Happy Christmas everyone,
R?mi
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: jpms-spec-experts at openjdk.java.net
> Envoy?: Mercredi 21 D?cembre 2016 22:12:16
> Objet: Draft JPMS EDR specification
> I've posted a draft EDR specification for your review, here:
>
> http://cr.openjdk.java.net/~mr/jigsaw/spec/
>
> This document gathers the collected changes proposed for the JLS, the
> JVMS, the Java SE API, and various low-level interfaces (JNI, JVM TI,
> and JDWP).
>
> The intent of the EDR is to provide a comprehensive snapshot of our work
> to JCP participants who do not follow our activity closely. It by no
> means implies that we are finished -- we'll continue to work through the
> issues already identified, and on issues raised by feedback yet to come.
>
> I plan to submit this to the PMO as the Early Draft Review Specification
> on 10 January. Please let me know by 17:00 UTC on Monday, 9 January if
> you think any changes are required, or if you need more time for review.
>
> - Mark
From forax at univ-mlv.fr Sat Dec 24 13:34:02 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Sat, 24 Dec 2016 14:34:02 +0100 (CET)
Subject: Draft JPMS EDR specification
In-Reply-To: <854291806.1578063.1482584002002.JavaMail.zimbra@u-pem.fr>
References: <20161221211216.71D3D27C41@eggemoggin.niobe.net>
<854291806.1578063.1482584002002.JavaMail.zimbra@u-pem.fr>
Message-ID: <68826354.1579666.1482586442884.JavaMail.zimbra@u-pem.fr>
s/now/not
R?mi
----- Mail original -----
> De: "Remi Forax" <forax at univ-mlv.fr>
> ?: "mark reinhold" <mark.reinhold at oracle.com>
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Samedi 24 D?cembre 2016 13:53:22
> Objet: Re: Draft JPMS EDR specification
> Minor nit,
> in the VM spec part, section 2.1, the attribute ModuleVersion is mentioned while
> it has disappeared
> (the module version is now a field of the Module attribute).
>
> I will modify ASM next week to be in sync with the spec, i do not expect any
> problems.
>
> I still think that encoding a version for the requires inside the Module
> attribute is now a good idea (cf my previous message).
>
> Happy Christmas everyone,
> R?mi
>
> ----- Mail original -----
>> De: "mark reinhold" <mark.reinhold at oracle.com>
>> ?: jpms-spec-experts at openjdk.java.net
>> Envoy?: Mercredi 21 D?cembre 2016 22:12:16
>> Objet: Draft JPMS EDR specification
>
>> I've posted a draft EDR specification for your review, here:
>>
>> http://cr.openjdk.java.net/~mr/jigsaw/spec/
>>
>> This document gathers the collected changes proposed for the JLS, the
>> JVMS, the Java SE API, and various low-level interfaces (JNI, JVM TI,
>> and JDWP).
>>
>> The intent of the EDR is to provide a comprehensive snapshot of our work
>> to JCP participants who do not follow our activity closely. It by no
>> means implies that we are finished -- we'll continue to work through the
>> issues already identified, and on issues raised by feedback yet to come.
>>
>> I plan to submit this to the PMO as the Early Draft Review Specification
>> on 10 January. Please let me know by 17:00 UTC on Monday, 9 January if
>> you think any changes are required, or if you need more time for review.
>>
> > - Mark