From mark.reinhold at oracle.com Fri Jul 1 00:36:20 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu, 30 Jun 2016 17:36:20 -0700
Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources
In-Reply-To: <5773CB63.6030308@redhat.com>
References: <20160628212015.99A06B59CC@eggemoggin.niobe.net>
<5773CB63.6030308@redhat.com>
Message-ID: <20160630173620.105142642eggemoggin.niobe.net>
2016/6/29 6:21:39 -0700, david.lloyd at redhat.com:
> On 06/28/2016 04:20 PM, Mark Reinhold wrote:
>> ...
>>
>> Make the following changes to the various `getResource*` methods:
>>
>> - The `ClassLoader::getResource*` methods will first delegate to the
>> loader's parent, if any, and will then search for resources in the
>> named modules defined to the loader, and will then search in the
>> loader's unnamed module, typically defined by a class path. In the
>> case of the `getResources` method the order of the resources in the
>> returned enumeration will be unspecified, except that the values of
>> `cl.getResources(name).nextElement()` and `cl.getResource(name)`
>> will be equal.
>
> I am really not a fan of unspecified resource order, especially when you
> can easily derive a stable order from the graph topology.
>
> Resources however should always load from the child-most loader first.
The `getResource*` methods in java.lang.ClassLoader have delegated to
the parent first since JDK 1.2, so for compatibility they'll have to
keep doing that. Custom loaders can, of course, continue to do whatever
they want.
> This isn't a problem in Jigsaw for classes because packages cannot be
> duplicated between modules (in a visible manner), but it becomes a
> problem with resources, especially if/when run-time cycles are
> permitted: you can run into an awkward situation where two mutually
> dependent modules each may see the other's resource "first" before their
> own, which is almost always undesirable.
This suggests that deriving a resource order from the graph topology
is maybe not as easy as you think. Is resource order, within a single
class loader, really so important that we should go to the trouble of
analyzing module relationships within that loader? If a container
application really needs fine-grained control over resource order then
it could always create a separate class loader for every module.
>> - The `Class::getResource*` methods will only search for resources in
>> the module that defines the class. In the case of a class in an
>> unnamed module this will typically result in searching the class
>> path of the corresponding class loader.
>
> Does this also apply to automatic modules?
Yes, of course.
>> The `java.lang.reflect.Module::getResourceAsStream` method will remain,
>> so that it's possible to look up a resource in a module without having a
>> reference to a class defined in that module. It will always be the case
>> that, for a given a class `c`, `c.getResourceAsStream(name)` will be
>> equivalent to `c.getModule().getResourceAsStream(name)`.
>
> I believe that in the former case, the "name" was historically relative
> to the location of the class. Is that behavior changing?
Good catch -- that's an error in my description.
>> Rationale
>> ---------
>>
>> The encapsulation of resources was intended to be part of the overall
>> encapsulation story, on the view that a module's resources are just
>> internal implementation details that need not, and should not, be
>> available to code in other modules. Allowing external code to depend
>> upon a module's internal resources is just as problematic as allowing
>> such code to depend upon internal APIs.
>>
>> The reality of existing code today, however, is rather different, with
>> two common use cases:
>>
>> - Resources are often used intentionally to convey information to
>> external code, e.g., as a way to publish configuration files such as
>> `persistence.xml`. This sort of thing should, ideally, be done via
>> services [4][5], but that is not the current practice and it will
>> take years to migrate to it.
>
> In many cases services are too limiting. It is useful for users to have
> a more powerful, general facility available that can use the same basic
> mechanism as services, but for arbitrary resources. For example, one
> case that came up quite recently (on Jigsaw-dev I think) was that it is
> useful to acquire not the service instance, but the class name, so that
> it could be constructed in various ways. Services as they are now
> require another front-end class to encapsulate this kind of variation.
> Also it is not uncommon for a situation to arise where many classes must
> be produced for variant services which all have the same basic behavior
> but differ only in details, which is awkward as well as it can result in
> large quantities of similar classes.
>
> It would be nice if service loading and resource loading continued to
> follow a similar set of rules. When Jigsaw switched ServiceLoader from
> a convenience API over resources to a separate concept, it lost the
> concept of using resources *as* an API component or participant, which
> is a powerful feature. This change will restore that to some degree,
> but it is still strange that loading a META-INF/services/com.foo.Service
> file will have different search semantics than using ServiceLoader.
> Basically we are taking this capability from users unless they do it the
> way that Jigsaw wants you to: via ServiceLoader, and no other way; it
> almost seems like Jigsaw is attempting to take on the characteristics of
> a container (not a very powerful one though because it only supports
> linking via services (a pretty thin concept when you think about it),
> not by (for example) annotations or anything else).
>
> Anyway I hope you can extract some kind of cogency from all that.
Um, no, not really.
At any rate this proposal is about resource loading, not services.
If you have a specific issue to raise or proposal to make about
services then please do so in a separate thread.
- Mark
From mark.reinhold at oracle.com Fri Jul 1 00:45:41 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu, 30 Jun 2016 17:45:41 -0700
Subject: Proposal: #BootstrapClassLoaderSearchInJVMTI
In-Reply-To: <OF6F3BBEB3.9A590156-ON80257FE2.004DA707-80257FE2.004DF799@notes.na.collabserv.com>
References: <20160628212215.9E7D9B59D4@eggemoggin.niobe.net>
<OF0A511979.B5B99FBD-ON80257FE1.0038EFAB-80257FE1.003B224E@notes.na.collabserv.com>
<20160629163100.841006773eggemoggin.niobe.net>
<OF6F3BBEB3.9A590156-ON80257FE2.004DA707-80257FE2.004DF799@notes.na.collabserv.com>
Message-ID: <20160630174541.601998191eggemoggin.niobe.net>
2016/6/30 7:11:36 -0700, tim_ellison at uk.ibm.com:
> mark.reinhold at oracle.com wrote on 30/06/2016 00:31:00:
>> 2016/6/29 3:45:53 -0700, tim_ellison at uk.ibm.com:
>>> ...
>>>
>>> What would be the effect calling this function on a modular runtime when
>>> passed a directory search path segment? a modular JAR? a regular JAR?
>>
>> It will work just as it does today.
>>
>> If the bootstrap class loader cannot locate a suitable class file within
>> the run-time system then it will consult the path defined by calls to
>> this function. Each element of the path can be either a JAR file or an
>> "exploded" directory tree corresponding to a package hierarchy. Whether
>> a JAR file is a modular JAR is of no import -- any module descriptor
>> within it will be ignored.
>
> Just to check I fully understand, this function will affect the search path
> of the unnamed module associated with the bootstrap class loader, correct?
Correct.
> If that's so then I'm happy with the proposed solution.
Good!
- Mark
From nipa at codefx.org Fri Jul 1 05:45:43 2016
From: nipa at codefx.org (Nicolai Parlog)
Date: Fri, 1 Jul 2016 07:45:43 +0200
Subject: Proposal: #CompileTimeDependences: `requires static`
In-Reply-To: <20160628163150.929695476eggemoggin.niobe.net>
References: <20160628211715.84610B59BF@eggemoggin.niobe.net>
<1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr>
<20160628163150.929695476eggemoggin.niobe.net>
Message-ID: <57760387.9040405@codefx.org>
Hi!
Because the naming discussion is more lively over here, I'll crosspost:
> Having the symmetric pair "static vs dynamic" looks nice but it
> takes a lot of words to explain why exactly these terms apply here.
> We seem to be looking for qualifiers for "requires" and "exports".
> Maybe just using other terms solves the problem?
>
> requires & consumes exports & contains
So it would look as follows:
module com.xyz {
requires guava;
requires joda;
consumes hibernate;
exports com.xyz.a;
exports com.xyz.b;
contains com.xyz.c;
}
so long ... Nicolai
On 29.06.2016 01:31, mark.reinhold at oracle.com wrote:
> 2016/6/28 15:50:52 -0700, Remi Forax <forax at univ-mlv.fr>:
>> Apart the fact that 'static' should be spelled 'optional', there
>> is no reason to reuse static as it doesn't convey the semantics
>> we want, i.e. optional at runtime, i fully agree with this
>> proposal.
>
> In this context `static` is intended to mean "at compile time" or,
> equivalently, before the more "dynamic" phases of run time and
> link time (the latter of which is more like run time than compile
> time).
>
> I agree that `static` is potentially confusing since its meaning
> here is different from what `static` means when used on a member of
> a class.
>
> It does, however, fit nicely with its (nearly) dual directive,
> `exports dynamic`, proposed for
> #ReflectiveAccessToNonExportedTypes.
>
> I think `optional` is a non-starter, since `requires optional`
> reads as an oxymoron, and it's optional at run time but mandatory
> at compile time, so in what sense is it, really, "optional"?
>
> Suggestions of other alternatives are welcome ...
>
> - Mark
>
--
PGP Key:
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
Web:
http://codefx.org
a blog about software development
http://do-foss.de
Free and Open Source Software for the City of Dortmund
Twitter:
https://twitter.com/nipafx
From nipa at codefx.org Fri Jul 1 07:27:44 2016
From: nipa at codefx.org (Nicolai Parlog)
Date: Fri, 1 Jul 2016 09:27:44 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
Message-ID: <57761B70.10403@codefx.org>
Hi!
> I do not think that shielding internals is a problem with regards
> to reflection.
I think it is. I've not been emerged in the community for very long so
I know only two examples (but still): Unsafe and JUnit 4. In both
cases internals were accessed via reflection. And in both cases
dependencies became so ubiquitous that changing any details became
impossible for the maintainers.
I think the outcry when Unsafe was planned to go away and the fact
that JUnit 4 had to be rewritten because evolution was effectively
prevented by unwanted dependencies make my point here.
> I do neither think that reflection only interests internals.
I agree. Not "only" but "all too often", though.
> For example, maybe you use Hibernate for your persistance layer
> without exposing those DTOs from your module.
Why would I do that?! This is exactly where reflection got us used to
sloppy architecture. If a tool uses my DTOs to implement my
persistence layer, then it looks like those DTOs are part of my API.
Maybe not public to everyone but qualified and dynamic exports give me
tools to reduce visibility.
> I do not care about how Hibernate works as long as it does work.
> Almost any library uses reflection and I would not want to dig
> into implementation details to find out how to set up my dynamic
> exports.
Maybe I'm missing something here but to me it looks like "export to a
module whatever that module is supposed to interact with" is a very
sensible default that should cover a vast majority of uses cases, no?
I have DTOs && Hibernate needs DTOs -> export DTOs to Hibernate.
Why would I look at the implementation? And if there were non-obvious
scenarios they should be properly documented so I can react accordingly.
> As a library implementor, you aim for the biggest convenience.
Maybe we should aim for clarity and well-defined interactions instead.
so long ... Nicolai
On 01.07.2016 00:20, Rafael Winterhalter wrote:
> I disagree on two ends here.
>
> I do not think that shielding internals is a problem with regards
> to reflection. You want to shield internals in order to not break
> dependants by prohibiting their existance. With reflection, you do
> however typically react to the actual implementation by discovering
> it at runtime. If your use pattern was fixed, you would always
> require a static dependency over reflection in the first place. I
> do neither think that reflection only interests internals. For
> example, maybe you use Hibernate for your persistance layer without
> exposing those DTOs from your module. The problem is that you
> invert responsibilities. I do not care about how Hibernate works as
> long as it does work. Almost any library uses reflection and I
> would not want to dig into implementation details to find out how
> to set up my dynamic exports. I want to require a module and
> hopefully the compiler tells me about anything that's wrong.
> Reflection access errors would only be an awkward side effect that
> can be avoided by not applying modules alltogether. I think this
> limitation would therefore hurt Java 9 adoption.
>
> The second problem I have is the issue of attempting to fix an old
> regret. From the perspective of Hibernate, it would be easiest to
> not attempt modules as their introduction would require its users
> to apply additional configuration. As a library implementor, you
> aim for the biggest convenience. Adding this restriction neither
> adds additional security where a security manager should be used. I
> again think this would hurt adoption.
>
> I agree with Stephen's argument even though I do not think that
> forbidding dynamic access would be a feature anybody would use so
> it should not be added. Rather, something should be integrated in
> the security manager, if this is considered necessary.
>
> Cheers, Rafael
>
> Am 30.06.2016 11:37 nachm. schrieb "Nicolai Parlog"
> <nipa at codefx.org>:
>>
>> Hi!
>>
>>> In my opinion, reflection should not apply any module-boundary
>>> checks.
>>
>> Maybe this list is not a representative selection of Java
>> developers, maybe I'm just the only one thinking this but, to
>> paraphrase:
>>
>> Reflection over internals must die in a fire!
>>
>> Yes, yes, tools to great things, we need to see everything, ...
>> Still, I think that modules need a safe space that they can use
>> without anybody intruding. Maybe I'm a hardliner but I already
>> view the command line parameters with suspicion.
>>
>> In that light, I like the "dynamic" proposal.
>>
>> Not the name, though. Having the symmetric pair "static vs
>> dynamic" looks nice but it takes a lot of words to explain why
>> exactly these terms apply here. We seem to be looking for
>> qualifiers for "requires" and "exports". Maybe just using other
>> terms solves the problem?
>>
>> requires & consumes exports & contains
>>
>> so long ... Nicolai
>>
>>
>>
>> On 30.06.2016 01:23, Rafael Winterhalter wrote:
>>> I also agree here that the use of reflection should not require
>>> any form of explicit permission by the user of a library. In a
>>> way, this ties an implementation detail (reflection in this
>>> case) to the usage of the library. The library cannot work
>>> without the explicit configuration by a user and this only adds
>>> boilerplate to using Hibernate.
>>>
>>> Unfortunately, many Java developers still do not know much
>>> about reflection or about the inner workings of many libraries
>>> in the market. If a user needed to explicitly export packages,
>>> this might just lead to people "over exporting" their packages
>>> just to "make Hibernate work".
>>>
>>> In my opinion, reflection should not apply any module-boundary
>>> checks.
>>>
>>> Best regards, Rafael
>>>
>>> 2016-06-29 20:13 GMT+02:00 Paul Benedict
>>> <pbenedict at apache.org>:
>>>
>>>> Replying to "observers" list...
>>>>
>>>> I think the main problem here is that core reflection has
>>>> become too restrictive. Reflection should be "magical" and
>>>> have no knowledge of any kind of module and/or visibility
>>>> boundaries. If you really want to control what can be
>>>> reflected upon, that should be done via the Policy file of
>>>> the Security Manager.
>>>>
>>>> This proposal is trying to hack out-of-the-box created by
>>>> too strict of a design. It is very awkward to denote which
>>>> packages must be reflectable ahead of time --- how can you
>>>> figure that out?? The consumers who are doing the reflection
>>>> are innumerable. I'd go as far to say it's it's impossible to
>>>> devise a preemptive comprehensive strategy that may satisfy
>>>> the kinds of tooling in the wild.
>>>>
>>>> It's best just to open up reflection so it can continue
>>>> doing what it always has done well: let me see anything I
>>>> want unless the Security Manager says no.
>>>>
>>>> Cheers, Paul
>>>>
>>>> On Wed, Jun 29, 2016 at 3:51 AM, Remi Forax
>>>> <forax at univ-mlv.fr> wrote:
>>>>
>>>>>
>>>>>
>>>>> ----- Mail original -----
>>>>>> De: "Mark Reinhold" <mark.reinhold at oracle.com> ?:
>>>>>> jpms-spec-experts at openjdk.java.net Envoy?: Mardi 28 Juin
>>>>>> 2016 23:18:15 Objet: Proposal:
>>>>>> #ReflectiveAccessToNonExportedTypes: `exports dynamic`
>>>>>>
>>>>>> Issue summary -------------
>>>>>>
>>>>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of
>>>>>> framework libraries require reflective access to members
>>>>>> of the non-exported types of other modules; examples
>>>>>> include dependency injection
>>>> (Guice),
>>>>>> persistence (JPA), debugging tools, code-automation
>>>>>> tools, and serialization (XStream). In some cases the
>>>>>> particular library to be used is not known until run time
>>>>>> (e.g., Hibernate and EclipseLink
>>>> both
>>>>>> implement JPA). This capability is also sometimes used
>>>>>> to work
>>>> around
>>>>>> bugs in unchangeable code. Access to non-exported
>>>>>> packages can, at present, only be done via command-line
>>>>>> flags, which is extremely awkward. Provide an easier way
>>>>>> for reflective code to access such non-exported types.
>>>>>> [1]
>>>>>>
>>>>>> Proposal --------
>>>>>>
>>>>>> Extend the language of module declarations so that a
>>>>>> package can be declared to be exported at run time but
>>>>>> not at compile time. This is, roughly, the dual of the
>>>>>> `requires static` construct proposed for
>>>>>> #CompileTimeDependences, hence we propose to introduce a
>>>>>> new modifier, `dynamic`, for use on the `exports`
>>>>>> directive. It has the following meanings:
>>>>>>
>>>>>> - At compile time, `exports dynamic P` does not cause
>>>>>> the package `P` to be exported, though it does require
>>>>>> `P` to be a package defined in the module.
>>>>>>
>>>>>> - In phases after compile time, `exports dynamic P`
>>>>>> behaves in
>>>> exactly
>>>>>> the same way as `exports P`. It therefore takes part
>>>>>> fully in resolution and configuration, and is subject to
>>>>>> the same
>>>> consistency
>>>>>> constraints as normally-exported packages (e.g., no
>>>>>> split
>>>> packages).
>>>>>>
>>>>>> Thus a module declaration of the form
>>>>>>
>>>>>> module com.foo.app { requires hibernate.core; requires
>>>>>> hibernate.entitymanager; exports dynamic
>>>>>> com.foo.app.model; }
>>>>>>
>>>>>> makes the types in the `com.foo.app.model` package
>>>>>> accessible at run
>>>> time
>>>>>> but not at compile time. In combination with the
>>>>>> earlier change to enable #ReflectionWithoutReadability
>>>>>> [2] this means that frameworks
>>>> that
>>>>>> today use core reflection to manipulate user classes at
>>>>>> run time are
>>>> more
>>>>>> likely to work out-of-the-box, without change, as
>>>>>> automatic modules.
>>>> If
>>>>>> the `com.foo.app.model` package in this example includes
>>>>>> entity classes to be managed by Hibernate then the
>>>>>> framework will be able to access
>>>> them
>>>>>> without further ado, but under normal circumstances an
>>>>>> attempt to
>>>> compile
>>>>>> code that refers directly to those classes will fail.
>>>>>>
>>>>>> The `dynamic` modifier can be applied to both unqualified
>>>>>> and qualified `exports` directives, though the caveats on
>>>>>> using qualified exports [3] still apply.
>>>>>>
>>>>>> Notes -----
>>>>>>
>>>>>> - To access a non-public member in a
>>>>>> dynamically-exported package you must still invoke the
>>>>>> appropriate `setAccessible` method, just as you do today
>>>>>> to access non-public members in an exported package.
>>>>>>
>>>>>> - This proposal is similar to R?mi's suggestion to add a
>>>>>> way to
>>>> export
>>>>>> a package only for reflection, still requiring that
>>>>>> `setAccessible` be invoked [4]. This proposal differs
>>>>>> in that it does not require `setAccessible` to be invoked
>>>>>> to access public members in a dynamically-exported
>>>>>> package.
>>>>>>
>>>>>> - This proposal primarily addresses "friendly" uses of
>>>>>> reflection,
>>>> such
>>>>>> as dependency injection and persistence, in which the
>>>>>> author of a module knows in advance that one or more
>>>>>> packages must be exported
>>>> at
>>>>>> run time for reflective access by frameworks. This
>>>>>> proposal is
>>>> also
>>>>>> convenient for frameworks that generate code at run
>>>>>> time, since the constant pool of a generated class file
>>>>>> can include static
>>>> references
>>>>>> to types in dynamically exported packages.
>>>>>>
>>>>>> - This proposal opens the door to a category of
>>>>>> second-class APIs.
>>>> If
>>>>>> a package is exported dynamically then you can still
>>>>>> compile code that refers to types in the package, by
>>>>>> using `-XaddExports` or its equivalent at compile time,
>>>>>> and it will work as expected at run
>>>> time.
>>>>>> It thus may be useful to use `exports dynamic` for
>>>>>> packages that contain legacy APIs whose use is strongly
>>>>>> discouraged, e.g., those
>>>> in
>>>>>> the `jdk.unsupported` module of the reference
>>>>>> implementation,
>>>> thereby
>>>>>> forcing anyone who wants to compile against them to go
>>>>>> to the
>>>> trouble
>>>>>> of using `-XaddExports`.
>>>>>>
>>>>>> - Intrusive access to arbitrary packages of arbitrary
>>>>>> modules by,
>>>> e.g.,
>>>>>> debugging tools, will still require the use of sharp
>>>>>> knives such as the `-XaddExports` command-line option or
>>>>>> its equivalent, or JVM
>>>> TI.
>>>>>>
>>>>>> - Using the `-XaddExports` option or its equivalent
>>>>>> remains awkward, and sometimes it's the only way out. To
>>>>>> ease migration I think
>>>> it's
>>>>>> worth considering some way for an application packaged as
>>>>>> a JAR
>>>> file
>>>>>> to include such options in its `MANIFEST.MF` file, as
>>>>>> suggested by Simon Nash [5]. I'll create a separate
>>>>>> issue for that.
>>>>>>
>>>>>> - New kinds of resolution failures are possible. If
>>>>>> module `A` requires `B`, and `B` requires `C`, then if
>>>>>> `B` and `C` both
>>>> declare
>>>>>> some package `P` to be exported dynamically, then a
>>>>>> split-package error will be reported. This may
>>>>>> surprise, since the packages are just internal
>>>>>> implementation details that were exported dynamically so
>>>>>> that some framework could access their types. This is
>>>>>> not, however, all that different from the kinds of
>>>>>> resolution and layer-creation failures that are already
>>>>>> possible due to package collisions. It's unlikely to
>>>>>> happen all that often in practice, and the fix is
>>>>>> straightforward: Just use reverse-DNS or otherwise
>>>>>> sufficiently-unique package names, as most people already
>>>>>> do. It is worth exploring whether javac can detect at
>>>>>> least some of these kinds of collisions and issue
>>>>>> appropriate warnings.
>>>>>
>>>>> I would like to agree on that proposal only on the ground
>>>>> that it
>>>> requires
>>>>> to change the classfile format thus allows in the same time
>>>>> to fix the issues of the current classfile spec (encoding
>>>>> of the name module as a Class instead of UTF8, wrong flag
>>>>> for requires public), but i'm not sure this proposal
>>>>> support it's own weight.
>>>>>
>>>>> Without dynamic, a 'classical' export works the same way
>>>>> at runtime so adding 'dynamic' to an export only restrict
>>>>> the usage at compile time. Given that we can already
>>>>> selectively export packages to some chosen modules which
>>>>> hide them for the rest of the other modules at compile
>>>> time,
>>>>> adding dynamic to the spec adds little value.
>>>>>
>>>>> And it has several drawbacks, the main one is that it
>>>>> weaken the compiletime/runtime fidelity story, with a
>>>>> dynamic export the runtime
>>>> world
>>>>> is not the compiletime world anymore. The main motivation
>>>>> to add dynamic seems to be for backward compatibility
>>>>> during the transition between the classpath world and the
>>>>> full module
>>>> world,
>>>>> for several other cases, we use the command line for that,
>>>>> i don't see
>>>> why
>>>>> this case is so special that it should not work like the
>>>>> other compatibility issues.
>>>>>
>>>>> Moreover, an agent, an annotation processor and a jlink
>>>>> plugin are all able to implement the same trick, add export
>>>>> after the compilation and before the runtime, so i don't
>>>>> think it's a good idea to change the spec for that.
>>>>>
>>>>> R?mi
>>>>>
>>>>>>
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>
>>>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccess
>>
>>>>
ToNonExportedTypes
>>>>>>
>>>>
>> [2]
>>>>>>
>>>>>
>>>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithou
>>
>>>>
tReadability
>>>>>>
>>>>
>> [3]
>>>>> http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-export
>>
>>>>>
s
>>>>>>
>>>>>
>> [4]
>>>>>>
>>>>>
>>>> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-Decembe
>>
>>>>
r/000205.html
>>>>>>
>>>>
>> [5]
>>>>>>
>>>>>
>>>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/00574
>>
>>>>
5.html
>>>>>>
>>>>>
>>>>
>>>
>>>>
>> --
>>
>> PGP Key:
>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>>
>>
>>
Web:
>> http://codefx.org a blog about software development
>> http://do-foss.de Free and Open Source Software for the City of
>> Dortmund
>>
>> Twitter: https://twitter.com/nipafx
>>
>
--
PGP Key:
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
Web:
http://codefx.org
a blog about software development
http://do-foss.de
Free and Open Source Software for the City of Dortmund
Twitter:
https://twitter.com/nipafx
From forax at univ-mlv.fr Fri Jul 1 07:39:00 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Fri, 1 Jul 2016 09:39:00 +0200 (CEST)
Subject: Proposal: #FixClassFileFormat
In-Reply-To: <1813643129.1709819.1467357511504.JavaMail.zimbra@u-pem.fr>
Message-ID: <1751220489.1722677.1467358740078.JavaMail.zimbra@u-pem.fr>
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Jeudi 30 Juin 2016 02:35:59
> Objet: Re: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
>
> 2016/6/29 1:51:16 -0700, Remi Forax <forax at univ-mlv.fr>:
> > I would like to agree on that proposal only on the ground that it
> > requires to change the classfile format thus allows in the same time
> > to fix the issues of the current classfile spec (encoding of the name
> > module as a Class instead of UTF8, wrong flag for requires public),
>
> I think that these are completely distinct issues (though your
> class-file issues are not yet in the issue list -- would you care
> to suggest some text?).
yes, sure.
when implementing the support of module into ASM, i've discovered several issues in the classfile format as defined by [1]
- module name is not encoded as UTF8.
The spec makes great cares to encode reference to module in 'requires' by example as "UTF8" constant pool entry, a name with no other constraints, but fail to do the same thing for the name of the module itself.
The name of the module is derived from the name of the class which is encoded as a "Class" constant pool entry. The way to solve that is to avoid to derive the name of the module from the class, the name of the class should be module-info, with no package, which seems logical given that the source file is module-info.java (not prefixed by a package) and the name of the module should be encodedat the beginning of the "Module" attribute as a reference to the constant pool entry of type UTF8. This format can also be more compact despite adding two bytes in the "Module" attribute in the fairly common case where one exported package as the same name as the module name because of the constant poll sharing the names.
- there is what i believe is a typo in the current spec when specifying the modifier of a required module (requires_flags), the modifier ACC_PUBLIC is said to have the value 0x0020 instead of 0x0001 in the JVM spec. (0x0020 is either ACC_SUPER or ACC_SYNCHRONIZED).
- the attributes for specifying the main-class, the module-version, a concealed module, etc are not specified, and i don't know if there is a reason to not specifying them.
R?mi
[1] http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html
From forax at univ-mlv.fr Fri Jul 1 07:52:56 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Fri, 1 Jul 2016 09:52:56 +0200 (CEST)
Subject: Proposal: #ModuleAnnotations and #ModuleDeprecation
In-Reply-To: <20160629163718.727221014eggemoggin.niobe.net>
References: <20160628211915.9706BB59C8@eggemoggin.niobe.net>
<1683195038.1023897.1467191318948.JavaMail.zimbra@u-pem.fr>
<20160629163718.727221014eggemoggin.niobe.net>
Message-ID: <699821914.1730784.1467359576240.JavaMail.zimbra@u-pem.fr>
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Jeudi 30 Juin 2016 01:37:18
> Objet: Re: Proposal: #ModuleAnnotations and #ModuleDeprecation
>
> 2016/6/29 2:08:38 -0700, Remi Forax <forax at univ-mlv.fr>:
> > ...
> >
> > I like both proposals, adding annotations and imports on module.
> >
> > I disagree that those annotations should not be visible at runtime, i
> > don't see why we need a special case here, it's not complex to
> > implement because the VM already provide the hook to access to those
> > annotations at runtime, @Deprecated is an annotation visible at
> > runtime for a good reason, it allows by example to implement policy
> > like disallowing or emit a warning when calling a deprecated code at
> > runtime.
>
> I completely agree, and the intent of the proposal is to make
> annotations on module declarations available at run time -- that's
> why it specifically mentions extending the `java.lang.reflect` API.
>
> If there's text in the proposal that suggests otherwise then please
> point it out and I'll fix it. Thanks.
>
> - Mark
>
sorry,
it's me not being able to parse "This proposal will not add support to ModuleDescriptor".
So it seems we all agree on this issue :)
I understand why you don't want to add support of annotations to ModuleDescriptor because creating a piece of metadata at runtime that will be seem as an annotation is something several people have tried to do (by example CDI/Weld) but the result is not very elegant.
R?mi
From sander.mak at luminis.eu Fri Jul 1 08:41:19 2016
From: sander.mak at luminis.eu (Sander Mak)
Date: Fri, 1 Jul 2016 08:41:19 +0000
Subject: Proposal: #CompileTimeDependences: `requires static`
In-Reply-To: <57760387.9040405@codefx.org>
References: <20160628211715.84610B59BF@eggemoggin.niobe.net>
<1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr>
<20160628163150.929695476eggemoggin.niobe.net>
<57760387.9040405@codefx.org>
Message-ID: <CE0BB143-9FF5-4D4A-8730-24D4FC84A1DD@luminis.eu>
>> requires & consumes exports & contains
>
> So it would look as follows:
>
> module com.xyz {
> requires guava;
> requires joda;
> consumes hibernate;
> exports com.xyz.a;
> exports com.xyz.b;
> contains com.xyz.c;
> }
>
Going down this route, I would avoid 'consumes' since IMO it appears to be the inverse of 'provides' used with services. Also, the the proposed 'contains' seems to be equivalent to what is now called 'conceals' (when looking at output of for example `java -listmods:java.base`), which I find to be more descriptive. Or did you mean that 'contains' replaces 'exports dynamic'? In that case, I don't really see how that's any clearer.
Sander
From sven.reimers at gmail.com Fri Jul 1 08:41:48 2016
From: sven.reimers at gmail.com (Sven Reimers)
Date: Fri, 1 Jul 2016 10:41:48 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <57761B70.10403@codefx.org>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
Message-ID: <CAP+Jvx4NPxhJVYoi6+ckB88hLm9Hw8B0gACDMVjP81cPc9BoXA@mail.gmail.com>
Hi all,
regarding exporting DTO to hibernate..
I think the whole point of having JPA is to not have explicit dependency to
hibernate..
So if I want to make my DTO's visible the declaration should be towards
JPA..
-Sven
Am 01.07.2016 09:28 schrieb "Nicolai Parlog" <nipa at codefx.org>:
> Hi!
>
> > I do not think that shielding internals is a problem with regards
> > to reflection.
>
> I think it is. I've not been emerged in the community for very long so
> I know only two examples (but still): Unsafe and JUnit 4. In both
> cases internals were accessed via reflection. And in both cases
> dependencies became so ubiquitous that changing any details became
> impossible for the maintainers.
>
> I think the outcry when Unsafe was planned to go away and the fact
> that JUnit 4 had to be rewritten because evolution was effectively
> prevented by unwanted dependencies make my point here.
>
> > I do neither think that reflection only interests internals.
>
> I agree. Not "only" but "all too often", though.
>
> > For example, maybe you use Hibernate for your persistance layer
> > without exposing those DTOs from your module.
>
> Why would I do that?! This is exactly where reflection got us used to
> sloppy architecture. If a tool uses my DTOs to implement my
> persistence layer, then it looks like those DTOs are part of my API.
> Maybe not public to everyone but qualified and dynamic exports give me
> tools to reduce visibility.
>
> > I do not care about how Hibernate works as long as it does work.
> > Almost any library uses reflection and I would not want to dig
> > into implementation details to find out how to set up my dynamic
> > exports.
>
> Maybe I'm missing something here but to me it looks like "export to a
> module whatever that module is supposed to interact with" is a very
> sensible default that should cover a vast majority of uses cases, no?
>
> I have DTOs && Hibernate needs DTOs -> export DTOs to Hibernate.
>
> Why would I look at the implementation? And if there were non-obvious
> scenarios they should be properly documented so I can react accordingly.
>
> > As a library implementor, you aim for the biggest convenience.
>
> Maybe we should aim for clarity and well-defined interactions instead.
>
> so long ... Nicolai
>
>
>
> On 01.07.2016 00:20, Rafael Winterhalter wrote:
> > I disagree on two ends here.
> >
> > I do not think that shielding internals is a problem with regards
> > to reflection. You want to shield internals in order to not break
> > dependants by prohibiting their existance. With reflection, you do
> > however typically react to the actual implementation by discovering
> > it at runtime. If your use pattern was fixed, you would always
> > require a static dependency over reflection in the first place. I
> > do neither think that reflection only interests internals. For
> > example, maybe you use Hibernate for your persistance layer without
> > exposing those DTOs from your module. The problem is that you
> > invert responsibilities. I do not care about how Hibernate works as
> > long as it does work. Almost any library uses reflection and I
> > would not want to dig into implementation details to find out how
> > to set up my dynamic exports. I want to require a module and
> > hopefully the compiler tells me about anything that's wrong.
> > Reflection access errors would only be an awkward side effect that
> > can be avoided by not applying modules alltogether. I think this
> > limitation would therefore hurt Java 9 adoption.
> >
> > The second problem I have is the issue of attempting to fix an old
> > regret. From the perspective of Hibernate, it would be easiest to
> > not attempt modules as their introduction would require its users
> > to apply additional configuration. As a library implementor, you
> > aim for the biggest convenience. Adding this restriction neither
> > adds additional security where a security manager should be used. I
> > again think this would hurt adoption.
> >
> > I agree with Stephen's argument even though I do not think that
> > forbidding dynamic access would be a feature anybody would use so
> > it should not be added. Rather, something should be integrated in
> > the security manager, if this is considered necessary.
> >
> > Cheers, Rafael
> >
> > Am 30.06.2016 11:37 nachm. schrieb "Nicolai Parlog"
> > <nipa at codefx.org>:
> >>
> >> Hi!
> >>
> >>> In my opinion, reflection should not apply any module-boundary
> >>> checks.
> >>
> >> Maybe this list is not a representative selection of Java
> >> developers, maybe I'm just the only one thinking this but, to
> >> paraphrase:
> >>
> >> Reflection over internals must die in a fire!
> >>
> >> Yes, yes, tools to great things, we need to see everything, ...
> >> Still, I think that modules need a safe space that they can use
> >> without anybody intruding. Maybe I'm a hardliner but I already
> >> view the command line parameters with suspicion.
> >>
> >> In that light, I like the "dynamic" proposal.
> >>
> >> Not the name, though. Having the symmetric pair "static vs
> >> dynamic" looks nice but it takes a lot of words to explain why
> >> exactly these terms apply here. We seem to be looking for
> >> qualifiers for "requires" and "exports". Maybe just using other
> >> terms solves the problem?
> >>
> >> requires & consumes exports & contains
> >>
> >> so long ... Nicolai
> >>
> >>
> >>
> >> On 30.06.2016 01:23, Rafael Winterhalter wrote:
> >>> I also agree here that the use of reflection should not require
> >>> any form of explicit permission by the user of a library. In a
> >>> way, this ties an implementation detail (reflection in this
> >>> case) to the usage of the library. The library cannot work
> >>> without the explicit configuration by a user and this only adds
> >>> boilerplate to using Hibernate.
> >>>
> >>> Unfortunately, many Java developers still do not know much
> >>> about reflection or about the inner workings of many libraries
> >>> in the market. If a user needed to explicitly export packages,
> >>> this might just lead to people "over exporting" their packages
> >>> just to "make Hibernate work".
> >>>
> >>> In my opinion, reflection should not apply any module-boundary
> >>> checks.
> >>>
> >>> Best regards, Rafael
> >>>
> >>> 2016-06-29 20:13 GMT+02:00 Paul Benedict
> >>> <pbenedict at apache.org>:
> >>>
> >>>> Replying to "observers" list...
> >>>>
> >>>> I think the main problem here is that core reflection has
> >>>> become too restrictive. Reflection should be "magical" and
> >>>> have no knowledge of any kind of module and/or visibility
> >>>> boundaries. If you really want to control what can be
> >>>> reflected upon, that should be done via the Policy file of
> >>>> the Security Manager.
> >>>>
> >>>> This proposal is trying to hack out-of-the-box created by
> >>>> too strict of a design. It is very awkward to denote which
> >>>> packages must be reflectable ahead of time --- how can you
> >>>> figure that out?? The consumers who are doing the reflection
> >>>> are innumerable. I'd go as far to say it's it's impossible to
> >>>> devise a preemptive comprehensive strategy that may satisfy
> >>>> the kinds of tooling in the wild.
> >>>>
> >>>> It's best just to open up reflection so it can continue
> >>>> doing what it always has done well: let me see anything I
> >>>> want unless the Security Manager says no.
> >>>>
> >>>> Cheers, Paul
> >>>>
> >>>> On Wed, Jun 29, 2016 at 3:51 AM, Remi Forax
> >>>> <forax at univ-mlv.fr> wrote:
> >>>>
> >>>>>
> >>>>>
> >>>>> ----- Mail original -----
> >>>>>> De: "Mark Reinhold" <mark.reinhold at oracle.com> ?:
> >>>>>> jpms-spec-experts at openjdk.java.net Envoy?: Mardi 28 Juin
> >>>>>> 2016 23:18:15 Objet: Proposal:
> >>>>>> #ReflectiveAccessToNonExportedTypes: `exports dynamic`
> >>>>>>
> >>>>>> Issue summary -------------
> >>>>>>
> >>>>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of
> >>>>>> framework libraries require reflective access to members
> >>>>>> of the non-exported types of other modules; examples
> >>>>>> include dependency injection
> >>>> (Guice),
> >>>>>> persistence (JPA), debugging tools, code-automation
> >>>>>> tools, and serialization (XStream). In some cases the
> >>>>>> particular library to be used is not known until run time
> >>>>>> (e.g., Hibernate and EclipseLink
> >>>> both
> >>>>>> implement JPA). This capability is also sometimes used
> >>>>>> to work
> >>>> around
> >>>>>> bugs in unchangeable code. Access to non-exported
> >>>>>> packages can, at present, only be done via command-line
> >>>>>> flags, which is extremely awkward. Provide an easier way
> >>>>>> for reflective code to access such non-exported types.
> >>>>>> [1]
> >>>>>>
> >>>>>> Proposal --------
> >>>>>>
> >>>>>> Extend the language of module declarations so that a
> >>>>>> package can be declared to be exported at run time but
> >>>>>> not at compile time. This is, roughly, the dual of the
> >>>>>> `requires static` construct proposed for
> >>>>>> #CompileTimeDependences, hence we propose to introduce a
> >>>>>> new modifier, `dynamic`, for use on the `exports`
> >>>>>> directive. It has the following meanings:
> >>>>>>
> >>>>>> - At compile time, `exports dynamic P` does not cause
> >>>>>> the package `P` to be exported, though it does require
> >>>>>> `P` to be a package defined in the module.
> >>>>>>
> >>>>>> - In phases after compile time, `exports dynamic P`
> >>>>>> behaves in
> >>>> exactly
> >>>>>> the same way as `exports P`. It therefore takes part
> >>>>>> fully in resolution and configuration, and is subject to
> >>>>>> the same
> >>>> consistency
> >>>>>> constraints as normally-exported packages (e.g., no
> >>>>>> split
> >>>> packages).
> >>>>>>
> >>>>>> Thus a module declaration of the form
> >>>>>>
> >>>>>> module com.foo.app { requires hibernate.core; requires
> >>>>>> hibernate.entitymanager; exports dynamic
> >>>>>> com.foo.app.model; }
> >>>>>>
> >>>>>> makes the types in the `com.foo.app.model` package
> >>>>>> accessible at run
> >>>> time
> >>>>>> but not at compile time. In combination with the
> >>>>>> earlier change to enable #ReflectionWithoutReadability
> >>>>>> [2] this means that frameworks
> >>>> that
> >>>>>> today use core reflection to manipulate user classes at
> >>>>>> run time are
> >>>> more
> >>>>>> likely to work out-of-the-box, without change, as
> >>>>>> automatic modules.
> >>>> If
> >>>>>> the `com.foo.app.model` package in this example includes
> >>>>>> entity classes to be managed by Hibernate then the
> >>>>>> framework will be able to access
> >>>> them
> >>>>>> without further ado, but under normal circumstances an
> >>>>>> attempt to
> >>>> compile
> >>>>>> code that refers directly to those classes will fail.
> >>>>>>
> >>>>>> The `dynamic` modifier can be applied to both unqualified
> >>>>>> and qualified `exports` directives, though the caveats on
> >>>>>> using qualified exports [3] still apply.
> >>>>>>
> >>>>>> Notes -----
> >>>>>>
> >>>>>> - To access a non-public member in a
> >>>>>> dynamically-exported package you must still invoke the
> >>>>>> appropriate `setAccessible` method, just as you do today
> >>>>>> to access non-public members in an exported package.
> >>>>>>
> >>>>>> - This proposal is similar to R?mi's suggestion to add a
> >>>>>> way to
> >>>> export
> >>>>>> a package only for reflection, still requiring that
> >>>>>> `setAccessible` be invoked [4]. This proposal differs
> >>>>>> in that it does not require `setAccessible` to be invoked
> >>>>>> to access public members in a dynamically-exported
> >>>>>> package.
> >>>>>>
> >>>>>> - This proposal primarily addresses "friendly" uses of
> >>>>>> reflection,
> >>>> such
> >>>>>> as dependency injection and persistence, in which the
> >>>>>> author of a module knows in advance that one or more
> >>>>>> packages must be exported
> >>>> at
> >>>>>> run time for reflective access by frameworks. This
> >>>>>> proposal is
> >>>> also
> >>>>>> convenient for frameworks that generate code at run
> >>>>>> time, since the constant pool of a generated class file
> >>>>>> can include static
> >>>> references
> >>>>>> to types in dynamically exported packages.
> >>>>>>
> >>>>>> - This proposal opens the door to a category of
> >>>>>> second-class APIs.
> >>>> If
> >>>>>> a package is exported dynamically then you can still
> >>>>>> compile code that refers to types in the package, by
> >>>>>> using `-XaddExports` or its equivalent at compile time,
> >>>>>> and it will work as expected at run
> >>>> time.
> >>>>>> It thus may be useful to use `exports dynamic` for
> >>>>>> packages that contain legacy APIs whose use is strongly
> >>>>>> discouraged, e.g., those
> >>>> in
> >>>>>> the `jdk.unsupported` module of the reference
> >>>>>> implementation,
> >>>> thereby
> >>>>>> forcing anyone who wants to compile against them to go
> >>>>>> to the
> >>>> trouble
> >>>>>> of using `-XaddExports`.
> >>>>>>
> >>>>>> - Intrusive access to arbitrary packages of arbitrary
> >>>>>> modules by,
> >>>> e.g.,
> >>>>>> debugging tools, will still require the use of sharp
> >>>>>> knives such as the `-XaddExports` command-line option or
> >>>>>> its equivalent, or JVM
> >>>> TI.
> >>>>>>
> >>>>>> - Using the `-XaddExports` option or its equivalent
> >>>>>> remains awkward, and sometimes it's the only way out. To
> >>>>>> ease migration I think
> >>>> it's
> >>>>>> worth considering some way for an application packaged as
> >>>>>> a JAR
> >>>> file
> >>>>>> to include such options in its `MANIFEST.MF` file, as
> >>>>>> suggested by Simon Nash [5]. I'll create a separate
> >>>>>> issue for that.
> >>>>>>
> >>>>>> - New kinds of resolution failures are possible. If
> >>>>>> module `A` requires `B`, and `B` requires `C`, then if
> >>>>>> `B` and `C` both
> >>>> declare
> >>>>>> some package `P` to be exported dynamically, then a
> >>>>>> split-package error will be reported. This may
> >>>>>> surprise, since the packages are just internal
> >>>>>> implementation details that were exported dynamically so
> >>>>>> that some framework could access their types. This is
> >>>>>> not, however, all that different from the kinds of
> >>>>>> resolution and layer-creation failures that are already
> >>>>>> possible due to package collisions. It's unlikely to
> >>>>>> happen all that often in practice, and the fix is
> >>>>>> straightforward: Just use reverse-DNS or otherwise
> >>>>>> sufficiently-unique package names, as most people already
> >>>>>> do. It is worth exploring whether javac can detect at
> >>>>>> least some of these kinds of collisions and issue
> >>>>>> appropriate warnings.
> >>>>>
> >>>>> I would like to agree on that proposal only on the ground
> >>>>> that it
> >>>> requires
> >>>>> to change the classfile format thus allows in the same time
> >>>>> to fix the issues of the current classfile spec (encoding
> >>>>> of the name module as a Class instead of UTF8, wrong flag
> >>>>> for requires public), but i'm not sure this proposal
> >>>>> support it's own weight.
> >>>>>
> >>>>> Without dynamic, a 'classical' export works the same way
> >>>>> at runtime so adding 'dynamic' to an export only restrict
> >>>>> the usage at compile time. Given that we can already
> >>>>> selectively export packages to some chosen modules which
> >>>>> hide them for the rest of the other modules at compile
> >>>> time,
> >>>>> adding dynamic to the spec adds little value.
> >>>>>
> >>>>> And it has several drawbacks, the main one is that it
> >>>>> weaken the compiletime/runtime fidelity story, with a
> >>>>> dynamic export the runtime
> >>>> world
> >>>>> is not the compiletime world anymore. The main motivation
> >>>>> to add dynamic seems to be for backward compatibility
> >>>>> during the transition between the classpath world and the
> >>>>> full module
> >>>> world,
> >>>>> for several other cases, we use the command line for that,
> >>>>> i don't see
> >>>> why
> >>>>> this case is so special that it should not work like the
> >>>>> other compatibility issues.
> >>>>>
> >>>>> Moreover, an agent, an annotation processor and a jlink
> >>>>> plugin are all able to implement the same trick, add export
> >>>>> after the compilation and before the runtime, so i don't
> >>>>> think it's a good idea to change the spec for that.
> >>>>>
> >>>>> R?mi
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> [1]
> >>>>>>
> >>>>>
> >>>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccess
> >>
> >>>>
> ToNonExportedTypes
> >>>>>>
> >>>>
> >> [2]
> >>>>>>
> >>>>>
> >>>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithou
> >>
> >>>>
> tReadability
> >>>>>>
> >>>>
> >> [3]
> >>>>> http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-export
> >>
> >>>>>
> s
> >>>>>>
> >>>>>
> >> [4]
> >>>>>>
> >>>>>
> >>>> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-Decembe
> >>
> >>>>
> r/000205.html
> >>>>>>
> >>>>
> >> [5]
> >>>>>>
> >>>>>
> >>>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/00574
> >>
> >>>>
> 5.html
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>>>
> >> --
> >>
> >> PGP Key:
> >> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
> >>
> >>
> >>
> Web:
> >> http://codefx.org a blog about software development
> >> http://do-foss.de Free and Open Source Software for the City of
> >> Dortmund
> >>
> >> Twitter: https://twitter.com/nipafx
> >>
> >
>
> --
>
> PGP Key:
> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>
> Web:
> http://codefx.org
> a blog about software development
> http://do-foss.de
> Free and Open Source Software for the City of Dortmund
>
> Twitter:
> https://twitter.com/nipafx
>
>
From forax at univ-mlv.fr Fri Jul 1 08:48:07 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Fri, 1 Jul 2016 10:48:07 +0200 (CEST)
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <20160629173559.18960061eggemoggin.niobe.net>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<20160629173559.18960061eggemoggin.niobe.net>
Message-ID: <1720977211.1766068.1467362887039.JavaMail.zimbra@u-pem.fr>
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Jeudi 30 Juin 2016 02:35:59
> Objet: Re: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
>
[...]
>
> > but i'm not sure this proposal support it's own weight.
> >
> > Without dynamic, a 'classical' export works the same way at runtime so
> > adding 'dynamic' to an export only restrict the usage at compile time.
> > Given that we can already selectively export packages to some chosen
> > modules which hide them for the rest of the other modules at compile
> > time, adding dynamic to the spec adds little value.
>
> Qualified exports aren't really a satisfactory substitute for dynamic
> exports. They still export packages at compile time, thereby tempting
> unwanted usage. They're also really meant only for use within a group
> of tightly-associated modules, where hashing can be used to ensure
> integrity, rather than amongst unrelated modules, between which names
> can all too easily be spoofed.
>
> More to the point, qualified exports are not adequate for all use cases.
> If you want to write a module that uses JPA but is independent of JPA
> implementation then you'll need to make your entity classes available
> to whichever JPA implementation you do wind up using (Hibernate or
> EclipseLink or ...), but you don't know the name of that
> implementation's module. In other words, you can't write
>
> exports com.foo.app.model to jpa;
>
> because `jpa` is not an actual module, it's a stand-in.
>
> We could introduce a level of indirection so that you could do that sort
> of thing, say some notion of "abstract module", but that'd be a very,
> very deep change and it doesn't fit at all well with existing practices
> around JAR files.
Ok, qualified export to a specific module is a bad idea here,
export to anybody seems a better fit.
>
> The nice thing about `exports dynamic` is that it lets you say "just
> make the types in this package available for reflection by anybody, I'm
> okay with that".
There are two differences between unqualified export and exports dynamic:
- exported types are only available at runtime not at compile time,
- exported types are available by anybody so a module like hibernate doesn't have to requires a user module.
I fail to see why these two guys are bundled together, i may want to have exported types available to anybody and visible at compile time.
but i understand that having a way to say i want to export a package to anybody is interesting especially for library like hibernate.
Moreover, i don't like the fact that Java the language has a new way to interact with the reflection. The more we introduce such coupling the less we will be able to move in the future.
so instead of exports dynamic i propose to introduce a way to say i want to export a package to anybody,
an exports to * (star).
Using the example of the initial proposal, the module-info will be
module com.foo.app {
requires hibernate.core;
requires hibernate.entitymanager;
exports com.foo.app.model to *;
}
>
> > And it has several drawbacks, the main one is that it weaken the
> > compiletime/runtime fidelity story, with a dynamic export the runtime
> > world is not the compiletime world anymore. The main motivation to
> > add dynamic seems to be for backward compatibility during the
> > transition between the classpath world and the full module world, for
> > several other cases, we use the command line for that, i don't see why
> > this case is so special that it should not work like the other
> > compatibility issues.
>
> Yes, it does weaken the fidelity story in order to accommodate existing
> practice, but at least it does so in an explicit manner so that what's
> going on is apparent just from reading the code.
>
> I don't think, however, that this change is just about compatibility and
> migration. Even in an idealistic, Utopian, fully-modular world I expect
> that we'll still want to use frameworks that use reflection on our own
> types to help us be more productive.
yes, that's very true. Reflection allows to write meta object protocols but making a special case for the reflection seems wrong to me.
>
> > Moreover, an agent, an annotation processor and a jlink plugin are all
> > able to implement the same trick, add export after the compilation and
> > before the runtime, so i don't think it's a good idea to change the
> > spec for that.
>
> Then how do you express, in your source code, that you want an export to
> be added at run time?
>
> You could use an annotation, but that would require anybody who wants
> to do this sort of thing to configure their environment to include the
> module that defines that annotation and to add the required annotation
> processor at the appropriate point. Do we really want to tell people
> to do that when we can solve the problem for them so simply?
>
> - Mark
>
R?mi
From sander.mak at luminis.eu Fri Jul 1 08:49:13 2016
From: sander.mak at luminis.eu (Sander Mak)
Date: Fri, 1 Jul 2016 08:49:13 +0000
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <CAP+Jvx4NPxhJVYoi6+ckB88hLm9Hw8B0gACDMVjP81cPc9BoXA@mail.gmail.com>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
<CAP+Jvx4NPxhJVYoi6+ckB88hLm9Hw8B0gACDMVjP81cPc9BoXA@mail.gmail.com>
Message-ID: <05815A9E-FAEB-438E-A8F9-A51C9F7DFAC7@luminis.eu>
> On 01 Jul 2016, at 10:41, Sven Reimers <sven.reimers at gmail.com> wrote:
>
> Hi all,
>
> regarding exporting DTO to hibernate..
>
> I think the whole point of having JPA is to not have explicit dependency to
> hibernate..
Hence the proposal for `exports dynamic`, where you don't have to qualify the export to a certain module. The relation between the DTO module and JPA would be expressed by having a `requires jpa` (or whatever module the spec annotations will live in). The `dynamic` part makes sure no other modules take an accidental dependency on your DTOs (if that's what you want) at compile-time. All sounds good to me.
Sander
From scolebourne at joda.org Fri Jul 1 08:53:55 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Fri, 1 Jul 2016 09:53:55 +0100
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <5775A765.7090301@oracle.com>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<5775A765.7090301@oracle.com>
Message-ID: <CACzrW9CzBFZ6VLBgLz5K4pDFHj801Ruu_F_bjTxQoBKXhRYdpQ@mail.gmail.com>
On 1 July 2016 at 00:12, Alex Buckley <alex.buckley at oracle.com> wrote:
> That said, as soon as you mention inverting responsibilities, it becomes
> clear that services are the answer. Rather than the framework reflecting
> over the internals of a user module to discover suitable classes, the
> framework should define a service interface and a user module should provide
> a service implementation. It seems like this is a style of OO programming
> that has passed many framework developers by
Not really. The original frameworks in Java operated this way, with
specific interfaces that applications had to implement. Spring and
Ruby on Rails changed this, with a string desire for non-intrusive
frameworks, POJOs and configuraton by convention.
So, while I understand the benefits of a ServiceLoader approach, I
think it is important to accept that it is very unpopular with
framework authors over the last 10-15 years. Annotations, reflection
and agents are where framework design is. (ServiceLoader also seems to
be a rather arcane API)
I do think Paul's arguments hit the nail on the head. Even if you
could work out which packages need exposure by reflection, you would
be doing so for today, and highly likely to cause yourself problems in
the future. This is especially so for open source libraries.
To give an example of a potential problem, if application A depends on
library B which today has reflection, A could declare a dynamic export
to B. But if B is later rewritten to use a third party library C for
reflection, then A's export is no longer valid, and the later version
of B will not work.
I also note that it is inconsistent that the proposed resolution of
this issue is different to that for #ResourceEncapsulation. With
resources, the proposal is to allow access to all resources. Yet here,
there is an attempt to limit reflection. The resolution to
#ReflectionWithoutReadability was good, but to be effective it also
needs open reflective access.
FWIW, I understand the fears of those that are burned by reflective
access making a private API effectively public. But issues like that
come about through a failure of the library author(s) to provide the
correct API, not a failure of the user.
In summary, if this feature remains, I fear my "Java 9 best practices"
talk will have to recommend marking all packages as "dynamic".
Stephen
From rafael.wth at gmail.com Fri Jul 1 09:15:29 2016
From: rafael.wth at gmail.com (Rafael Winterhalter)
Date: Fri, 1 Jul 2016 11:15:29 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <57761B70.10403@codefx.org>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
Message-ID: <CA+DM0AnfqM9j21ULBr61n3swWD605gaAeitDqZxYqtJzXwF56w@mail.gmail.com>
As for JUnit 4 or Unsafe, reflection is used because of an actual need to
interact with these libraries/APIs.
As for JUnit, it is mainly IDE vendors that needed to access internals in
order to display unit testing results and to extract details about testing
results. By making this extraction more awkward, this would not have
changed the fact that IDE-vendors want to interact with the internal APIs.
It would only have been more difficult to do. In my eyes, limiting
reflection is fighting the symptoms, not the cause. For example, JUnit 5
has adressed this issue by implementing a public API for extracting such
information what immediately solved the current problem. I do not think
anybody prefers reflection over an official API if the latter one satisfies
all needs.
The same goes for Unsafe. I do not think anybody likes to use Unsafe if
there is a better alternative. The Unsafe API is awkward and - obviously -
unsafe. And despite the implementors attempt to shield the API to bootstrap
classes, people found their ways around because there has not been another
way to make certain things work. I am sure, the more the JVM offers
adequate alternatives, people will migrate happily.
Also, as a thought experiment, let us say accessing internal APIs had been
impossible in the past: we would have missed out on Hadoop, Cassandra,
Mockito, Spark; and only in 2016 we could start running JUnit from our IDE.
I think we can all agree that this is not the outcome we would have wanted.
Reflection - despite it breaking encapsulation - has done more good then
bad.
As for the argument that required dynamic exports should be documented;
ideally yes. Equally, APIs should be documented but many librararies fail
to offer good documentation, especially in the open-source space where time
is sparse. I already see myself answering questions to puzzled junior
developers that know Java for a few months that "this is required because
this library uses reflection" where they do not even know what the latter
is. Effectively, building this barrier makes Java more difficult to use and
I am convinced that many people will adapt adding dynamic exports for any
package as some form of "best practice" to get on with their lifes. I will
also bet that someone will deliver some "auto-export Maven plugin" to
automate this "best practive".
Finally, I do not believe the service loader API offers an adequate
alternative to many use-cases of reflection (while it can perfectly
substitute some of them). Many state-of-the art libraries actually
intentionally offer POJO approaches without requiring their users to create
interfaces for any shared interaction. I do not think that a gentle push
into a direction that contradicts most existing Java libraries would be a
taken well by the people implementing this software and in the end, the
module system will be as successful as its adoption. I rather see people
finding new ways around restrictions to keep things simple (for themselves).
For all these reasons I do not think that it is a good idea to limit
reflection to exported packages.
2016-07-01 9:27 GMT+02:00 Nicolai Parlog <nipa at codefx.org>:
> Hi!
>
> > I do not think that shielding internals is a problem with regards
> > to reflection.
>
> I think it is. I've not been emerged in the community for very long so
> I know only two examples (but still): Unsafe and JUnit 4. In both
> cases internals were accessed via reflection. And in both cases
> dependencies became so ubiquitous that changing any details became
> impossible for the maintainers.
>
> I think the outcry when Unsafe was planned to go away and the fact
> that JUnit 4 had to be rewritten because evolution was effectively
> prevented by unwanted dependencies make my point here.
>
> > I do neither think that reflection only interests internals.
>
> I agree. Not "only" but "all too often", though.
>
> > For example, maybe you use Hibernate for your persistance layer
> > without exposing those DTOs from your module.
>
> Why would I do that?! This is exactly where reflection got us used to
> sloppy architecture. If a tool uses my DTOs to implement my
> persistence layer, then it looks like those DTOs are part of my API.
> Maybe not public to everyone but qualified and dynamic exports give me
> tools to reduce visibility.
>
> > I do not care about how Hibernate works as long as it does work.
> > Almost any library uses reflection and I would not want to dig
> > into implementation details to find out how to set up my dynamic
> > exports.
>
> Maybe I'm missing something here but to me it looks like "export to a
> module whatever that module is supposed to interact with" is a very
> sensible default that should cover a vast majority of uses cases, no?
>
> I have DTOs && Hibernate needs DTOs -> export DTOs to Hibernate.
>
> Why would I look at the implementation? And if there were non-obvious
> scenarios they should be properly documented so I can react accordingly.
>
> > As a library implementor, you aim for the biggest convenience.
>
> Maybe we should aim for clarity and well-defined interactions instead.
>
> so long ... Nicolai
>
>
>
> On 01.07.2016 00:20, Rafael Winterhalter wrote:
> > I disagree on two ends here.
> >
> > I do not think that shielding internals is a problem with regards
> > to reflection. You want to shield internals in order to not break
> > dependants by prohibiting their existance. With reflection, you do
> > however typically react to the actual implementation by discovering
> > it at runtime. If your use pattern was fixed, you would always
> > require a static dependency over reflection in the first place. I
> > do neither think that reflection only interests internals. For
> > example, maybe you use Hibernate for your persistance layer without
> > exposing those DTOs from your module. The problem is that you
> > invert responsibilities. I do not care about how Hibernate works as
> > long as it does work. Almost any library uses reflection and I
> > would not want to dig into implementation details to find out how
> > to set up my dynamic exports. I want to require a module and
> > hopefully the compiler tells me about anything that's wrong.
> > Reflection access errors would only be an awkward side effect that
> > can be avoided by not applying modules alltogether. I think this
> > limitation would therefore hurt Java 9 adoption.
> >
> > The second problem I have is the issue of attempting to fix an old
> > regret. From the perspective of Hibernate, it would be easiest to
> > not attempt modules as their introduction would require its users
> > to apply additional configuration. As a library implementor, you
> > aim for the biggest convenience. Adding this restriction neither
> > adds additional security where a security manager should be used. I
> > again think this would hurt adoption.
> >
> > I agree with Stephen's argument even though I do not think that
> > forbidding dynamic access would be a feature anybody would use so
> > it should not be added. Rather, something should be integrated in
> > the security manager, if this is considered necessary.
> >
> > Cheers, Rafael
> >
> > Am 30.06.2016 11:37 nachm. schrieb "Nicolai Parlog"
> > <nipa at codefx.org>:
> >>
> >> Hi!
> >>
> >>> In my opinion, reflection should not apply any module-boundary
> >>> checks.
> >>
> >> Maybe this list is not a representative selection of Java
> >> developers, maybe I'm just the only one thinking this but, to
> >> paraphrase:
> >>
> >> Reflection over internals must die in a fire!
> >>
> >> Yes, yes, tools to great things, we need to see everything, ...
> >> Still, I think that modules need a safe space that they can use
> >> without anybody intruding. Maybe I'm a hardliner but I already
> >> view the command line parameters with suspicion.
> >>
> >> In that light, I like the "dynamic" proposal.
> >>
> >> Not the name, though. Having the symmetric pair "static vs
> >> dynamic" looks nice but it takes a lot of words to explain why
> >> exactly these terms apply here. We seem to be looking for
> >> qualifiers for "requires" and "exports". Maybe just using other
> >> terms solves the problem?
> >>
> >> requires & consumes exports & contains
> >>
> >> so long ... Nicolai
> >>
> >>
> >>
> >> On 30.06.2016 01:23, Rafael Winterhalter wrote:
> >>> I also agree here that the use of reflection should not require
> >>> any form of explicit permission by the user of a library. In a
> >>> way, this ties an implementation detail (reflection in this
> >>> case) to the usage of the library. The library cannot work
> >>> without the explicit configuration by a user and this only adds
> >>> boilerplate to using Hibernate.
> >>>
> >>> Unfortunately, many Java developers still do not know much
> >>> about reflection or about the inner workings of many libraries
> >>> in the market. If a user needed to explicitly export packages,
> >>> this might just lead to people "over exporting" their packages
> >>> just to "make Hibernate work".
> >>>
> >>> In my opinion, reflection should not apply any module-boundary
> >>> checks.
> >>>
> >>> Best regards, Rafael
> >>>
> >>> 2016-06-29 20:13 GMT+02:00 Paul Benedict
> >>> <pbenedict at apache.org>:
> >>>
> >>>> Replying to "observers" list...
> >>>>
> >>>> I think the main problem here is that core reflection has
> >>>> become too restrictive. Reflection should be "magical" and
> >>>> have no knowledge of any kind of module and/or visibility
> >>>> boundaries. If you really want to control what can be
> >>>> reflected upon, that should be done via the Policy file of
> >>>> the Security Manager.
> >>>>
> >>>> This proposal is trying to hack out-of-the-box created by
> >>>> too strict of a design. It is very awkward to denote which
> >>>> packages must be reflectable ahead of time --- how can you
> >>>> figure that out?? The consumers who are doing the reflection
> >>>> are innumerable. I'd go as far to say it's it's impossible to
> >>>> devise a preemptive comprehensive strategy that may satisfy
> >>>> the kinds of tooling in the wild.
> >>>>
> >>>> It's best just to open up reflection so it can continue
> >>>> doing what it always has done well: let me see anything I
> >>>> want unless the Security Manager says no.
> >>>>
> >>>> Cheers, Paul
> >>>>
> >>>> On Wed, Jun 29, 2016 at 3:51 AM, Remi Forax
> >>>> <forax at univ-mlv.fr> wrote:
> >>>>
> >>>>>
> >>>>>
> >>>>> ----- Mail original -----
> >>>>>> De: "Mark Reinhold" <mark.reinhold at oracle.com> ?:
> >>>>>> jpms-spec-experts at openjdk.java.net Envoy?: Mardi 28 Juin
> >>>>>> 2016 23:18:15 Objet: Proposal:
> >>>>>> #ReflectiveAccessToNonExportedTypes: `exports dynamic`
> >>>>>>
> >>>>>> Issue summary -------------
> >>>>>>
> >>>>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of
> >>>>>> framework libraries require reflective access to members
> >>>>>> of the non-exported types of other modules; examples
> >>>>>> include dependency injection
> >>>> (Guice),
> >>>>>> persistence (JPA), debugging tools, code-automation
> >>>>>> tools, and serialization (XStream). In some cases the
> >>>>>> particular library to be used is not known until run time
> >>>>>> (e.g., Hibernate and EclipseLink
> >>>> both
> >>>>>> implement JPA). This capability is also sometimes used
> >>>>>> to work
> >>>> around
> >>>>>> bugs in unchangeable code. Access to non-exported
> >>>>>> packages can, at present, only be done via command-line
> >>>>>> flags, which is extremely awkward. Provide an easier way
> >>>>>> for reflective code to access such non-exported types.
> >>>>>> [1]
> >>>>>>
> >>>>>> Proposal --------
> >>>>>>
> >>>>>> Extend the language of module declarations so that a
> >>>>>> package can be declared to be exported at run time but
> >>>>>> not at compile time. This is, roughly, the dual of the
> >>>>>> `requires static` construct proposed for
> >>>>>> #CompileTimeDependences, hence we propose to introduce a
> >>>>>> new modifier, `dynamic`, for use on the `exports`
> >>>>>> directive. It has the following meanings:
> >>>>>>
> >>>>>> - At compile time, `exports dynamic P` does not cause
> >>>>>> the package `P` to be exported, though it does require
> >>>>>> `P` to be a package defined in the module.
> >>>>>>
> >>>>>> - In phases after compile time, `exports dynamic P`
> >>>>>> behaves in
> >>>> exactly
> >>>>>> the same way as `exports P`. It therefore takes part
> >>>>>> fully in resolution and configuration, and is subject to
> >>>>>> the same
> >>>> consistency
> >>>>>> constraints as normally-exported packages (e.g., no
> >>>>>> split
> >>>> packages).
> >>>>>>
> >>>>>> Thus a module declaration of the form
> >>>>>>
> >>>>>> module com.foo.app { requires hibernate.core; requires
> >>>>>> hibernate.entitymanager; exports dynamic
> >>>>>> com.foo.app.model; }
> >>>>>>
> >>>>>> makes the types in the `com.foo.app.model` package
> >>>>>> accessible at run
> >>>> time
> >>>>>> but not at compile time. In combination with the
> >>>>>> earlier change to enable #ReflectionWithoutReadability
> >>>>>> [2] this means that frameworks
> >>>> that
> >>>>>> today use core reflection to manipulate user classes at
> >>>>>> run time are
> >>>> more
> >>>>>> likely to work out-of-the-box, without change, as
> >>>>>> automatic modules.
> >>>> If
> >>>>>> the `com.foo.app.model` package in this example includes
> >>>>>> entity classes to be managed by Hibernate then the
> >>>>>> framework will be able to access
> >>>> them
> >>>>>> without further ado, but under normal circumstances an
> >>>>>> attempt to
> >>>> compile
> >>>>>> code that refers directly to those classes will fail.
> >>>>>>
> >>>>>> The `dynamic` modifier can be applied to both unqualified
> >>>>>> and qualified `exports` directives, though the caveats on
> >>>>>> using qualified exports [3] still apply.
> >>>>>>
> >>>>>> Notes -----
> >>>>>>
> >>>>>> - To access a non-public member in a
> >>>>>> dynamically-exported package you must still invoke the
> >>>>>> appropriate `setAccessible` method, just as you do today
> >>>>>> to access non-public members in an exported package.
> >>>>>>
> >>>>>> - This proposal is similar to R?mi's suggestion to add a
> >>>>>> way to
> >>>> export
> >>>>>> a package only for reflection, still requiring that
> >>>>>> `setAccessible` be invoked [4]. This proposal differs
> >>>>>> in that it does not require `setAccessible` to be invoked
> >>>>>> to access public members in a dynamically-exported
> >>>>>> package.
> >>>>>>
> >>>>>> - This proposal primarily addresses "friendly" uses of
> >>>>>> reflection,
> >>>> such
> >>>>>> as dependency injection and persistence, in which the
> >>>>>> author of a module knows in advance that one or more
> >>>>>> packages must be exported
> >>>> at
> >>>>>> run time for reflective access by frameworks. This
> >>>>>> proposal is
> >>>> also
> >>>>>> convenient for frameworks that generate code at run
> >>>>>> time, since the constant pool of a generated class file
> >>>>>> can include static
> >>>> references
> >>>>>> to types in dynamically exported packages.
> >>>>>>
> >>>>>> - This proposal opens the door to a category of
> >>>>>> second-class APIs.
> >>>> If
> >>>>>> a package is exported dynamically then you can still
> >>>>>> compile code that refers to types in the package, by
> >>>>>> using `-XaddExports` or its equivalent at compile time,
> >>>>>> and it will work as expected at run
> >>>> time.
> >>>>>> It thus may be useful to use `exports dynamic` for
> >>>>>> packages that contain legacy APIs whose use is strongly
> >>>>>> discouraged, e.g., those
> >>>> in
> >>>>>> the `jdk.unsupported` module of the reference
> >>>>>> implementation,
> >>>> thereby
> >>>>>> forcing anyone who wants to compile against them to go
> >>>>>> to the
> >>>> trouble
> >>>>>> of using `-XaddExports`.
> >>>>>>
> >>>>>> - Intrusive access to arbitrary packages of arbitrary
> >>>>>> modules by,
> >>>> e.g.,
> >>>>>> debugging tools, will still require the use of sharp
> >>>>>> knives such as the `-XaddExports` command-line option or
> >>>>>> its equivalent, or JVM
> >>>> TI.
> >>>>>>
> >>>>>> - Using the `-XaddExports` option or its equivalent
> >>>>>> remains awkward, and sometimes it's the only way out. To
> >>>>>> ease migration I think
> >>>> it's
> >>>>>> worth considering some way for an application packaged as
> >>>>>> a JAR
> >>>> file
> >>>>>> to include such options in its `MANIFEST.MF` file, as
> >>>>>> suggested by Simon Nash [5]. I'll create a separate
> >>>>>> issue for that.
> >>>>>>
> >>>>>> - New kinds of resolution failures are possible. If
> >>>>>> module `A` requires `B`, and `B` requires `C`, then if
> >>>>>> `B` and `C` both
> >>>> declare
> >>>>>> some package `P` to be exported dynamically, then a
> >>>>>> split-package error will be reported. This may
> >>>>>> surprise, since the packages are just internal
> >>>>>> implementation details that were exported dynamically so
> >>>>>> that some framework could access their types. This is
> >>>>>> not, however, all that different from the kinds of
> >>>>>> resolution and layer-creation failures that are already
> >>>>>> possible due to package collisions. It's unlikely to
> >>>>>> happen all that often in practice, and the fix is
> >>>>>> straightforward: Just use reverse-DNS or otherwise
> >>>>>> sufficiently-unique package names, as most people already
> >>>>>> do. It is worth exploring whether javac can detect at
> >>>>>> least some of these kinds of collisions and issue
> >>>>>> appropriate warnings.
> >>>>>
> >>>>> I would like to agree on that proposal only on the ground
> >>>>> that it
> >>>> requires
> >>>>> to change the classfile format thus allows in the same time
> >>>>> to fix the issues of the current classfile spec (encoding
> >>>>> of the name module as a Class instead of UTF8, wrong flag
> >>>>> for requires public), but i'm not sure this proposal
> >>>>> support it's own weight.
> >>>>>
> >>>>> Without dynamic, a 'classical' export works the same way
> >>>>> at runtime so adding 'dynamic' to an export only restrict
> >>>>> the usage at compile time. Given that we can already
> >>>>> selectively export packages to some chosen modules which
> >>>>> hide them for the rest of the other modules at compile
> >>>> time,
> >>>>> adding dynamic to the spec adds little value.
> >>>>>
> >>>>> And it has several drawbacks, the main one is that it
> >>>>> weaken the compiletime/runtime fidelity story, with a
> >>>>> dynamic export the runtime
> >>>> world
> >>>>> is not the compiletime world anymore. The main motivation
> >>>>> to add dynamic seems to be for backward compatibility
> >>>>> during the transition between the classpath world and the
> >>>>> full module
> >>>> world,
> >>>>> for several other cases, we use the command line for that,
> >>>>> i don't see
> >>>> why
> >>>>> this case is so special that it should not work like the
> >>>>> other compatibility issues.
> >>>>>
> >>>>> Moreover, an agent, an annotation processor and a jlink
> >>>>> plugin are all able to implement the same trick, add export
> >>>>> after the compilation and before the runtime, so i don't
> >>>>> think it's a good idea to change the spec for that.
> >>>>>
> >>>>> R?mi
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> [1]
> >>>>>>
> >>>>>
> >>>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccess
> >>
> >>>>
> ToNonExportedTypes
> >>>>>>
> >>>>
> >> [2]
> >>>>>>
> >>>>>
> >>>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithou
> >>
> >>>>
> tReadability
> >>>>>>
> >>>>
> >> [3]
> >>>>> http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-export
> >>
> >>>>>
> s
> >>>>>>
> >>>>>
> >> [4]
> >>>>>>
> >>>>>
> >>>> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-Decembe
> >>
> >>>>
> r/000205.html
> >>>>>>
> >>>>
> >> [5]
> >>>>>>
> >>>>>
> >>>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/00574
> >>
> >>>>
> 5.html
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>>>
> >> --
> >>
> >> PGP Key:
> >> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
> >>
> >>
> >>
> Web:
> >> http://codefx.org a blog about software development
> >> http://do-foss.de Free and Open Source Software for the City of
> >> Dortmund
> >>
> >> Twitter: https://twitter.com/nipafx
> >>
> >
>
> --
>
> PGP Key:
> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>
> Web:
> http://codefx.org
> a blog about software development
> http://do-foss.de
> Free and Open Source Software for the City of Dortmund
>
> Twitter:
> https://twitter.com/nipafx
>
>
From scolebourne at joda.org Fri Jul 1 09:27:39 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Fri, 1 Jul 2016 10:27:39 +0100
Subject: Issue #ServiceLoaderEnhancement
Message-ID: <CACzrW9Cp0tpZnU-wF3a+GYg2RBU=LeHixqAiZSFZyJjA10kawg@mail.gmail.com>
The ServiceLoader API is to be pushed as a key class by the module
system. This issue is proposing enhancements to the API. Others may
want to add their own proposals.
Currently, ServiceLoader requires service implementations to have a
public no-args constructor on a public class. This is very
restrictive, especially in a Java 8 world of immutable classes and
factory methods (with constructors actively avoided).
JSR-310 has a service for loading Chronology implementations. In
ThreeTen-Extra, I have a number of additional implementations [1].
Unfortunately, all the implementations are forced to have a public
no-args constructor, as ServiceLoader cannot make use of the singleton
INSTANCE constant. As can be seen, I have to mark the public no-args
constructor as deprecated [2]. This state of affairs is frankly
ridiculous, and is one of the reasons putting me off using
ServiceLoader non-JDK code.
I propose that the spec of ServiceLoader is extended to make it more useful.
Firstly, I propose that all reflection by ServiceLoader should use
setAccessible(true) to allow providers to be non-public.
Secondly, I propose that if no public no-args constructor is found on
the provider class, ServiceLoader should locate a public static
constant named "INSTANCE" and use that instead.
This proposal could be extended further. Instead of searching for
"INSTANCE", it could look for all public static constants of the
provider type, allowing one provider class to result in many provider
instances.
Thirdly, it should be possible to access the set of provider classes
without instantiating them. This would allow applications to use the
module-info design for services without using ServiceLoader, eg to
reflect on the provider classes, perhaps to search for annotations.
I hope this proposal is given serious consideration, as at present the
design of ServiceLoader is simply too restrictive to be appropriate to
be hard coded into the module specification.
Stephen
[1] https://github.com/ThreeTen/threeten-extra/tree/master/src/main/java/org/threeten/extra/chrono
[2] https://github.com/ThreeTen/threeten-extra/blob/master/src/main/java/org/threeten/extra/chrono/CopticChronology.java#L71
From sven.reimers at gmail.com Fri Jul 1 09:34:35 2016
From: sven.reimers at gmail.com (Sven Reimers)
Date: Fri, 1 Jul 2016 11:34:35 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <05815A9E-FAEB-438E-A8F9-A51C9F7DFAC7@luminis.eu>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
<CAP+Jvx4NPxhJVYoi6+ckB88hLm9Hw8B0gACDMVjP81cPc9BoXA@mail.gmail.com>
<05815A9E-FAEB-438E-A8F9-A51C9F7DFAC7@luminis.eu>
Message-ID: <CAP+Jvx5LEA+2y2uj4z7fXWFgpxRa4M1JpYrpOcvF=XM5rvab4w@mail.gmail.com>
The point was if you want to limit reflective access to special modules you
would need some kind of alias or token as the target of your export, e.g.
jpa and not hibernate.
If dynamic export is not limited, all is fine - agreed.
Sven
Am 01.07.2016 10:49 schrieb "Sander Mak" <sander.mak at luminis.eu>:
>
> > On 01 Jul 2016, at 10:41, Sven Reimers <sven.reimers at gmail.com> wrote:
> >
> > Hi all,
> >
> > regarding exporting DTO to hibernate..
> >
> > I think the whole point of having JPA is to not have explicit dependency
> to
> > hibernate..
>
> Hence the proposal for `exports dynamic`, where you don't have to qualify
> the export to a certain module. The relation between the DTO module and JPA
> would be expressed by having a `requires jpa` (or whatever module the spec
> annotations will live in). The `dynamic` part makes sure no other modules
> take an accidental dependency on your DTOs (if that's what you want) at
> compile-time. All sounds good to me.
>
>
> Sander
>
>
From nipa at codefx.org Fri Jul 1 09:48:59 2016
From: nipa at codefx.org (Nicolai Parlog)
Date: Fri, 1 Jul 2016 11:48:59 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <CAP+Jvx5LEA+2y2uj4z7fXWFgpxRa4M1JpYrpOcvF=XM5rvab4w@mail.gmail.com>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
<CAP+Jvx4NPxhJVYoi6+ckB88hLm9Hw8B0gACDMVjP81cPc9BoXA@mail.gmail.com>
<05815A9E-FAEB-438E-A8F9-A51C9F7DFAC7@luminis.eu>
<CAP+Jvx5LEA+2y2uj4z7fXWFgpxRa4M1JpYrpOcvF=XM5rvab4w@mail.gmail.com>
Message-ID: <57763C8B.3010400@codefx.org>
Hi Sven,
yes, I was not very precise. The "problem" (in scare quotes because it
is only a problem to the JPMS) with Hibernate is exactly that it is
"only" an implementation, which we should be able to replace with any
other at link time.
So modules should of course not have to "export to Hibernate". This
was the original problem with qualified exports.
The current proposal will fix this by allowing me to export to anybody
but at run time only, which will keep people from depending on it at
compile time.
so long ... Nicolai
On 01.07.2016 11:34, Sven Reimers wrote:
> The point was if you want to limit reflective access to special
> modules you would need some kind of alias or token as the target of
> your export, e.g. jpa and not hibernate.
>
> If dynamic export is not limited, all is fine - agreed.
>
> Sven Am 01.07.2016 10:49 schrieb "Sander Mak"
> <sander.mak at luminis.eu>:
>
>>
>>> On 01 Jul 2016, at 10:41, Sven Reimers <sven.reimers at gmail.com>
>>> wrote:
>>>
>>> Hi all,
>>>
>>> regarding exporting DTO to hibernate..
>>>
>>> I think the whole point of having JPA is to not have explicit
>>> dependency
>> to
>>> hibernate..
>>
>> Hence the proposal for `exports dynamic`, where you don't have to
>> qualify the export to a certain module. The relation between the
>> DTO module and JPA would be expressed by having a `requires jpa`
>> (or whatever module the spec annotations will live in). The
>> `dynamic` part makes sure no other modules take an accidental
>> dependency on your DTOs (if that's what you want) at
>> compile-time. All sounds good to me.
>>
>>
>> Sander
>>
>>
>
--
PGP Key:
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
Web:
http://codefx.org
a blog about software development
http://do-foss.de
Free and Open Source Software for the City of Dortmund
Twitter:
https://twitter.com/nipafx
From blackdrag at gmx.org Fri Jul 1 10:20:34 2016
From: blackdrag at gmx.org (Jochen Theodorou)
Date: Fri, 1 Jul 2016 12:20:34 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <CA+DM0AnfqM9j21ULBr61n3swWD605gaAeitDqZxYqtJzXwF56w@mail.gmail.com>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
<CA+DM0AnfqM9j21ULBr61n3swWD605gaAeitDqZxYqtJzXwF56w@mail.gmail.com>
Message-ID: <577643F2.1080407@gmx.org>
On 01.07.2016 11:15, Rafael Winterhalter wrote:
[...]
> Finally, I do not believe the service loader API offers an adequate
> alternative to many use-cases of reflection (while it can perfectly
> substitute some of them). Many state-of-the art libraries actually
> intentionally offer POJO approaches without requiring their users to create
> interfaces for any shared interaction. I do not think that a gentle push
> into a direction that contradicts most existing Java libraries would be a
> taken well by the people implementing this software and in the end, the
> module system will be as successful as its adoption. I rather see people
> finding new ways around restrictions to keep things simple (for themselves).
even in the cases the service API is a replacement, the usage of the
library will change and a breaking change is forced.
> For all these reasons I do not think that it is a good idea to limit
> reflection to exported packages.
or people will start just exporting everything.
bye Jochen
From rafael.wth at gmail.com Fri Jul 1 10:54:52 2016
From: rafael.wth at gmail.com (Rafael Winterhalter)
Date: Fri, 1 Jul 2016 12:54:52 +0200
Subject: Proposal: #ResourceEncapsulation and #ClassFilesAsResources
In-Reply-To: <5775b10a.c135620a.69615.ffff9dd0SMTPIN_ADDED_BROKEN@mx.google.com>
References: <20160628212015.99A06B59CC@eggemoggin.niobe.net>
<OF87CFDA2A.733F57C7-ON80257FE1.0037363B-80257FE1.00385D62@notes.na.collabserv.com>
<5775b10a.c135620a.69615.ffff9dd0SMTPIN_ADDED_BROKEN@mx.google.com>
Message-ID: <CA+DM0AnJR2FBcnP1ixV6eAB-hmFT_SOYZ8zHLDBzUpvBB3E8gw@mail.gmail.com>
I just checked a customer application that uses Apache Wicket. The
application has a widget (Maven) module that currently includes 173
packages where each package contains such files that would need to be
exported explicitly. I do not think that exporting each package would be a
realistic solution (you can only find out if your forgot a single package
at runtime) and most likely, the application would opt-out of
modularization. (I do not claim this is well-organized software, but in my
experience most enterprise applications are structured rather "organic"
after "urgent" customer demands.)
My main concern is that with explicit exports, some people would easily
adapt "publishing all resources" as a strange best practice. (Similarly to
how I argued with regards to reflection:
http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-July/000451.html).
Also - if the proposal would include class files - this would again break
libraries like Javassist (
http://mvnrepository.com/artifact/org.javassist/javassist/usages) or Byte
Buddy (http://mvnrepository.com/artifact/net.bytebuddy/byte-buddy/usages)
and all of their users. As a heavy user of Javaagents (I contracted with
different APM and tools vendors), I can only warn that this would be a huge
set-back to the platform.
In my eyes, the main advantage of shielding modules-bundled resources is to
avoid conflicts, e.g. the duplicate existance of some file
"/properties.json" in two modularized jars where the file is only relevant
to each particular module. This problem is however already solved by the
added Module::getResourceAsAStream API where I can explicitly request "my"
file from "my" module. As for inter-module interaction, I do not think it
is feasible to anticipate all established use-patterns and I want to
suggest to not limit visibility when reading resources via a ClassLoader.
2016-07-01 1:52 GMT+02:00 <mark.reinhold at oracle.com>:
> 2016/6/29 3:15:38 -0700, tim_ellison at uk.ibm.com:
> > mark.reinhold at oracle.com wrote on 28/06/2016 22:20:15:
> >>
> >> ...
> >>
> >> Proposal
> >> --------
> >>
> >> Drop the agreed resource-encapsulation requirement, which reads:
> >>
> >> Resource encapsulation --- The run-time system must ensure that the
> >> static resource files within a module are directly accessible only by
> >> code within that module. The existing resource-access APIs should
> >> continue to work as they do today when used to access a module's own
> >> resource files. [2]
> >>
> >> ...
> >
> > The proposal to drop this requirement leaves no way for a module to
> encapsulate
> > its resources. Have you considered an alternative that preserves the
> compatibility
> > of existing APIs which I agree is desirable to support existing
> applications,
> > but introduces a module-level resource API that allows those modules
> that want
> > to hide their implementation details?
>
> There are certainly other alternatives; what I proposed is just the
> simplest.
>
> One alternative is just to address the two specific use cases. Make
> META-INF/**, WEB-INF/**, and the class file for any class visible to
> the caller available via the resource-access APIs, but make all other
> resources module-private.
>
> This wouldn't address other use cases that require non-class-file
> resources in normal package directories to be available, of which the
> Wicket web framework is an example (pointed out by Rafael Winterhalter
> [1]). So a second alternative would be to go further and define a way
> to declare which packages of a module are considered to "export" their
> resources, similar to what Peter Levart suggested [2]). Maybe something
> like this:
>
> module com.foo.app {
> requires wicket.core;
> requires wicket.utils;
> publishes com.foo.app.pages.main;
> publishes com.foo.app.pages.aux;
> }
>
> would make resources in the com/foo/app/pages/{main,aux} directories of
> the module's artifact available for anyone to load. By fiat we'd make
> everything under {META,WEB}-INF available, for compatibility. Whether
> access to the class files for visible classes should be limited further,
> or perhaps solely, by this mechanism, I'm not sure.
>
> So yes, there are some alternatives here. Thoughts welcome.
>
> - Mark
>
>
> [1]
> http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-June/000417.html
> [2]
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-June/008355.html
>
From pbenedict at apache.org Fri Jul 1 15:21:38 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Fri, 1 Jul 2016 10:21:38 -0500
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <577643F2.1080407@gmx.org>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
<CA+DM0AnfqM9j21ULBr61n3swWD605gaAeitDqZxYqtJzXwF56w@mail.gmail.com>
<577643F2.1080407@gmx.org>
Message-ID: <CABLGb9xDZOG5poMckKB9xPdQf-_0FC-KHnOup-+cGP8-JcNG-A@mail.gmail.com>
Since Hibernate is a popular example for this topic, let's talk about it.
First, when I build JPA applications, I do have an implementation use in
mind (Hibernate, EclipseLink, etc.), but my code is standardized JPA as
much as possible. So if I am going to publish my JPA model and I want my
private types reflected to be loaded as entities, who am I going to export
it too?
Well I certainly am not going to export it to "jpa" because that's just a
specification...
The next logical answer would be the "hibernate" module, but now I'm
gambling on knowing the internals of Hibernate... I have used Hibernate
professionally for several years, but, off the top of my head, I honestly
couldn't tell you if HIbernate has its own reflection routines or if it
delegates it to third party libraries like "cglib" and "javassist". My gut
tells me it's the latter, but don't quote me on it :-)
So now "dynamic" has put me in a pickle. All I really want to do is have
Hibernate access my private types for JPA, but I have to roll several dice
and pray to be right:
(1) I got to know if Hibernate does its reflection or not
(2) Does Hibernate delegate to third party reflection libraries?
(3) Are the third party reflection libraries pluggable?
(4) What is going to happen to my code if Hibernate alters its reflection
mechanism?
#4 is pretty important, actually. Tools have been known to shade some of
their third party dependencies. So while, for example, Hibernate may rely
on "cglib" or "javassist" today, there is a possibility those may be shaded
and brought in and another package name in a future version. I've seen
those discussions. I am not implying what was or wasn't decided, but just
it's a possibility.
I just don't see a point in creating such a risky situation for myself. I'd
rather just have everything exported all the time to every module if this
is the way reflection is going.
Cheers,
Paul
On Fri, Jul 1, 2016 at 5:20 AM, Jochen Theodorou <blackdrag at gmx.org> wrote:
>
>
> On 01.07.2016 11:15, Rafael Winterhalter wrote:
> [...]
>
>> Finally, I do not believe the service loader API offers an adequate
>> alternative to many use-cases of reflection (while it can perfectly
>> substitute some of them). Many state-of-the art libraries actually
>> intentionally offer POJO approaches without requiring their users to
>> create
>> interfaces for any shared interaction. I do not think that a gentle push
>> into a direction that contradicts most existing Java libraries would be a
>> taken well by the people implementing this software and in the end, the
>> module system will be as successful as its adoption. I rather see people
>> finding new ways around restrictions to keep things simple (for
>> themselves).
>>
>
> even in the cases the service API is a replacement, the usage of the
> library will change and a breaking change is forced.
>
> For all these reasons I do not think that it is a good idea to limit
>> reflection to exported packages.
>>
>
> or people will start just exporting everything.
>
> bye Jochen
>
From mark.reinhold at oracle.com Fri Jul 1 17:10:55 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Fri, 01 Jul 2016 10:10:55 -0700
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <1720977211.1766068.1467362887039.JavaMail.zimbra@u-pem.fr>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<20160629173559.18960061eggemoggin.niobe.net>
<1720977211.1766068.1467362887039.JavaMail.zimbra@u-pem.fr>
Message-ID: <20160701101055.285920310eggemoggin.niobe.net>
2016/7/1 1:48:07 -0700, Remi Forax <forax at univ-mlv.fr>:
> mark.reinhold at oracle.com:
>> ...
>>
>> The nice thing about `exports dynamic` is that it lets you say "just
>> make the types in this package available for reflection by anybody, I'm
>> okay with that".
>
> There are two differences between unqualified export and exports dynamic:
> - exported types are only available at runtime not at compile time,
> - exported types are available by anybody so a module like hibernate
> doesn't have to requires a user module.
This second point is not correct: Using `exports dynamic` in a module
declaration does not mean that all other modules will automatically read
that module. In module declarations, `exports` directives do not affect
readability; only `requires` does that. Both normal, unqualified
exports and dynamic exports make the public types in a package available
to modules that read the exporting module.
In the case of a framework like Hibernate, which uses reflection, the
required readability will come "for free" thanks to the resolution of
the #ReflectionWithoutReadability issue [1] (your suggestion).
> I fail to see why these two guys are bundled together, i may want to
> have exported types available to anybody and visible at compile time.
> but i understand that having a way to say i want to export a package
> to anybody is interesting especially for library like hibernate.
>
> Moreover, i don't like the fact that Java the language has a new way
> to interact with the reflection. The more we introduce such coupling
> the less we will be able to move in the future.
We've already introduced new ways for the language to interact with
reflection, in fact in the solution to #ReflectionWithoutReadability.
At any rate, the relationship between the language and the various
reflective APIs is, almost by definition, very intimate, so I don't
think we should be surprised that revisions to the language introduce
new interactions in this relationship.
Do you have specific examples in mind of undesirable coupling due to
this new interaction?
> so instead of exports dynamic i propose to introduce a way to say i
> want to export a package to anybody, an exports to * (star).
>
> Using the example of the initial proposal, the module-info will be
>
> module com.foo.app {
> requires hibernate.core;
> requires hibernate.entitymanager;
> exports com.foo.app.model to *;
> }
This is exactly what `exports com.foo.app.model` already does today.
It exports the public types in that package to any module that reads
the `com.foo.app` module. There's no need to add `to *`.
Whether some other module reads `com.foo.app` is, as I wrote above,
an issue that's orthogonal to whether and how any particular package
is exported, and in the case of reflective access is a non-issue since
reflection now simply assumes readability.
>> Remi Forax <forax at univ-mlv.fr>:
>>> ...
>>>
>>> And it has several drawbacks, the main one is that it weaken the
>>> compiletime/runtime fidelity story, with a dynamic export the runtime
>>> world is not the compiletime world anymore. The main motivation to
>>> add dynamic seems to be for backward compatibility during the
>>> transition between the classpath world and the full module world, for
>>> several other cases, we use the command line for that, i don't see why
>>> this case is so special that it should not work like the other
>>> compatibility issues.
>>
>> Yes, it does weaken the fidelity story in order to accommodate existing
>> practice, but at least it does so in an explicit manner so that what's
>> going on is apparent just from reading the code.
>>
>> I don't think, however, that this change is just about compatibility and
>> migration. Even in an idealistic, Utopian, fully-modular world I expect
>> that we'll still want to use frameworks that use reflection on our own
>> types to help us be more productive.
>
> yes, that's very true. Reflection allows to write meta object
> protocols but making a special case for the reflection seems wrong to
> me.
Just to be clear, the `exports dynamic` idea is motivated by existing
uses of reflection, but it's not intimately tied to reflection. The
specification won't directly tie `exports dynamic` to the reflection
APIs. Those APIs will, however, work more naturally as a consequence
of the semantics of `exports dynamic`, which will be defined entirely
in terms of the modular access-control concepts we already have.
- Mark
[1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithoutReadability
From mark.reinhold at oracle.com Fri Jul 1 17:43:15 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Fri, 01 Jul 2016 10:43:15 -0700
Subject: Proposal: #FixClassFileFormat
In-Reply-To: <1751220489.1722677.1467358740078.JavaMail.zimbra@u-pem.fr>
References: <1813643129.1709819.1467357511504.JavaMail.zimbra@u-pem.fr>
<1751220489.1722677.1467358740078.JavaMail.zimbra@u-pem.fr>
Message-ID: <20160701104315.789031669eggemoggin.niobe.net>
2016/7/1 0:39:00 -0700, Remi Forax <forax at univ-mlv.fr>:
> mark.reinhold at oracle.com:
>> 2016/6/29 1:51:16 -0700, Remi Forax <forax at univ-mlv.fr>:
>>> I would like to agree on that proposal only on the ground that it
>>> requires to change the classfile format thus allows in the same time
>>> to fix the issues of the current classfile spec (encoding of the name
>>> module as a Class instead of UTF8, wrong flag for requires public),
>>
>> I think that these are completely distinct issues (though your
>> class-file issues are not yet in the issue list -- would you care
>> to suggest some text?).
>
> yes, sure.
>
> when implementing the support of module into ASM, i've discovered
> several issues in the classfile format as defined by [1]
>
> - module name is not encoded as UTF8.
>
> The spec makes great cares to encode reference to module in
> 'requires' by example as "UTF8" constant pool entry, a name with no
> other constraints, but fail to do the same thing for the name of the
> module itself.
>
> The name of the module is derived from the name of the class which
> is encoded as a "Class" constant pool entry. The way to solve that
> is to avoid to derive the name of the module from the class, the
> name of the class should be module-info, with no package, which
> seems logical given that the source file is module-info.java (not
> prefixed by a package) and the name of the module should be
> encoded at the beginning of the "Module" attribute as a reference to
> the constant pool entry of type UTF8. This format can also be more
> compact despite adding two bytes in the "Module" attribute in the
> fairly common case where one exported package as the same name as
> the module name because of the constant poll sharing the names.
>
> - there is what i believe is a typo in the current spec when
> specifying the modifier of a required module (requires_flags), the
> modifier ACC_PUBLIC is said to have the value 0x0020 instead of
> 0x0001 in the JVM spec. (0x0020 is either ACC_SUPER or
> ACC_SYNCHRONIZED).
>
> - the attributes for specifying the main-class, the module-version, a
> concealed module, etc are not specified, and i don't know if there
> is a reason to not specifying them.
On this last point: They should be specified, and will be specified in
an upcoming revision to the spec.
Before digging into the other two issues I'd like to try just to record
them, without any bias toward a particular solution, so I'll add the
following two items to the issue list:
#ClassFileModuleName --- The name of a module is not a simple UTF-8
string but is, rather, derived from the `this_class` field of the
`ClassFile` structure, which is awkward.
#ClassFileRequires --- The `ACC_PUBLIC` constant in a `requires_flags`
should be encoded as `0x0001`, as it is elsewhere in the JVMS, rather
than as `0x0020`, which has different meanings in other contexts.
- Mark
From forax at univ-mlv.fr Fri Jul 1 20:20:25 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Fri, 1 Jul 2016 22:20:25 +0200 (CEST)
Subject: Proposal: #FixClassFileFormat
In-Reply-To: <20160701104315.789031669eggemoggin.niobe.net>
References: <1813643129.1709819.1467357511504.JavaMail.zimbra@u-pem.fr>
<1751220489.1722677.1467358740078.JavaMail.zimbra@u-pem.fr>
<20160701104315.789031669eggemoggin.niobe.net>
Message-ID: <321002664.2003394.1467404425232.JavaMail.zimbra@u-pem.fr>
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Vendredi 1 Juillet 2016 19:43:15
> Objet: Re: Proposal: #FixClassFileFormat
>
> 2016/7/1 0:39:00 -0700, Remi Forax <forax at univ-mlv.fr>:
> > mark.reinhold at oracle.com:
> >> 2016/6/29 1:51:16 -0700, Remi Forax <forax at univ-mlv.fr>:
> >>> I would like to agree on that proposal only on the ground that it
> >>> requires to change the classfile format thus allows in the same time
> >>> to fix the issues of the current classfile spec (encoding of the name
> >>> module as a Class instead of UTF8, wrong flag for requires public),
> >>
> >> I think that these are completely distinct issues (though your
> >> class-file issues are not yet in the issue list -- would you care
> >> to suggest some text?).
> >
> > yes, sure.
> >
> > when implementing the support of module into ASM, i've discovered
> > several issues in the classfile format as defined by [1]
> >
> > - module name is not encoded as UTF8.
> >
> > The spec makes great cares to encode reference to module in
> > 'requires' by example as "UTF8" constant pool entry, a name with no
> > other constraints, but fail to do the same thing for the name of the
> > module itself.
> >
> > The name of the module is derived from the name of the class which
> > is encoded as a "Class" constant pool entry. The way to solve that
> > is to avoid to derive the name of the module from the class, the
> > name of the class should be module-info, with no package, which
> > seems logical given that the source file is module-info.java (not
> > prefixed by a package) and the name of the module should be
> > encoded at the beginning of the "Module" attribute as a reference to
> > the constant pool entry of type UTF8. This format can also be more
> > compact despite adding two bytes in the "Module" attribute in the
> > fairly common case where one exported package as the same name as
> > the module name because of the constant poll sharing the names.
> >
> > - there is what i believe is a typo in the current spec when
> > specifying the modifier of a required module (requires_flags), the
> > modifier ACC_PUBLIC is said to have the value 0x0020 instead of
> > 0x0001 in the JVM spec. (0x0020 is either ACC_SUPER or
> > ACC_SYNCHRONIZED).
> >
> > - the attributes for specifying the main-class, the module-version, a
> > concealed module, etc are not specified, and i don't know if there
> > is a reason to not specifying them.
>
> On this last point: They should be specified, and will be specified in
> an upcoming revision to the spec.
>
> Before digging into the other two issues I'd like to try just to record
> them, without any bias toward a particular solution, so I'll add the
> following two items to the issue list:
>
> #ClassFileModuleName --- The name of a module is not a simple UTF-8
> string but is, rather, derived from the `this_class` field of the
> `ClassFile` structure, which is awkward.
>
> #ClassFileRequires --- The `ACC_PUBLIC` constant in a `requires_flags`
> should be encoded as `0x0001`, as it is elsewhere in the JVMS, rather
> than as `0x0020`, which has different meanings in other contexts.
>
yes !
and i've forgotten a third issue, ACC_MODULE is specified as 0x8000,
0x8000 is very special value because it's the only one that left which is available on class, method and field.
Historically, in the first specs, "module" was a modifier like public or private thus using the same value for all the elements was making sense,
but now ACC_MODULE can only applied on class so using a value available on all the elements make less sense, the value 0x8000 should be free in order to be used later.
The possible value for ACC_MODULE are: 0x0008 (ACC_STATIC), 0x0040 (ACC_VOLATILE or ACC_BRIDGE), 0x0080 (ACC_VARARGS or ACC_TRANSIENT), 0x0100 (ACC_NATIVE), 0x0800 (ACC_STRICT).
So either, 0x0040 or 0x0080 are the best candidate because the other values are only valid either on field or on method but not on both. And now, because 0x0080 is a permutation of 0x8000 in term of characters , i think 0x0040 is the best value for ACC_MODULE.
Changing the value of ACC_MODULE is interesting if it's done at the same time as the other changes because the compiler and the VM can easily detect and reject the classfile encoded with what will become an old format.
> - Mark
>
R?mi
From volker.berlin at goebel-clan.de Sat Jul 2 22:09:21 2016
From: volker.berlin at goebel-clan.de (Volker Berlin)
Date: Sun, 3 Jul 2016 00:09:21 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <577588EA.6060202@oracle.com>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<e432a1f4-603d-17f0-406e-e53c961db756@goebel-clan.de>
<577588EA.6060202@oracle.com>
Message-ID: <4ab19576-4e42-d77a-7669-48ee319e694a@goebel-clan.de>
Alex,
It is to compensate bugs or handle bad design of JDK components. Mostly
this are classloader problems. The impossible to make the JDK classes
via dynamic URLClassLoader available.
For example:
* register dynamically a JDBC that is not in the appclassloader to
all our plugins.
* register a URL protocol with a protocol handler that is not in the
appclassloader
Many hacks are needed for test environments. For example:
* the simulation that there is no console (javaw)
* inverse classloder hierarchy. In the appclassloader there is a class
which implements/extends a class in a dynamic loaded library. The
current tested classes and the JUnit tests are typical in the
appclassloader
This are only some points. There can be more. We have used.
Volker Berlin
i-net software
Am 30.06.2016 um 23:02 schrieb Alex Buckley:
> Volker,
>
> Please share details of why your application needs to reflect over the
> internals of another module.
>
> Alex
>
> On 6/30/2016 1:09 PM, Volker Berlin wrote:
>> Reflection should be enabled by default. My current launcher for Java 9
>> check if the needed refection is possible. If not possible then it
>> restart the application with a very long list of -XaddExports switches.
>> I think this is not what we want.
>>
>> Volker Berlin
>> i-net software
>>
>>
>> Am 28.06.2016 um 23:18 schrieb Mark Reinhold:
>>> Issue summary
>>> -------------
>>>
>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of framework
>>> libraries require reflective access to members of the non-exported
>>> types of other modules; examples include dependency injection
>>> (Guice),
>>> persistence (JPA), debugging tools, code-automation tools, and
>>> serialization (XStream). In some cases the particular library to be
>>> used is not known until run time (e.g., Hibernate and EclipseLink
>>> both
>>> implement JPA). This capability is also sometimes used to work
>>> around
>>> bugs in unchangeable code. Access to non-exported packages can, at
>>> present, only be done via command-line flags, which is extremely
>>> awkward. Provide an easier way for reflective code to access such
>>> non-exported types. [1]
>>>
>>> Proposal
>>> --------
>>>
>>> Extend the language of module declarations so that a package can be
>>> declared to be exported at run time but not at compile time. This is,
>>> roughly, the dual of the `requires static` construct proposed for
>>> #CompileTimeDependences, hence we propose to introduce a new modifier,
>>> `dynamic`, for use on the `exports` directive. It has the following
>>> meanings:
>>>
>>> - At compile time, `exports dynamic P` does not cause the package
>>> `P`
>>> to be exported, though it does require `P` to be a package defined
>>> in the module.
>>>
>>> - In phases after compile time, `exports dynamic P` behaves in
>>> exactly
>>> the same way as `exports P`. It therefore takes part fully in
>>> resolution and configuration, and is subject to the same
>>> consistency
>>> constraints as normally-exported packages (e.g., no split
>>> packages).
>
From nipa at codefx.org Sat Jul 2 22:50:59 2016
From: nipa at codefx.org (Nicolai Parlog)
Date: Sun, 3 Jul 2016 00:50:59 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <CABLGb9xDZOG5poMckKB9xPdQf-_0FC-KHnOup-+cGP8-JcNG-A@mail.gmail.com>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
<CA+DM0AnfqM9j21ULBr61n3swWD605gaAeitDqZxYqtJzXwF56w@mail.gmail.com>
<577643F2.1080407@gmx.org>
<CABLGb9xDZOG5poMckKB9xPdQf-_0FC-KHnOup-+cGP8-JcNG-A@mail.gmail.com>
Message-ID: <57784553.5090100@codefx.org>
Hi Paul,
either I'm misunderstanding you or one of us misunderstood the proposal.
> So now "dynamic" has put me in a pickle. All I really want to do
> is have Hibernate access my private types for JPA, but I have to
> roll several dice and pray to be right: (1) I got to know if
> Hibernate does its reflection or not (2) Does Hibernate delegate to
> third party reflection libraries? (3) Are the third party
> reflection libraries pluggable? (4) What is going to happen to my
> code if Hibernate alters its reflection mechanism?
You seem to be talking about qualified exports, which let you export
packages _to_ a specific module (at compile and at run time). Dynamic
exports, on the other hand, will export packages to every module
(although I assume there would also be qualified dynamic exports) but
only at run time.
So your module will simply "export dynamic package.with.dtos" and
every module can access those types at runtime if it reads the your
module. If it uses reflection to access some DTOs, the read edge is
added automatically.
To me this seems like a clean solution where
(a) nothing changes for Hibernate
(b) a module can expose it's DTOs at run time
while preventing compile-time dependencies
so long ... Nicolai
On 01.07.2016 17:21, Paul Benedict wrote:
> Since Hibernate is a popular example for this topic, let's talk
> about it.
>
> First, when I build JPA applications, I do have an implementation
> use in mind (Hibernate, EclipseLink, etc.), but my code is
> standardized JPA as much as possible. So if I am going to publish
> my JPA model and I want my private types reflected to be loaded as
> entities, who am I going to export it too?
>
> Well I certainly am not going to export it to "jpa" because that's
> just a specification...
>
> The next logical answer would be the "hibernate" module, but now
> I'm gambling on knowing the internals of Hibernate... I have used
> Hibernate professionally for several years, but, off the top of my
> head, I honestly couldn't tell you if HIbernate has its own
> reflection routines or if it delegates it to third party libraries
> like "cglib" and "javassist". My gut tells me it's the latter, but
> don't quote me on it :-)
>
> So now "dynamic" has put me in a pickle. All I really want to do is
> have Hibernate access my private types for JPA, but I have to roll
> several dice and pray to be right: (1) I got to know if Hibernate
> does its reflection or not (2) Does Hibernate delegate to third
> party reflection libraries? (3) Are the third party reflection
> libraries pluggable? (4) What is going to happen to my code if
> Hibernate alters its reflection mechanism?
>
> #4 is pretty important, actually. Tools have been known to shade
> some of their third party dependencies. So while, for example,
> Hibernate may rely on "cglib" or "javassist" today, there is a
> possibility those may be shaded and brought in and another package
> name in a future version. I've seen those discussions. I am not
> implying what was or wasn't decided, but just it's a possibility.
>
> I just don't see a point in creating such a risky situation for
> myself. I'd rather just have everything exported all the time to
> every module if this is the way reflection is going.
>
>
> Cheers, Paul
>
> On Fri, Jul 1, 2016 at 5:20 AM, Jochen Theodorou
> <blackdrag at gmx.org> wrote:
>
>>
>>
>> On 01.07.2016 11:15, Rafael Winterhalter wrote: [...]
>>
>>> Finally, I do not believe the service loader API offers an
>>> adequate alternative to many use-cases of reflection (while it
>>> can perfectly substitute some of them). Many state-of-the art
>>> libraries actually intentionally offer POJO approaches without
>>> requiring their users to create interfaces for any shared
>>> interaction. I do not think that a gentle push into a direction
>>> that contradicts most existing Java libraries would be a taken
>>> well by the people implementing this software and in the end,
>>> the module system will be as successful as its adoption. I
>>> rather see people finding new ways around restrictions to keep
>>> things simple (for themselves).
>>>
>>
>> even in the cases the service API is a replacement, the usage of
>> the library will change and a breaking change is forced.
>>
>> For all these reasons I do not think that it is a good idea to
>> limit
>>> reflection to exported packages.
>>>
>>
>> or people will start just exporting everything.
>>
>> bye Jochen
>>
>
--
PGP Key:
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
Web:
http://codefx.org
a blog about software development
http://do-foss.de
Free and Open Source Software for the City of Dortmund
Twitter:
https://twitter.com/nipafx
From nipa at codefx.org Sat Jul 2 23:02:03 2016
From: nipa at codefx.org (Nicolai Parlog)
Date: Sun, 3 Jul 2016 01:02:03 +0200
Subject: Proposal: #CompileTimeDependences: `requires static`
In-Reply-To: <CE0BB143-9FF5-4D4A-8730-24D4FC84A1DD@luminis.eu>
References: <20160628211715.84610B59BF@eggemoggin.niobe.net>
<1808187852.884711.1467154252074.JavaMail.zimbra@u-pem.fr>
<20160628163150.929695476eggemoggin.niobe.net>
<57760387.9040405@codefx.org>
<CE0BB143-9FF5-4D4A-8730-24D4FC84A1DD@luminis.eu>
Message-ID: <577847EB.9080609@codefx.org>
Hi Sander,
I think qualifiers to "exports" and "requires" would be nice but new
terms (instead of the proposed "exports dynamic" and "requires
static") might fix the problem as well (maybe even better) and should
hence be considered. A discussion going down this route could be
worthwhile.
My proposal was a strawman, shot from the hip, so I'm not going to
defend it much. :) But I want to clarify my intent.
"consumes" was proposed to replace "requires static". The rationale
being, that "requires" obviously says "I need this" while "consumes"
sounds much more relaxed: "I'll use it if it's there", which is
precisely what we want. Your argument regarding services is absolutely
valid, though.
Similarly "contains" was supposed to replace "exports dynamic". Two
things:
(a) "exports x.y.z" basically reads as "exports package x.y.z"
(b) to the outside world packages that are not mentioned in
module-info.java might as well not even exist.
So I thought that "contains" would say something like "Look, this
package is here; I'm not exporting it but if you find a way around
that limitation, be my guest to use it".
So I do not think that this clashes with "conceals".
What do you think?
Nicolai
On 01.07.2016 10:41, Sander Mak wrote:
>>> requires & consumes exports & contains
>>
>> So it would look as follows:
>>
>> module com.xyz { requires guava; requires joda; consumes
>> hibernate; exports com.xyz.a; exports com.xyz.b; contains
>> com.xyz.c; }
>>
>
> Going down this route, I would avoid 'consumes' since IMO it
> appears to be the inverse of 'provides' used with services. Also,
> the the proposed 'contains' seems to be equivalent to what is now
> called 'conceals' (when looking at output of for example `java
> -listmods:java.base`), which I find to be more descriptive. Or did
> you mean that 'contains' replaces 'exports dynamic'? In that case,
> I don't really see how that's any clearer.
>
>
> Sander
>
>
>
--
PGP Key:
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
Web:
http://codefx.org
a blog about software development
http://do-foss.de
Free and Open Source Software for the City of Dortmund
Twitter:
https://twitter.com/nipafx
From rafael.wth at gmail.com Mon Jul 4 09:36:20 2016
From: rafael.wth at gmail.com (Rafael Winterhalter)
Date: Mon, 4 Jul 2016 11:36:20 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <57784553.5090100@codefx.org>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
<CA+DM0AnfqM9j21ULBr61n3swWD605gaAeitDqZxYqtJzXwF56w@mail.gmail.com>
<577643F2.1080407@gmx.org>
<CABLGb9xDZOG5poMckKB9xPdQf-_0FC-KHnOup-+cGP8-JcNG-A@mail.gmail.com>
<57784553.5090100@codefx.org>
Message-ID: <CA+DM0Amp9TQADR9B=NX-1YLKcp8dsFFtUzA9v=6TRyfmGQuW8Q@mail.gmail.com>
This statement is where I disagree: So your module will simply "export
dynamic package.with.dtos"
If you need to explicitly white-list packages that are accessible via
reflection, this makes things more difficult for most users while adding a
minimum benefit for very few users (i.e. making it more difficult to "jail
break" an internal API using reflection). In practice, dynamic exports
would not require to export one package but most likely a whole bunch of
packages where finding out about what packages to white-list would require
quite some effort and the correctnes could only be validated at runtime.
I just checked what this would mean for one of our production applications
by setting a break point in Method::invoke and we would probably choose to
not apply modules rather than adding every single package as a dynamic
export as there are so many libraries that use reflection and we have no
overview how this is organized. Again, I do not argue that this is
perfectly structured software but I think this is still representative of
the reality of most enterprise apps.
Finally, I do not believe that the benefit of protecting internal APIs is
as big as it is argued here. Most of the time, reflection is used by
libraries to access my code where reflection is used for (annotation-based)
discovery and susequent interaction. The other way around, very few
developers use reflection to access internal APIs and those mostly know
what it means to circumvent offical APIs. In this case, reflection is
really only used as a last resort where there is no other way like in the
cases of JUnit or Unsafe.
I therefore still hope Java will continue to allow reflection across module
boundaries without requireing any additional configuration.
2016-07-03 0:50 GMT+02:00 Nicolai Parlog <nipa at codefx.org>:
> Hi Paul,
>
> either I'm misunderstanding you or one of us misunderstood the proposal.
>
> > So now "dynamic" has put me in a pickle. All I really want to do
> > is have Hibernate access my private types for JPA, but I have to
> > roll several dice and pray to be right: (1) I got to know if
> > Hibernate does its reflection or not (2) Does Hibernate delegate to
> > third party reflection libraries? (3) Are the third party
> > reflection libraries pluggable? (4) What is going to happen to my
> > code if Hibernate alters its reflection mechanism?
>
> You seem to be talking about qualified exports, which let you export
> packages _to_ a specific module (at compile and at run time). Dynamic
> exports, on the other hand, will export packages to every module
> (although I assume there would also be qualified dynamic exports) but
> only at run time.
>
> So your module will simply "export dynamic package.with.dtos" and
> every module can access those types at runtime if it reads the your
> module. If it uses reflection to access some DTOs, the read edge is
> added automatically.
>
> To me this seems like a clean solution where
>
> (a) nothing changes for Hibernate
> (b) a module can expose it's DTOs at run time
> while preventing compile-time dependencies
>
> so long ... Nicolai
>
>
>
> On 01.07.2016 17:21, Paul Benedict wrote:
> > Since Hibernate is a popular example for this topic, let's talk
> > about it.
> >
> > First, when I build JPA applications, I do have an implementation
> > use in mind (Hibernate, EclipseLink, etc.), but my code is
> > standardized JPA as much as possible. So if I am going to publish
> > my JPA model and I want my private types reflected to be loaded as
> > entities, who am I going to export it too?
> >
> > Well I certainly am not going to export it to "jpa" because that's
> > just a specification...
> >
> > The next logical answer would be the "hibernate" module, but now
> > I'm gambling on knowing the internals of Hibernate... I have used
> > Hibernate professionally for several years, but, off the top of my
> > head, I honestly couldn't tell you if HIbernate has its own
> > reflection routines or if it delegates it to third party libraries
> > like "cglib" and "javassist". My gut tells me it's the latter, but
> > don't quote me on it :-)
> >
> > So now "dynamic" has put me in a pickle. All I really want to do is
> > have Hibernate access my private types for JPA, but I have to roll
> > several dice and pray to be right: (1) I got to know if Hibernate
> > does its reflection or not (2) Does Hibernate delegate to third
> > party reflection libraries? (3) Are the third party reflection
> > libraries pluggable? (4) What is going to happen to my code if
> > Hibernate alters its reflection mechanism?
> >
> > #4 is pretty important, actually. Tools have been known to shade
> > some of their third party dependencies. So while, for example,
> > Hibernate may rely on "cglib" or "javassist" today, there is a
> > possibility those may be shaded and brought in and another package
> > name in a future version. I've seen those discussions. I am not
> > implying what was or wasn't decided, but just it's a possibility.
> >
> > I just don't see a point in creating such a risky situation for
> > myself. I'd rather just have everything exported all the time to
> > every module if this is the way reflection is going.
> >
> >
> > Cheers, Paul
> >
> > On Fri, Jul 1, 2016 at 5:20 AM, Jochen Theodorou
> > <blackdrag at gmx.org> wrote:
> >
> >>
> >>
> >> On 01.07.2016 11:15, Rafael Winterhalter wrote: [...]
> >>
> >>> Finally, I do not believe the service loader API offers an
> >>> adequate alternative to many use-cases of reflection (while it
> >>> can perfectly substitute some of them). Many state-of-the art
> >>> libraries actually intentionally offer POJO approaches without
> >>> requiring their users to create interfaces for any shared
> >>> interaction. I do not think that a gentle push into a direction
> >>> that contradicts most existing Java libraries would be a taken
> >>> well by the people implementing this software and in the end,
> >>> the module system will be as successful as its adoption. I
> >>> rather see people finding new ways around restrictions to keep
> >>> things simple (for themselves).
> >>>
> >>
> >> even in the cases the service API is a replacement, the usage of
> >> the library will change and a breaking change is forced.
> >>
> >> For all these reasons I do not think that it is a good idea to
> >> limit
> >>> reflection to exported packages.
> >>>
> >>
> >> or people will start just exporting everything.
> >>
> >> bye Jochen
> >>
> >
>
> --
>
> PGP Key:
> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>
> Web:
> http://codefx.org
> a blog about software development
> http://do-foss.de
> Free and Open Source Software for the City of Dortmund
>
> Twitter:
> https://twitter.com/nipafx
>
>
From Alan.Bateman at oracle.com Tue Jul 5 12:11:44 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 5 Jul 2016 13:11:44 +0100
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <57784553.5090100@codefx.org>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
<CA+DM0AnfqM9j21ULBr61n3swWD605gaAeitDqZxYqtJzXwF56w@mail.gmail.com>
<577643F2.1080407@gmx.org>
<CABLGb9xDZOG5poMckKB9xPdQf-_0FC-KHnOup-+cGP8-JcNG-A@mail.gmail.com>
<57784553.5090100@codefx.org>
Message-ID: <c78b3d40-4e6c-a6e2-7086-89d684516b1d@oracle.com>
On 02/07/2016 23:50, Nicolai Parlog wrote:
> Hi Paul,
>
> either I'm misunderstanding you or one of us misunderstood the proposal.
>
>> So now "dynamic" has put me in a pickle. All I really want to do
>> is have Hibernate access my private types for JPA, but I have to
>> roll several dice and pray to be right: (1) I got to know if
>> Hibernate does its reflection or not (2) Does Hibernate delegate to
>> third party reflection libraries? (3) Are the third party
>> reflection libraries pluggable? (4) What is going to happen to my
>> code if Hibernate alters its reflection mechanism?
> You seem to be talking about qualified exports, which let you export
> packages _to_ a specific module (at compile and at run time). Dynamic
> exports, on the other hand, will export packages to every module
> (although I assume there would also be qualified dynamic exports) but
> only at run time.
That's right and it shouldn't matter if Hibernate is using core
reflection, method handles, or spinning code at runtime. In the case of
the latter two then the framework will need to arrange the readability
but that should be straight-forward to do as part of upgrading the
framework to support modules.
The other point that I don't see mentioned in this thread is that some
of the cases should be checkable at build time. If I have @Inject on a
member in a non-exported package then it would be possible to catch this
at build time. Same thing with configuration files where the types are
listed in the configuration.
-Alan
From david.lloyd at redhat.com Tue Jul 5 14:41:52 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Tue, 5 Jul 2016 09:41:52 -0500
Subject: Proposal: #DefaultModule
Message-ID: <577BC730.1040706@redhat.com>
I propose that the concept of "unnamed module" be dropped in favor of
"default module". The main difference is that the class loader (or
module finder or layer configuration or someone else) would be allowed
to (but not required to) assign a free-form name and version string to
this module. This would allow existing module systems to bring their
module concept into some form of consonance with Jigsaw without
compromising any of the restrictions that Jigsaw-style modules have.
Effecting this change would suggest the addition of an "isDefault()"
method on Module, possibly replacing "isNamed()" (which is arguably
already somewhat redundant with respect to getName()). Also at some
stage, something would have to establish the default module name and
version strings, probably defaulting to the (current) null strings to
keep a stable status quo.
--
- DML
From nipa at codefx.org Tue Jul 5 16:05:34 2016
From: nipa at codefx.org (Nicolai Parlog)
Date: Tue, 5 Jul 2016 18:05:34 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <CA+DM0AnfqM9j21ULBr61n3swWD605gaAeitDqZxYqtJzXwF56w@mail.gmail.com>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
<CA+DM0AnfqM9j21ULBr61n3swWD605gaAeitDqZxYqtJzXwF56w@mail.gmail.com>
Message-ID: <577BDACE.1080503@codefx.org>
Hi Rafael.
> As for JUnit 4 or Unsafe, reflection is used because of an actual
> need to interact with these libraries/APIs.
Yes, definitely. I hope I didn't come across as "these stupid Unsafe
users didn't know what they were doing!".
> As for JUnit, it is mainly IDE vendors that needed to access
> internals in order to display unit testing results and to extract
> details about testing results. By making this extraction more
> awkward, this would not have changed the fact that IDE-vendors want
> to interact with the internal APIs. It would only have been more
> difficult to do.
Alternatively, they could've improved JUnit 4's API, leading to the
following (ideally):
* faster improvements of JUnit 4 because less barriers to refactoring
* one less need for a total rewrite (!)
* no DRY across tools, leading to more time for other improvements
* an API for new tools to build on (enabling innovation),
which would otherwise have to repeat the same time-intensive hacks
And the only downside being that JUnit integration would initially have
been a little slower, which I think is a fair price to pay. I am
convinced that JUnit is a perfect example where the ease, with which
reflection allows to break encapsulation, showed immense detrimental
effects with no clear advantage.
The argument about Unsafe is pretty much the same; with a critical
difference: It is much harder and much slower to move the JDK along than
it is to do the some to "some open source project". This considerably
weakens my argument, of course, maybe making your point for you. Some
innovations may not have happened were it not for said weak encapsulation.
> Also, as a thought experiment, let us say accessing internal APIs had
> been impossible in the past: we would have missed out on Hadoop,
> Cassandra, Mockito, Spark; and only in 2016 we could start running
> JUnit from our IDE.
But what about the command line flags ("addexports" et al)? They
would've made adoption of these innovations slower but if they are
addressing a real need, wouldn't that be worth it? And much like in the
case of JUnit would it not also increase pressure on the JDK developers
to create a supported alternative?
Honestly, I just don't know. :)
Regarding the rest of your arguments. Many of them also apply to
visibility modifiers. As so often it's a trade-off (duh). And as so
often we only have past data (warped by what was possible) and
hypothesis (warped, amongst other things, by what we work on) to decide
what to do. So I think it is important to argue both sides of it and see
what comes out in the end. :)
I, for one, think that Carthago... eh that reflection should not be
allowed to break encapsulation (without command line flags).
so long ... Nicolai
On 01.07.2016 11:15, Rafael Winterhalter wrote:
> As for JUnit 4 or Unsafe, reflection is used because of an actual need to
> interact with these libraries/APIs.
>
> As for JUnit, it is mainly IDE vendors that needed to access internals in
> order to display unit testing results and to extract details about testing
> results. By making this extraction more awkward, this would not have
> changed the fact that IDE-vendors want to interact with the internal APIs.
> It would only have been more difficult to do. In my eyes, limiting
> reflection is fighting the symptoms, not the cause. For example, JUnit 5
> has adressed this issue by implementing a public API for extracting such
> information what immediately solved the current problem. I do not think
> anybody prefers reflection over an official API if the latter one satisfies
> all needs.
>
> The same goes for Unsafe. I do not think anybody likes to use Unsafe if
> there is a better alternative. The Unsafe API is awkward and - obviously -
> unsafe. And despite the implementors attempt to shield the API to bootstrap
> classes, people found their ways around because there has not been another
> way to make certain things work. I am sure, the more the JVM offers
> adequate alternatives, people will migrate happily.
>
> Also, as a thought experiment, let us say accessing internal APIs had been
> impossible in the past: we would have missed out on Hadoop, Cassandra,
> Mockito, Spark; and only in 2016 we could start running JUnit from our IDE.
> I think we can all agree that this is not the outcome we would have wanted.
> Reflection - despite it breaking encapsulation - has done more good then
> bad.
>
> As for the argument that required dynamic exports should be documented;
> ideally yes. Equally, APIs should be documented but many librararies fail
> to offer good documentation, especially in the open-source space where time
> is sparse. I already see myself answering questions to puzzled junior
> developers that know Java for a few months that "this is required because
> this library uses reflection" where they do not even know what the latter
> is. Effectively, building this barrier makes Java more difficult to use and
> I am convinced that many people will adapt adding dynamic exports for any
> package as some form of "best practice" to get on with their lifes. I will
> also bet that someone will deliver some "auto-export Maven plugin" to
> automate this "best practive".
>
> Finally, I do not believe the service loader API offers an adequate
> alternative to many use-cases of reflection (while it can perfectly
> substitute some of them). Many state-of-the art libraries actually
> intentionally offer POJO approaches without requiring their users to create
> interfaces for any shared interaction. I do not think that a gentle push
> into a direction that contradicts most existing Java libraries would be a
> taken well by the people implementing this software and in the end, the
> module system will be as successful as its adoption. I rather see people
> finding new ways around restrictions to keep things simple (for themselves).
>
> For all these reasons I do not think that it is a good idea to limit
> reflection to exported packages.
>
> 2016-07-01 9:27 GMT+02:00 Nicolai Parlog <nipa at codefx.org>:
>
>> Hi!
>>
>>> I do not think that shielding internals is a problem with regards
>>> to reflection.
>>
>> I think it is. I've not been emerged in the community for very long so
>> I know only two examples (but still): Unsafe and JUnit 4. In both
>> cases internals were accessed via reflection. And in both cases
>> dependencies became so ubiquitous that changing any details became
>> impossible for the maintainers.
>>
>> I think the outcry when Unsafe was planned to go away and the fact
>> that JUnit 4 had to be rewritten because evolution was effectively
>> prevented by unwanted dependencies make my point here.
>>
>>> I do neither think that reflection only interests internals.
>>
>> I agree. Not "only" but "all too often", though.
>>
>>> For example, maybe you use Hibernate for your persistance layer
>>> without exposing those DTOs from your module.
>>
>> Why would I do that?! This is exactly where reflection got us used to
>> sloppy architecture. If a tool uses my DTOs to implement my
>> persistence layer, then it looks like those DTOs are part of my API.
>> Maybe not public to everyone but qualified and dynamic exports give me
>> tools to reduce visibility.
>>
>>> I do not care about how Hibernate works as long as it does work.
>>> Almost any library uses reflection and I would not want to dig
>>> into implementation details to find out how to set up my dynamic
>>> exports.
>>
>> Maybe I'm missing something here but to me it looks like "export to a
>> module whatever that module is supposed to interact with" is a very
>> sensible default that should cover a vast majority of uses cases, no?
>>
>> I have DTOs && Hibernate needs DTOs -> export DTOs to Hibernate.
>>
>> Why would I look at the implementation? And if there were non-obvious
>> scenarios they should be properly documented so I can react accordingly.
>>
>>> As a library implementor, you aim for the biggest convenience.
>>
>> Maybe we should aim for clarity and well-defined interactions instead.
>>
>> so long ... Nicolai
>>
>>
>>
>> On 01.07.2016 00:20, Rafael Winterhalter wrote:
>>> I disagree on two ends here.
>>>
>>> I do not think that shielding internals is a problem with regards
>>> to reflection. You want to shield internals in order to not break
>>> dependants by prohibiting their existance. With reflection, you do
>>> however typically react to the actual implementation by discovering
>>> it at runtime. If your use pattern was fixed, you would always
>>> require a static dependency over reflection in the first place. I
>>> do neither think that reflection only interests internals. For
>>> example, maybe you use Hibernate for your persistance layer without
>>> exposing those DTOs from your module. The problem is that you
>>> invert responsibilities. I do not care about how Hibernate works as
>>> long as it does work. Almost any library uses reflection and I
>>> would not want to dig into implementation details to find out how
>>> to set up my dynamic exports. I want to require a module and
>>> hopefully the compiler tells me about anything that's wrong.
>>> Reflection access errors would only be an awkward side effect that
>>> can be avoided by not applying modules alltogether. I think this
>>> limitation would therefore hurt Java 9 adoption.
>>>
>>> The second problem I have is the issue of attempting to fix an old
>>> regret. From the perspective of Hibernate, it would be easiest to
>>> not attempt modules as their introduction would require its users
>>> to apply additional configuration. As a library implementor, you
>>> aim for the biggest convenience. Adding this restriction neither
>>> adds additional security where a security manager should be used. I
>>> again think this would hurt adoption.
>>>
>>> I agree with Stephen's argument even though I do not think that
>>> forbidding dynamic access would be a feature anybody would use so
>>> it should not be added. Rather, something should be integrated in
>>> the security manager, if this is considered necessary.
>>>
>>> Cheers, Rafael
>>>
>>> Am 30.06.2016 11:37 nachm. schrieb "Nicolai Parlog"
>>> <nipa at codefx.org>:
>>>>
>>>> Hi!
>>>>
>>>>> In my opinion, reflection should not apply any module-boundary
>>>>> checks.
>>>>
>>>> Maybe this list is not a representative selection of Java
>>>> developers, maybe I'm just the only one thinking this but, to
>>>> paraphrase:
>>>>
>>>> Reflection over internals must die in a fire!
>>>>
>>>> Yes, yes, tools to great things, we need to see everything, ...
>>>> Still, I think that modules need a safe space that they can use
>>>> without anybody intruding. Maybe I'm a hardliner but I already
>>>> view the command line parameters with suspicion.
>>>>
>>>> In that light, I like the "dynamic" proposal.
>>>>
>>>> Not the name, though. Having the symmetric pair "static vs
>>>> dynamic" looks nice but it takes a lot of words to explain why
>>>> exactly these terms apply here. We seem to be looking for
>>>> qualifiers for "requires" and "exports". Maybe just using other
>>>> terms solves the problem?
>>>>
>>>> requires & consumes exports & contains
>>>>
>>>> so long ... Nicolai
>>>>
>>>>
>>>>
>>>> On 30.06.2016 01:23, Rafael Winterhalter wrote:
>>>>> I also agree here that the use of reflection should not require
>>>>> any form of explicit permission by the user of a library. In a
>>>>> way, this ties an implementation detail (reflection in this
>>>>> case) to the usage of the library. The library cannot work
>>>>> without the explicit configuration by a user and this only adds
>>>>> boilerplate to using Hibernate.
>>>>>
>>>>> Unfortunately, many Java developers still do not know much
>>>>> about reflection or about the inner workings of many libraries
>>>>> in the market. If a user needed to explicitly export packages,
>>>>> this might just lead to people "over exporting" their packages
>>>>> just to "make Hibernate work".
>>>>>
>>>>> In my opinion, reflection should not apply any module-boundary
>>>>> checks.
>>>>>
>>>>> Best regards, Rafael
>>>>>
>>>>> 2016-06-29 20:13 GMT+02:00 Paul Benedict
>>>>> <pbenedict at apache.org>:
>>>>>
>>>>>> Replying to "observers" list...
>>>>>>
>>>>>> I think the main problem here is that core reflection has
>>>>>> become too restrictive. Reflection should be "magical" and
>>>>>> have no knowledge of any kind of module and/or visibility
>>>>>> boundaries. If you really want to control what can be
>>>>>> reflected upon, that should be done via the Policy file of
>>>>>> the Security Manager.
>>>>>>
>>>>>> This proposal is trying to hack out-of-the-box created by
>>>>>> too strict of a design. It is very awkward to denote which
>>>>>> packages must be reflectable ahead of time --- how can you
>>>>>> figure that out?? The consumers who are doing the reflection
>>>>>> are innumerable. I'd go as far to say it's it's impossible to
>>>>>> devise a preemptive comprehensive strategy that may satisfy
>>>>>> the kinds of tooling in the wild.
>>>>>>
>>>>>> It's best just to open up reflection so it can continue
>>>>>> doing what it always has done well: let me see anything I
>>>>>> want unless the Security Manager says no.
>>>>>>
>>>>>> Cheers, Paul
>>>>>>
>>>>>> On Wed, Jun 29, 2016 at 3:51 AM, Remi Forax
>>>>>> <forax at univ-mlv.fr> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ----- Mail original -----
>>>>>>>> De: "Mark Reinhold" <mark.reinhold at oracle.com> ?:
>>>>>>>> jpms-spec-experts at openjdk.java.net Envoy?: Mardi 28 Juin
>>>>>>>> 2016 23:18:15 Objet: Proposal:
>>>>>>>> #ReflectiveAccessToNonExportedTypes: `exports dynamic`
>>>>>>>>
>>>>>>>> Issue summary -------------
>>>>>>>>
>>>>>>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of
>>>>>>>> framework libraries require reflective access to members
>>>>>>>> of the non-exported types of other modules; examples
>>>>>>>> include dependency injection
>>>>>> (Guice),
>>>>>>>> persistence (JPA), debugging tools, code-automation
>>>>>>>> tools, and serialization (XStream). In some cases the
>>>>>>>> particular library to be used is not known until run time
>>>>>>>> (e.g., Hibernate and EclipseLink
>>>>>> both
>>>>>>>> implement JPA). This capability is also sometimes used
>>>>>>>> to work
>>>>>> around
>>>>>>>> bugs in unchangeable code. Access to non-exported
>>>>>>>> packages can, at present, only be done via command-line
>>>>>>>> flags, which is extremely awkward. Provide an easier way
>>>>>>>> for reflective code to access such non-exported types.
>>>>>>>> [1]
>>>>>>>>
>>>>>>>> Proposal --------
>>>>>>>>
>>>>>>>> Extend the language of module declarations so that a
>>>>>>>> package can be declared to be exported at run time but
>>>>>>>> not at compile time. This is, roughly, the dual of the
>>>>>>>> `requires static` construct proposed for
>>>>>>>> #CompileTimeDependences, hence we propose to introduce a
>>>>>>>> new modifier, `dynamic`, for use on the `exports`
>>>>>>>> directive. It has the following meanings:
>>>>>>>>
>>>>>>>> - At compile time, `exports dynamic P` does not cause
>>>>>>>> the package `P` to be exported, though it does require
>>>>>>>> `P` to be a package defined in the module.
>>>>>>>>
>>>>>>>> - In phases after compile time, `exports dynamic P`
>>>>>>>> behaves in
>>>>>> exactly
>>>>>>>> the same way as `exports P`. It therefore takes part
>>>>>>>> fully in resolution and configuration, and is subject to
>>>>>>>> the same
>>>>>> consistency
>>>>>>>> constraints as normally-exported packages (e.g., no
>>>>>>>> split
>>>>>> packages).
>>>>>>>>
>>>>>>>> Thus a module declaration of the form
>>>>>>>>
>>>>>>>> module com.foo.app { requires hibernate.core; requires
>>>>>>>> hibernate.entitymanager; exports dynamic
>>>>>>>> com.foo.app.model; }
>>>>>>>>
>>>>>>>> makes the types in the `com.foo.app.model` package
>>>>>>>> accessible at run
>>>>>> time
>>>>>>>> but not at compile time. In combination with the
>>>>>>>> earlier change to enable #ReflectionWithoutReadability
>>>>>>>> [2] this means that frameworks
>>>>>> that
>>>>>>>> today use core reflection to manipulate user classes at
>>>>>>>> run time are
>>>>>> more
>>>>>>>> likely to work out-of-the-box, without change, as
>>>>>>>> automatic modules.
>>>>>> If
>>>>>>>> the `com.foo.app.model` package in this example includes
>>>>>>>> entity classes to be managed by Hibernate then the
>>>>>>>> framework will be able to access
>>>>>> them
>>>>>>>> without further ado, but under normal circumstances an
>>>>>>>> attempt to
>>>>>> compile
>>>>>>>> code that refers directly to those classes will fail.
>>>>>>>>
>>>>>>>> The `dynamic` modifier can be applied to both unqualified
>>>>>>>> and qualified `exports` directives, though the caveats on
>>>>>>>> using qualified exports [3] still apply.
>>>>>>>>
>>>>>>>> Notes -----
>>>>>>>>
>>>>>>>> - To access a non-public member in a
>>>>>>>> dynamically-exported package you must still invoke the
>>>>>>>> appropriate `setAccessible` method, just as you do today
>>>>>>>> to access non-public members in an exported package.
>>>>>>>>
>>>>>>>> - This proposal is similar to R?mi's suggestion to add a
>>>>>>>> way to
>>>>>> export
>>>>>>>> a package only for reflection, still requiring that
>>>>>>>> `setAccessible` be invoked [4]. This proposal differs
>>>>>>>> in that it does not require `setAccessible` to be invoked
>>>>>>>> to access public members in a dynamically-exported
>>>>>>>> package.
>>>>>>>>
>>>>>>>> - This proposal primarily addresses "friendly" uses of
>>>>>>>> reflection,
>>>>>> such
>>>>>>>> as dependency injection and persistence, in which the
>>>>>>>> author of a module knows in advance that one or more
>>>>>>>> packages must be exported
>>>>>> at
>>>>>>>> run time for reflective access by frameworks. This
>>>>>>>> proposal is
>>>>>> also
>>>>>>>> convenient for frameworks that generate code at run
>>>>>>>> time, since the constant pool of a generated class file
>>>>>>>> can include static
>>>>>> references
>>>>>>>> to types in dynamically exported packages.
>>>>>>>>
>>>>>>>> - This proposal opens the door to a category of
>>>>>>>> second-class APIs.
>>>>>> If
>>>>>>>> a package is exported dynamically then you can still
>>>>>>>> compile code that refers to types in the package, by
>>>>>>>> using `-XaddExports` or its equivalent at compile time,
>>>>>>>> and it will work as expected at run
>>>>>> time.
>>>>>>>> It thus may be useful to use `exports dynamic` for
>>>>>>>> packages that contain legacy APIs whose use is strongly
>>>>>>>> discouraged, e.g., those
>>>>>> in
>>>>>>>> the `jdk.unsupported` module of the reference
>>>>>>>> implementation,
>>>>>> thereby
>>>>>>>> forcing anyone who wants to compile against them to go
>>>>>>>> to the
>>>>>> trouble
>>>>>>>> of using `-XaddExports`.
>>>>>>>>
>>>>>>>> - Intrusive access to arbitrary packages of arbitrary
>>>>>>>> modules by,
>>>>>> e.g.,
>>>>>>>> debugging tools, will still require the use of sharp
>>>>>>>> knives such as the `-XaddExports` command-line option or
>>>>>>>> its equivalent, or JVM
>>>>>> TI.
>>>>>>>>
>>>>>>>> - Using the `-XaddExports` option or its equivalent
>>>>>>>> remains awkward, and sometimes it's the only way out. To
>>>>>>>> ease migration I think
>>>>>> it's
>>>>>>>> worth considering some way for an application packaged as
>>>>>>>> a JAR
>>>>>> file
>>>>>>>> to include such options in its `MANIFEST.MF` file, as
>>>>>>>> suggested by Simon Nash [5]. I'll create a separate
>>>>>>>> issue for that.
>>>>>>>>
>>>>>>>> - New kinds of resolution failures are possible. If
>>>>>>>> module `A` requires `B`, and `B` requires `C`, then if
>>>>>>>> `B` and `C` both
>>>>>> declare
>>>>>>>> some package `P` to be exported dynamically, then a
>>>>>>>> split-package error will be reported. This may
>>>>>>>> surprise, since the packages are just internal
>>>>>>>> implementation details that were exported dynamically so
>>>>>>>> that some framework could access their types. This is
>>>>>>>> not, however, all that different from the kinds of
>>>>>>>> resolution and layer-creation failures that are already
>>>>>>>> possible due to package collisions. It's unlikely to
>>>>>>>> happen all that often in practice, and the fix is
>>>>>>>> straightforward: Just use reverse-DNS or otherwise
>>>>>>>> sufficiently-unique package names, as most people already
>>>>>>>> do. It is worth exploring whether javac can detect at
>>>>>>>> least some of these kinds of collisions and issue
>>>>>>>> appropriate warnings.
>>>>>>>
>>>>>>> I would like to agree on that proposal only on the ground
>>>>>>> that it
>>>>>> requires
>>>>>>> to change the classfile format thus allows in the same time
>>>>>>> to fix the issues of the current classfile spec (encoding
>>>>>>> of the name module as a Class instead of UTF8, wrong flag
>>>>>>> for requires public), but i'm not sure this proposal
>>>>>>> support it's own weight.
>>>>>>>
>>>>>>> Without dynamic, a 'classical' export works the same way
>>>>>>> at runtime so adding 'dynamic' to an export only restrict
>>>>>>> the usage at compile time. Given that we can already
>>>>>>> selectively export packages to some chosen modules which
>>>>>>> hide them for the rest of the other modules at compile
>>>>>> time,
>>>>>>> adding dynamic to the spec adds little value.
>>>>>>>
>>>>>>> And it has several drawbacks, the main one is that it
>>>>>>> weaken the compiletime/runtime fidelity story, with a
>>>>>>> dynamic export the runtime
>>>>>> world
>>>>>>> is not the compiletime world anymore. The main motivation
>>>>>>> to add dynamic seems to be for backward compatibility
>>>>>>> during the transition between the classpath world and the
>>>>>>> full module
>>>>>> world,
>>>>>>> for several other cases, we use the command line for that,
>>>>>>> i don't see
>>>>>> why
>>>>>>> this case is so special that it should not work like the
>>>>>>> other compatibility issues.
>>>>>>>
>>>>>>> Moreover, an agent, an annotation processor and a jlink
>>>>>>> plugin are all able to implement the same trick, add export
>>>>>>> after the compilation and before the runtime, so i don't
>>>>>>> think it's a good idea to change the spec for that.
>>>>>>>
>>>>>>> R?mi
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>>>>>>>
>>>>>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectiveAccess
>>>>
>>>>>>
>> ToNonExportedTypes
>>>>>>>>
>>>>>>
>>>> [2]
>>>>>>>>
>>>>>>>
>>>>>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ReflectionWithou
>>>>
>>>>>>
>> tReadability
>>>>>>>>
>>>>>>
>>>> [3]
>>>>>>> http://openjdk.java.net/projects/jigsaw/spec/sotms/#qualified-export
>>>>
>>>>>>>
>> s
>>>>>>>>
>>>>>>>
>>>> [4]
>>>>>>>>
>>>>>>>
>>>>>> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-Decembe
>>>>
>>>>>>
>> r/000205.html
>>>>>>>>
>>>>>>
>>>> [5]
>>>>>>>>
>>>>>>>
>>>>>> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/00574
>>>>
>>>>>>
>> 5.html
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>>
>>>> --
>>>>
>>>> PGP Key:
>>>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>>>>
>>>>
>>>>
>> Web:
>>>> http://codefx.org a blog about software development
>>>> http://do-foss.de Free and Open Source Software for the City of
>>>> Dortmund
>>>>
>>>> Twitter: https://twitter.com/nipafx
>>>>
>>>
>>
>> --
>>
>> PGP Key:
>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>>
>> Web:
>> http://codefx.org
>> a blog about software development
>> http://do-foss.de
>> Free and Open Source Software for the City of Dortmund
>>
>> Twitter:
>> https://twitter.com/nipafx
>>
>>
>
--
PGP Key:
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
Web:
http://codefx.org
a blog about software development
http://do-foss.de
Free and Open Source Software for the City of Dortmund
Twitter:
https://twitter.com/nipafx
From nipa at codefx.org Tue Jul 5 16:21:19 2016
From: nipa at codefx.org (Nicolai Parlog)
Date: Tue, 5 Jul 2016 18:21:19 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <CA+DM0Amp9TQADR9B=NX-1YLKcp8dsFFtUzA9v=6TRyfmGQuW8Q@mail.gmail.com>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<1254086925.1013150.1467190276220.JavaMail.zimbra@u-pem.fr>
<CABLGb9w_1OuLYE69T5a0qn0efEhjRkEzhYWmNqDdfvqkVkK1GA@mail.gmail.com>
<CA+DM0A=01MiaC6d=MK2B0tNF=EoTbYFAVOGbjLtQXHuEx=3wWg@mail.gmail.com>
<577590E2.7020907@codefx.org>
<CA+DM0AmXQQ_qk7GGDDv0x_6m6fEd6js67rN0dDkUywqH+z6P2g@mail.gmail.com>
<57761B70.10403@codefx.org>
<CA+DM0AnfqM9j21ULBr61n3swWD605gaAeitDqZxYqtJzXwF56w@mail.gmail.com>
<577643F2.1080407@gmx.org>
<CABLGb9xDZOG5poMckKB9xPdQf-_0FC-KHnOup-+cGP8-JcNG-A@mail.gmail.com>
<57784553.5090100@codefx.org>
<CA+DM0Amp9TQADR9B=NX-1YLKcp8dsFFtUzA9v=6TRyfmGQuW8Q@mail.gmail.com>
Message-ID: <577BDE7F.2060405@codefx.org>
Hi Rafael.
> If you need to explicitly white-list packages that are accessible
> via reflection, this makes things more difficult for most users
> while adding a minimum benefit for very few users (i.e. making it
> more difficult to "jail break" an internal API using reflection).
As I explained in the mail I just sent to the other fork of this
thread, I am convinced that users get harmed by broken encapsulation
even if they are not the ones actually doing it.
> In practice, dynamic exports would not require to export one
> package but most likely a whole bunch of packages where finding out
> about what packages to white-list would require quite some effort
> ...
(Which is of course also true for "regular "exports.)
> ... and the correctnes could only be validated at runtime.
Yes, good point. This sucks. Tools might help (see Alan's comment) but
otherwise you are right. Retroactively adding dynamic exports might
require quite some effort.
> I just checked what this would mean for one of our production
> applications by setting a break point in Method::invoke and we
> would probably choose to not apply modules rather than adding every
> single package as a dynamic export as there are so many libraries
> that use reflection and we have no overview how this is organized.
It seems to be quite obvious that the application is not ready to be
modularized (no offense intended; ours isn't either!), so why force
it? Just put it on the class path (where reflection continues to work
as today), split of smaller modules over the next couple of years, and
migrate slowly.
Maybe this is the underlying assumption that makes this such an
overwhelming and apparently fruitless task: the idea that all code has
to be put in modules soon. Because that's not so. If a project is
fighting so hard to stay on the class path, then let it (for another
year or two). No harm done.
> [V]ery few developers use reflection to access internal APIs and
> those mostly know what it means to circumvent offical APIs.
Do they really? Because a lot of those developers were complaining
quite loudly that this is going to break their code in a way they were
obviously not prepared for. And not only did they risk their projects'
livelihood but also that of their users! Because I am pretty sure that
many of those had no idea what fickle internal details they were
indirectly depending on.
> In this case, reflection is really only used as a last resort
> where there is no other way like in the cases of JUnit or Unsafe.
I addressed that in my other mail.
> I therefore still hope Java will continue to allow reflection
> across module boundaries without requireing any additional
> configuration.
:)
I still agree that reflection should loose its super powers and be
rendered unable to break encapsulation (without command line flags).
so long ... Nicolai
On 04.07.2016 11:36, Rafael Winterhalter wrote:
> This statement is where I disagree: So your module will simply
> "export dynamic package.with.dtos"
>
> If you need to explicitly white-list packages that are accessible
> via reflection, this makes things more difficult for most users
> while adding a minimum benefit for very few users (i.e. making it
> more difficult to "jail break" an internal API using reflection).
> In practice, dynamic exports would not require to export one
> package but most likely a whole bunch of packages where finding out
> about what packages to white-list would require quite some effort
> and the correctnes could only be validated at runtime.
>
> I just checked what this would mean for one of our production
> applications by setting a break point in Method::invoke and we
> would probably choose to not apply modules rather than adding every
> single package as a dynamic export as there are so many libraries
> that use reflection and we have no overview how this is organized.
> Again, I do not argue that this is perfectly structured software
> but I think this is still representative of the reality of most
> enterprise apps.
>
> Finally, I do not believe that the benefit of protecting internal
> APIs is as big as it is argued here. Most of the time, reflection
> is used by libraries to access my code where reflection is used for
> (annotation-based) discovery and susequent interaction. The other
> way around, very few developers use reflection to access internal
> APIs and those mostly know what it means to circumvent offical
> APIs. In this case, reflection is really only used as a last resort
> where there is no other way like in the cases of JUnit or Unsafe.
>
> I therefore still hope Java will continue to allow reflection
> across module boundaries without requireing any additional
> configuration.
>
> 2016-07-03 0:50 GMT+02:00 Nicolai Parlog <nipa at codefx.org>:
>
>> Hi Paul,
>>
>> either I'm misunderstanding you or one of us misunderstood the
>> proposal.
>>
>>> So now "dynamic" has put me in a pickle. All I really want to
>>> do is have Hibernate access my private types for JPA, but I
>>> have to roll several dice and pray to be right: (1) I got to
>>> know if Hibernate does its reflection or not (2) Does Hibernate
>>> delegate to third party reflection libraries? (3) Are the third
>>> party reflection libraries pluggable? (4) What is going to
>>> happen to my code if Hibernate alters its reflection
>>> mechanism?
>>
>> You seem to be talking about qualified exports, which let you
>> export packages _to_ a specific module (at compile and at run
>> time). Dynamic exports, on the other hand, will export packages
>> to every module (although I assume there would also be qualified
>> dynamic exports) but only at run time.
>>
>> So your module will simply "export dynamic package.with.dtos"
>> and every module can access those types at runtime if it reads
>> the your module. If it uses reflection to access some DTOs, the
>> read edge is added automatically.
>>
>> To me this seems like a clean solution where
>>
>> (a) nothing changes for Hibernate (b) a module can expose it's
>> DTOs at run time while preventing compile-time dependencies
>>
>> so long ... Nicolai
>>
>>
>>
>> On 01.07.2016 17:21, Paul Benedict wrote:
>>> Since Hibernate is a popular example for this topic, let's
>>> talk about it.
>>>
>>> First, when I build JPA applications, I do have an
>>> implementation use in mind (Hibernate, EclipseLink, etc.), but
>>> my code is standardized JPA as much as possible. So if I am
>>> going to publish my JPA model and I want my private types
>>> reflected to be loaded as entities, who am I going to export it
>>> too?
>>>
>>> Well I certainly am not going to export it to "jpa" because
>>> that's just a specification...
>>>
>>> The next logical answer would be the "hibernate" module, but
>>> now I'm gambling on knowing the internals of Hibernate... I
>>> have used Hibernate professionally for several years, but, off
>>> the top of my head, I honestly couldn't tell you if HIbernate
>>> has its own reflection routines or if it delegates it to third
>>> party libraries like "cglib" and "javassist". My gut tells me
>>> it's the latter, but don't quote me on it :-)
>>>
>>> So now "dynamic" has put me in a pickle. All I really want to
>>> do is have Hibernate access my private types for JPA, but I
>>> have to roll several dice and pray to be right: (1) I got to
>>> know if Hibernate does its reflection or not (2) Does Hibernate
>>> delegate to third party reflection libraries? (3) Are the third
>>> party reflection libraries pluggable? (4) What is going to
>>> happen to my code if Hibernate alters its reflection
>>> mechanism?
>>>
>>> #4 is pretty important, actually. Tools have been known to
>>> shade some of their third party dependencies. So while, for
>>> example, Hibernate may rely on "cglib" or "javassist" today,
>>> there is a possibility those may be shaded and brought in and
>>> another package name in a future version. I've seen those
>>> discussions. I am not implying what was or wasn't decided, but
>>> just it's a possibility.
>>>
>>> I just don't see a point in creating such a risky situation
>>> for myself. I'd rather just have everything exported all the
>>> time to every module if this is the way reflection is going.
>>>
>>>
>>> Cheers, Paul
>>>
>>> On Fri, Jul 1, 2016 at 5:20 AM, Jochen Theodorou
>>> <blackdrag at gmx.org> wrote:
>>>
>>>>
>>>>
>>>> On 01.07.2016 11:15, Rafael Winterhalter wrote: [...]
>>>>
>>>>> Finally, I do not believe the service loader API offers an
>>>>> adequate alternative to many use-cases of reflection (while
>>>>> it can perfectly substitute some of them). Many
>>>>> state-of-the art libraries actually intentionally offer
>>>>> POJO approaches without requiring their users to create
>>>>> interfaces for any shared interaction. I do not think that
>>>>> a gentle push into a direction that contradicts most
>>>>> existing Java libraries would be a taken well by the people
>>>>> implementing this software and in the end, the module
>>>>> system will be as successful as its adoption. I rather see
>>>>> people finding new ways around restrictions to keep things
>>>>> simple (for themselves).
>>>>>
>>>>
>>>> even in the cases the service API is a replacement, the usage
>>>> of the library will change and a breaking change is forced.
>>>>
>>>> For all these reasons I do not think that it is a good idea
>>>> to limit
>>>>> reflection to exported packages.
>>>>>
>>>>
>>>> or people will start just exporting everything.
>>>>
>>>> bye Jochen
>>>>
>>>
>>
>> --
>>
>> PGP Key:
>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>>
>>
>>
Web:
>> http://codefx.org a blog about software development
>> http://do-foss.de Free and Open Source Software for the City of
>> Dortmund
>>
>> Twitter: https://twitter.com/nipafx
>>
>>
>
--
PGP Key:
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
Web:
http://codefx.org
a blog about software development
http://do-foss.de
Free and Open Source Software for the City of Dortmund
Twitter:
https://twitter.com/nipafx
From alex.buckley at oracle.com Tue Jul 5 20:09:38 2016
From: alex.buckley at oracle.com (Alex Buckley)
Date: Tue, 05 Jul 2016 13:09:38 -0700
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <4ab19576-4e42-d77a-7669-48ee319e694a@goebel-clan.de>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net> <e432a1f4-603d-17f0-406e-e53c961db756@goebel-clan.de> <577588EA.6060202@oracle.com>
<4ab19576-4e42-d77a-7669-48ee319e694a@goebel-clan.de>
Message-ID: <577C1402.1080704@oracle.com>
Volker,
Even if reflection was enabled by default, you could be sure that JDK
modules would choose to disable reflection on their internal APIs.
-XaddExports would still be needed to make your code run.
Also, it seems like your code is sensitive to the application class
loader whose implementation has changed in JDK 9. Please see JEP 260 and
the "Class loaders" section of JEP 261.
Alex
On 7/2/2016 3:09 PM, Volker Berlin wrote:
> Alex,
>
> It is to compensate bugs or handle bad design of JDK components. Mostly
> this are classloader problems. The impossible to make the JDK classes
> via dynamic URLClassLoader available.
>
> For example:
>
> * register dynamically a JDBC that is not in the appclassloader to
> all our plugins.
> * register a URL protocol with a protocol handler that is not in the
> appclassloader
>
> Many hacks are needed for test environments. For example:
>
> * the simulation that there is no console (javaw)
> * inverse classloder hierarchy. In the appclassloader there is a class
> which implements/extends a class in a dynamic loaded library. The
> current tested classes and the JUnit tests are typical in the
> appclassloader
>
> This are only some points. There can be more. We have used.
>
> Volker Berlin
> i-net software
>
>
> Am 30.06.2016 um 23:02 schrieb Alex Buckley:
>> Volker,
>>
>> Please share details of why your application needs to reflect over the
>> internals of another module.
>>
>> Alex
>>
>> On 6/30/2016 1:09 PM, Volker Berlin wrote:
>>> Reflection should be enabled by default. My current launcher for Java 9
>>> check if the needed refection is possible. If not possible then it
>>> restart the application with a very long list of -XaddExports switches.
>>> I think this is not what we want.
>>>
>>> Volker Berlin
>>> i-net software
>>>
>>>
>>> Am 28.06.2016 um 23:18 schrieb Mark Reinhold:
>>>> Issue summary
>>>> -------------
>>>>
>>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of framework
>>>> libraries require reflective access to members of the non-exported
>>>> types of other modules; examples include dependency injection
>>>> (Guice),
>>>> persistence (JPA), debugging tools, code-automation tools, and
>>>> serialization (XStream). In some cases the particular library to be
>>>> used is not known until run time (e.g., Hibernate and EclipseLink
>>>> both
>>>> implement JPA). This capability is also sometimes used to work
>>>> around
>>>> bugs in unchangeable code. Access to non-exported packages can, at
>>>> present, only be done via command-line flags, which is extremely
>>>> awkward. Provide an easier way for reflective code to access such
>>>> non-exported types. [1]
>>>>
>>>> Proposal
>>>> --------
>>>>
>>>> Extend the language of module declarations so that a package can be
>>>> declared to be exported at run time but not at compile time. This is,
>>>> roughly, the dual of the `requires static` construct proposed for
>>>> #CompileTimeDependences, hence we propose to introduce a new modifier,
>>>> `dynamic`, for use on the `exports` directive. It has the following
>>>> meanings:
>>>>
>>>> - At compile time, `exports dynamic P` does not cause the package
>>>> `P`
>>>> to be exported, though it does require `P` to be a package defined
>>>> in the module.
>>>>
>>>> - In phases after compile time, `exports dynamic P` behaves in
>>>> exactly
>>>> the same way as `exports P`. It therefore takes part fully in
>>>> resolution and configuration, and is subject to the same
>>>> consistency
>>>> constraints as normally-exported packages (e.g., no split
>>>> packages).
>>
>
From mark.reinhold at oracle.com Wed Jul 6 14:49:17 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 06 Jul 2016 07:49:17 -0700
Subject: Proposal: #FixClassFileFormat
In-Reply-To: <321002664.2003394.1467404425232.JavaMail.zimbra@u-pem.fr>
References: <1813643129.1709819.1467357511504.JavaMail.zimbra@u-pem.fr>
<1751220489.1722677.1467358740078.JavaMail.zimbra@u-pem.fr>
<20160701104315.789031669eggemoggin.niobe.net>
<321002664.2003394.1467404425232.JavaMail.zimbra@u-pem.fr>
Message-ID: <20160706074917.217942295eggemoggin.niobe.net>
2016/7/1 13:20:25 -0700, Remi Forax <forax at univ-mlv.fr>:
> ...
>
> and i've forgotten a third issue, ACC_MODULE is specified as 0x8000,
> 0x8000 is very special value because it's the only one that left which
> is available on class, method and field. Historically, in the first
> specs, "module" was a modifier like public or private thus using the
> same value for all the elements was making sense, but now ACC_MODULE
> can only applied on class so using a value available on all the
> elements make less sense, the value 0x8000 should be free in order to
> be used later. The possible value for ACC_MODULE are: 0x0008
> (ACC_STATIC), 0x0040 (ACC_VOLATILE or ACC_BRIDGE), 0x0080 (ACC_VARARGS
> or ACC_TRANSIENT), 0x0100 (ACC_NATIVE), 0x0800 (ACC_STRICT). So
> either, 0x0040 or 0x0080 are the best candidate because the other
> values are only valid either on field or on method but not on
> both. And now, because 0x0080 is a permutation of 0x8000 in term of
> characters , i think 0x0040 is the best value for ACC_MODULE.
>
> Changing the value of ACC_MODULE is interesting if it's done at the
> same time as the other changes because the compiler and the VM can
> easily detect and reject the classfile encoded with what will become
> an old format.
I'll enter this in the issue list as:
#ClassFileAccModule_ --- The `ACC_MODULE` constant is currently
specified to have the value `0x8000`. This is the last available bit
remaining across all of the various `access_flags` fields of a class
file, and thus should be reserved for some unspecified future purpose
where it may be useful to use the same value in all such fields.
Alternative candidates for `ACC_MODULE` include `0x0040` (overlaps
with `ACC_VOLATILE` and `ACC_BRIDGE`) and `0x0080` (`ACC_TRANSIENT`
and `ACC_VARARGS`).
- Mark
From mark.reinhold at oracle.com Wed Jul 6 15:15:36 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 06 Jul 2016 08:15:36 -0700
Subject: Issue list updated
Message-ID: <20160706081536.856940838eggemoggin.niobe.net>
FYI, I've updated the issue list to add #AddExportsInManifest and R?mi's
various class-file issues (#ClassFileModuleName, #ClassFileAccPublic,
and #ClassFileAccModule):
http://openjdk.java.net/projects/jigsaw/spec/issues/
- Mark
From scolebourne at joda.org Wed Jul 6 15:26:29 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Wed, 6 Jul 2016 16:26:29 +0100
Subject: Issue list updated
In-Reply-To: <577d20b6.2442c20a.93ef0.2404SMTPIN_ADDED_BROKEN@mx.google.com>
References: <577d20b6.2442c20a.93ef0.2404SMTPIN_ADDED_BROKEN@mx.google.com>
Message-ID: <CACzrW9C0qr4kepV=gAB1uDoBY3Xcvxpeq3oqnWzU0WM093M10Q@mail.gmail.com>
Can I ask that #ServiceLoaderEnhancement is added too?
http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-July/000452.html
Stephen
On 6 July 2016 at 16:15, <mark.reinhold at oracle.com> wrote:
> FYI, I've updated the issue list to add #AddExportsInManifest and R?mi's
> various class-file issues (#ClassFileModuleName, #ClassFileAccPublic,
> and #ClassFileAccModule):
>
> http://openjdk.java.net/projects/jigsaw/spec/issues/
>
> - Mark
From mark.reinhold at oracle.com Wed Jul 6 15:30:45 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 06 Jul 2016 08:30:45 -0700
Subject: Issue list updated
In-Reply-To: <CACzrW9C0qr4kepV=gAB1uDoBY3Xcvxpeq3oqnWzU0WM093M10Q@mail.gmail.com>
References: <577d20b6.2442c20a.93ef0.2404SMTPIN_ADDED_BROKEN@mx.google.com>
<CACzrW9C0qr4kepV=gAB1uDoBY3Xcvxpeq3oqnWzU0WM093M10Q@mail.gmail.com>
Message-ID: <20160706083045.233576305eggemoggin.niobe.net>
2016/7/6 8:26:29 -0700, Stephen Colebourne <scolebourne at joda.org>:
> Can I ask that #ServiceLoaderEnhancement is added too?
> http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-July/000452.html
Yes -- more updates are on the way.
- Mark
From mark.reinhold at oracle.com Wed Jul 6 15:32:36 2016
From: mark.reinhold at oracle.com (Mark Reinhold)
Date: Wed, 6 Jul 2016 08:32:36 -0700 (PDT)
Subject: Proposal: #ClassFileAccPublic
Message-ID: <20160706153236.A62DBB6B74@eggemoggin.niobe.net>
Issue summary
-------------
#ClassFileAccPublic --- The `ACC_PUBLIC` constant in a `requires_flags`
should be encoded as `0x0001`, as it is elsewhere in the JVMS, rather
than as `0x0020`, which has different meanings in other contexts.
Proposal
--------
Unlike the other `ACC_PUBLIC` constants, this constant does not express
an access mode but, rather, the addition of an implied transitive
readability edge from modules that depend upon this module to some other
module required by this module. To avoid confusion at the class-file
level, rename this constant to `ACC_TRANSITIVE`.
(Whether the syntax of module declarations should use a keyword other
than `public` for this purpose would be a separate issue.)
[1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ClassFileAccPublic
From forax at univ-mlv.fr Wed Jul 6 16:23:52 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 6 Jul 2016 18:23:52 +0200 (CEST)
Subject: Proposal: #ClassFileAccPublic
In-Reply-To: <20160706153236.A62DBB6B74@eggemoggin.niobe.net>
References: <20160706153236.A62DBB6B74@eggemoggin.niobe.net>
Message-ID: <823657001.1414112.1467822232509.JavaMail.zimbra@u-pem.fr>
ACC_TRANSITIVE looks fine to me,
even if it really means transitive at one hop only.
R?mi
----- Mail original -----
> De: "Mark Reinhold" <mark.reinhold at oracle.com>
> ?: jpms-spec-experts at openjdk.java.net
> Envoy?: Mercredi 6 Juillet 2016 17:32:36
> Objet: Proposal: #ClassFileAccPublic
>
> Issue summary
> -------------
>
> #ClassFileAccPublic --- The `ACC_PUBLIC` constant in a `requires_flags`
> should be encoded as `0x0001`, as it is elsewhere in the JVMS, rather
> than as `0x0020`, which has different meanings in other contexts.
>
> Proposal
> --------
>
> Unlike the other `ACC_PUBLIC` constants, this constant does not express
> an access mode but, rather, the addition of an implied transitive
> readability edge from modules that depend upon this module to some other
> module required by this module. To avoid confusion at the class-file
> level, rename this constant to `ACC_TRANSITIVE`.
>
> (Whether the syntax of module declarations should use a keyword other
> than `public` for this purpose would be a separate issue.)
>
>
> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ClassFileAccPublic
>
From forax at univ-mlv.fr Wed Jul 6 16:24:12 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Wed, 6 Jul 2016 18:24:12 +0200 (CEST)
Subject: Proposal: #FixClassFileFormat
In-Reply-To: <20160706074917.217942295eggemoggin.niobe.net>
References: <1813643129.1709819.1467357511504.JavaMail.zimbra@u-pem.fr>
<1751220489.1722677.1467358740078.JavaMail.zimbra@u-pem.fr>
<20160701104315.789031669eggemoggin.niobe.net>
<321002664.2003394.1467404425232.JavaMail.zimbra@u-pem.fr>
<20160706074917.217942295eggemoggin.niobe.net>
Message-ID: <1055247880.1414152.1467822252693.JavaMail.zimbra@u-pem.fr>
thanks,
R?mi
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: forax at univ-mlv.fr
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Mercredi 6 Juillet 2016 16:49:17
> Objet: Re: Proposal: #FixClassFileFormat
>
> 2016/7/1 13:20:25 -0700, Remi Forax <forax at univ-mlv.fr>:
> > ...
> >
> > and i've forgotten a third issue, ACC_MODULE is specified as 0x8000,
> > 0x8000 is very special value because it's the only one that left which
> > is available on class, method and field. Historically, in the first
> > specs, "module" was a modifier like public or private thus using the
> > same value for all the elements was making sense, but now ACC_MODULE
> > can only applied on class so using a value available on all the
> > elements make less sense, the value 0x8000 should be free in order to
> > be used later. The possible value for ACC_MODULE are: 0x0008
> > (ACC_STATIC), 0x0040 (ACC_VOLATILE or ACC_BRIDGE), 0x0080 (ACC_VARARGS
> > or ACC_TRANSIENT), 0x0100 (ACC_NATIVE), 0x0800 (ACC_STRICT). So
> > either, 0x0040 or 0x0080 are the best candidate because the other
> > values are only valid either on field or on method but not on
> > both. And now, because 0x0080 is a permutation of 0x8000 in term of
> > characters , i think 0x0040 is the best value for ACC_MODULE.
> >
> > Changing the value of ACC_MODULE is interesting if it's done at the
> > same time as the other changes because the compiler and the VM can
> > easily detect and reject the classfile encoded with what will become
> > an old format.
>
> I'll enter this in the issue list as:
>
> #ClassFileAccModule_ --- The `ACC_MODULE` constant is currently
> specified to have the value `0x8000`. This is the last available bit
> remaining across all of the various `access_flags` fields of a class
> file, and thus should be reserved for some unspecified future purpose
> where it may be useful to use the same value in all such fields.
> Alternative candidates for `ACC_MODULE` include `0x0040` (overlaps
> with `ACC_VOLATILE` and `ACC_BRIDGE`) and `0x0080` (`ACC_TRANSIENT`
> and `ACC_VARARGS`).
>
> - Mark
>
From forax at univ-mlv.fr Wed Jul 6 16:48:19 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 6 Jul 2016 18:48:19 +0200 (CEST)
Subject: Proposal: #DefaultModule
In-Reply-To: <577BC730.1040706@redhat.com>
References: <577BC730.1040706@redhat.com>
Message-ID: <1007392272.1419315.1467823699221.JavaMail.zimbra@u-pem.fr>
Hi David,
Correct me if i'm wrong,
it seems like the proposal to be able to specify how to find the name and the version of an automatic module (i.e. #CustomizableAutomaticModuleNameMapping) but for the default module.
The idea is that an existing module systems will be able to provide a name and a version for the default module of the layers it controls.
so this issue should be named #CustomizableDefaultModuleNameMapping and i'm fine with it
(obviously the devil is in the detail, i.e. how to do implement that ?)
and the name "default module" seems to be a better name that the unamed module.
When naming something, avoid name that refers to a property and use name that refers to the concept said an old professor of me.
R?mi
----- Mail original -----
> De: "David M. Lloyd" <david.lloyd at redhat.com>
> ?: jpms-spec-experts at openjdk.java.net
> Cc: "jigsaw-dev" <jigsaw-dev at openjdk.java.net>
> Envoy?: Mardi 5 Juillet 2016 16:41:52
> Objet: Proposal: #DefaultModule
>
> I propose that the concept of "unnamed module" be dropped in favor of
> "default module". The main difference is that the class loader (or
> module finder or layer configuration or someone else) would be allowed
> to (but not required to) assign a free-form name and version string to
> this module. This would allow existing module systems to bring their
> module concept into some form of consonance with Jigsaw without
> compromising any of the restrictions that Jigsaw-style modules have.
>
> Effecting this change would suggest the addition of an "isDefault()"
> method on Module, possibly replacing "isNamed()" (which is arguably
> already somewhat redundant with respect to getName()). Also at some
> stage, something would have to establish the default module name and
> version strings, probably defaulting to the (current) null strings to
> keep a stable status quo.
>
> --
> - DML
>
From pbenedict at apache.org Wed Jul 6 17:10:50 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Wed, 6 Jul 2016 12:10:50 -0500
Subject: Proposal: #DefaultModule
In-Reply-To: <1007392272.1419315.1467823699221.JavaMail.zimbra@u-pem.fr>
References: <577BC730.1040706@redhat.com>
<1007392272.1419315.1467823699221.JavaMail.zimbra@u-pem.fr>
Message-ID: <CABLGb9zBw4aFvNAfMUPgTrpB2dGbBQySp2UetTMfbFpP_PozRA@mail.gmail.com>
The only problem, I see, with renaming the "unnamed" to "default" module is
that it also changes the semantics. The unnnamed module has no name so it
cannot be depended upon by a named module. However, once you begin calling
it the "default" module and allow a name to be assigned, it no longer makes
sense for the current restrictions.
Is the purpose of #DefaultModule to also allow normal modules to explicitly
depend on the "default" module? Since it could have a name, I don't see why
it couldn't technically -- but it changes the design of the module system.
Cheers,
Paul
On Wed, Jul 6, 2016 at 11:48 AM, Remi Forax <forax at univ-mlv.fr> wrote:
> Hi David,
> Correct me if i'm wrong,
> it seems like the proposal to be able to specify how to find the name and
> the version of an automatic module (i.e.
> #CustomizableAutomaticModuleNameMapping) but for the default module.
> The idea is that an existing module systems will be able to provide a name
> and a version for the default module of the layers it controls.
>
> so this issue should be named #CustomizableDefaultModuleNameMapping and
> i'm fine with it
> (obviously the devil is in the detail, i.e. how to do implement that ?)
>
> and the name "default module" seems to be a better name that the unamed
> module.
> When naming something, avoid name that refers to a property and use name
> that refers to the concept said an old professor of me.
>
> R?mi
>
> ----- Mail original -----
> > De: "David M. Lloyd" <david.lloyd at redhat.com>
> > ?: jpms-spec-experts at openjdk.java.net
> > Cc: "jigsaw-dev" <jigsaw-dev at openjdk.java.net>
> > Envoy?: Mardi 5 Juillet 2016 16:41:52
> > Objet: Proposal: #DefaultModule
> >
> > I propose that the concept of "unnamed module" be dropped in favor of
> > "default module". The main difference is that the class loader (or
> > module finder or layer configuration or someone else) would be allowed
> > to (but not required to) assign a free-form name and version string to
> > this module. This would allow existing module systems to bring their
> > module concept into some form of consonance with Jigsaw without
> > compromising any of the restrictions that Jigsaw-style modules have.
> >
> > Effecting this change would suggest the addition of an "isDefault()"
> > method on Module, possibly replacing "isNamed()" (which is arguably
> > already somewhat redundant with respect to getName()). Also at some
> > stage, something would have to establish the default module name and
> > version strings, probably defaulting to the (current) null strings to
> > keep a stable status quo.
> >
> > --
> > - DML
> >
>
From david.lloyd at redhat.com Wed Jul 6 17:22:50 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 6 Jul 2016 12:22:50 -0500
Subject: Proposal: #DefaultModule
In-Reply-To: <CABLGb9zBw4aFvNAfMUPgTrpB2dGbBQySp2UetTMfbFpP_PozRA@mail.gmail.com>
References: <577BC730.1040706@redhat.com>
<1007392272.1419315.1467823699221.JavaMail.zimbra@u-pem.fr>
<CABLGb9zBw4aFvNAfMUPgTrpB2dGbBQySp2UetTMfbFpP_PozRA@mail.gmail.com>
Message-ID: <577D3E6A.4000804@redhat.com>
No, the intent is that default modules are still outside of resolution
altogether. Being unnamed isn't what puts the module outside the system;
it's just that you have to *have* one outside the system in order to
ensure that all classes have a Module instance, so I think we ought to
be able to put a name and version on it (ideally free-form, not subject
to the restrictions of a layer which otherwise has no control over this
module anyway).
I don't want to change the design of the module system to accommodate
this change, which is basically just allowing two fields to be filled in.
On 07/06/2016 12:10 PM, Paul Benedict wrote:
> The only problem, I see, with renaming the "unnamed" to "default" module
> is that it also changes the semantics. The unnnamed module has no name
> so it cannot be depended upon by a named module. However, once you begin
> calling it the "default" module and allow a name to be assigned, it no
> longer makes sense for the current restrictions.
>
> Is the purpose of #DefaultModule to also allow normal modules to
> explicitly depend on the "default" module? Since it could have a name, I
> don't see why it couldn't technically -- but it changes the design of
> the module system.
>
> Cheers,
> Paul
>
> On Wed, Jul 6, 2016 at 11:48 AM, Remi Forax <forax at univ-mlv.fr
> <mailto:forax at univ-mlv.fr>> wrote:
>
> Hi David,
> Correct me if i'm wrong,
> it seems like the proposal to be able to specify how to find the
> name and the version of an automatic module (i.e.
> #CustomizableAutomaticModuleNameMapping) but for the default module.
> The idea is that an existing module systems will be able to provide
> a name and a version for the default module of the layers it controls.
>
> so this issue should be named #CustomizableDefaultModuleNameMapping
> and i'm fine with it
> (obviously the devil is in the detail, i.e. how to do implement that ?)
>
> and the name "default module" seems to be a better name that the
> unamed module.
> When naming something, avoid name that refers to a property and use
> name that refers to the concept said an old professor of me.
>
> R?mi
>
> ----- Mail original -----
> > De: "David M. Lloyd" <david.lloyd at redhat.com <mailto:david.lloyd at redhat.com>>
> > ?:jpms-spec-experts at openjdk.java.net
> <mailto:jpms-spec-experts at openjdk.java.net>
> > Cc: "jigsaw-dev" <jigsaw-dev at openjdk.java.net <mailto:jigsaw-dev at openjdk.java.net>>
> > Envoy?: Mardi 5 Juillet 2016 16:41:52
> > Objet: Proposal: #DefaultModule
> >
> > I propose that the concept of "unnamed module" be dropped in favor of
> > "default module". The main difference is that the class loader (or
> > module finder or layer configuration or someone else) would be allowed
> > to (but not required to) assign a free-form name and version string to
> > this module. This would allow existing module systems to bring their
> > module concept into some form of consonance with Jigsaw without
> > compromising any of the restrictions that Jigsaw-style modules have.
> >
> > Effecting this change would suggest the addition of an "isDefault()"
> > method on Module, possibly replacing "isNamed()" (which is arguably
> > already somewhat redundant with respect to getName()). Also at some
> > stage, something would have to establish the default module name and
> > version strings, probably defaulting to the (current) null strings to
> > keep a stable status quo.
> >
> > --
> > - DML
> >
>
>
--
- DML
From pbenedict at apache.org Wed Jul 6 17:32:48 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Wed, 6 Jul 2016 12:32:48 -0500
Subject: Proposal: #DefaultModule
In-Reply-To: <577D3E6A.4000804@redhat.com>
References: <577BC730.1040706@redhat.com>
<1007392272.1419315.1467823699221.JavaMail.zimbra@u-pem.fr>
<CABLGb9zBw4aFvNAfMUPgTrpB2dGbBQySp2UetTMfbFpP_PozRA@mail.gmail.com>
<577D3E6A.4000804@redhat.com>
Message-ID: <CABLGb9ypK2E6z1shLn0duec9ZZuWW5_JUd+O_uvUTEm7Z6pbaQ@mail.gmail.com>
Okay. Well I still think it's strange for the default module to have a
name. I'm pretty sure it's meant to be analogous to the default package
which has no name either. It's the lack of a name that keeps it out of
resolution. Though to your point, maybe it's not a name you're looking for,
per se, as it is dynamic metadata. What is a Module had a Properties or
Map<String,Object>? Your need sounds custom enough that you could stuff it
into attributes.
Cheers,
Paul
On Wed, Jul 6, 2016 at 12:22 PM, David M. Lloyd <david.lloyd at redhat.com>
wrote:
> No, the intent is that default modules are still outside of resolution
> altogether. Being unnamed isn't what puts the module outside the system;
> it's just that you have to *have* one outside the system in order to ensure
> that all classes have a Module instance, so I think we ought to be able to
> put a name and version on it (ideally free-form, not subject to the
> restrictions of a layer which otherwise has no control over this module
> anyway).
>
> I don't want to change the design of the module system to accommodate this
> change, which is basically just allowing two fields to be filled in.
>
> On 07/06/2016 12:10 PM, Paul Benedict wrote:
>
>> The only problem, I see, with renaming the "unnamed" to "default" module
>> is that it also changes the semantics. The unnnamed module has no name
>> so it cannot be depended upon by a named module. However, once you begin
>> calling it the "default" module and allow a name to be assigned, it no
>> longer makes sense for the current restrictions.
>>
>> Is the purpose of #DefaultModule to also allow normal modules to
>> explicitly depend on the "default" module? Since it could have a name, I
>> don't see why it couldn't technically -- but it changes the design of
>> the module system.
>>
>> Cheers,
>> Paul
>>
>> On Wed, Jul 6, 2016 at 11:48 AM, Remi Forax <forax at univ-mlv.fr
>> <mailto:forax at univ-mlv.fr>> wrote:
>>
>> Hi David,
>> Correct me if i'm wrong,
>> it seems like the proposal to be able to specify how to find the
>> name and the version of an automatic module (i.e.
>> #CustomizableAutomaticModuleNameMapping) but for the default module.
>> The idea is that an existing module systems will be able to provide
>> a name and a version for the default module of the layers it controls.
>>
>> so this issue should be named #CustomizableDefaultModuleNameMapping
>> and i'm fine with it
>> (obviously the devil is in the detail, i.e. how to do implement that
>> ?)
>>
>> and the name "default module" seems to be a better name that the
>> unamed module.
>> When naming something, avoid name that refers to a property and use
>> name that refers to the concept said an old professor of me.
>>
>> R?mi
>>
>> ----- Mail original -----
>> > De: "David M. Lloyd" <david.lloyd at redhat.com <mailto:
>> david.lloyd at redhat.com>>
>> > ?:jpms-spec-experts at openjdk.java.net
>> <mailto:jpms-spec-experts at openjdk.java.net>
>> > Cc: "jigsaw-dev" <jigsaw-dev at openjdk.java.net <mailto:
>> jigsaw-dev at openjdk.java.net>>
>> > Envoy?: Mardi 5 Juillet 2016 16:41:52
>> > Objet: Proposal: #DefaultModule
>> >
>> > I propose that the concept of "unnamed module" be dropped in favor
>> of
>> > "default module". The main difference is that the class loader (or
>> > module finder or layer configuration or someone else) would be
>> allowed
>> > to (but not required to) assign a free-form name and version string
>> to
>> > this module. This would allow existing module systems to bring
>> their
>> > module concept into some form of consonance with Jigsaw without
>> > compromising any of the restrictions that Jigsaw-style modules have.
>> >
>> > Effecting this change would suggest the addition of an "isDefault()"
>> > method on Module, possibly replacing "isNamed()" (which is arguably
>> > already somewhat redundant with respect to getName()). Also at some
>> > stage, something would have to establish the default module name and
>> > version strings, probably defaulting to the (current) null strings
>> to
>> > keep a stable status quo.
>> >
>> > --
>> > - DML
>> >
>>
>>
>>
> --
> - DML
>
From david.lloyd at redhat.com Wed Jul 6 17:40:22 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 6 Jul 2016 12:40:22 -0500
Subject: Proposal: #DefaultModule
In-Reply-To: <CABLGb9ypK2E6z1shLn0duec9ZZuWW5_JUd+O_uvUTEm7Z6pbaQ@mail.gmail.com>
References: <577BC730.1040706@redhat.com>
<1007392272.1419315.1467823699221.JavaMail.zimbra@u-pem.fr>
<CABLGb9zBw4aFvNAfMUPgTrpB2dGbBQySp2UetTMfbFpP_PozRA@mail.gmail.com>
<577D3E6A.4000804@redhat.com>
<CABLGb9ypK2E6z1shLn0duec9ZZuWW5_JUd+O_uvUTEm7Z6pbaQ@mail.gmail.com>
Message-ID: <577D4286.2070506@redhat.com>
Unfortunately attributes cannot appear in stack traces, and there is
value in returning something meaningful for getName() in that module as
well.
Consider that OSGi modules (among other things) can never be Jigsaw
modules; at least it would be useful to allow them to have a clean
appearance in stack traces and have a meaningful result for getName().
Thanks to the possibility of multiple layers, it is already not
guaranteed that loading a module with a name found in getName() will
yield that module. Also consider that default modules actually do not
have a Layer at all.
Again it's *not* the lack of a name that keeps it out of resolution.
It's being outside of resolution that caused it to have no name - but
this is not a necessary condition for the existence of this module;
there *must* be some default in order for all classes to have a module
(even if they are loaded outside the module system). The class loader
should be allowed to choose the name of this default.
On 07/06/2016 12:32 PM, Paul Benedict wrote:
> Okay. Well I still think it's strange for the default module to have a
> name. I'm pretty sure it's meant to be analogous to the default package
> which has no name either. It's the lack of a name that keeps it out of
> resolution. Though to your point, maybe it's not a name you're looking
> for, per se, as it is dynamic metadata. What is a Module had a
> Properties or Map<String,Object>? Your need sounds custom enough that
> you could stuff it into attributes.
>
> Cheers,
> Paul
>
> On Wed, Jul 6, 2016 at 12:22 PM, David M. Lloyd <david.lloyd at redhat.com
> <mailto:david.lloyd at redhat.com>> wrote:
>
> No, the intent is that default modules are still outside of
> resolution altogether. Being unnamed isn't what puts the module
> outside the system; it's just that you have to *have* one outside
> the system in order to ensure that all classes have a Module
> instance, so I think we ought to be able to put a name and version
> on it (ideally free-form, not subject to the restrictions of a layer
> which otherwise has no control over this module anyway).
>
> I don't want to change the design of the module system to
> accommodate this change, which is basically just allowing two fields
> to be filled in.
>
> On 07/06/2016 12:10 PM, Paul Benedict wrote:
>
> The only problem, I see, with renaming the "unnamed" to
> "default" module
> is that it also changes the semantics. The unnnamed module has
> no name
> so it cannot be depended upon by a named module. However, once
> you begin
> calling it the "default" module and allow a name to be assigned,
> it no
> longer makes sense for the current restrictions.
>
> Is the purpose of #DefaultModule to also allow normal modules to
> explicitly depend on the "default" module? Since it could have a
> name, I
> don't see why it couldn't technically -- but it changes the
> design of
> the module system.
>
> Cheers,
> Paul
>
> On Wed, Jul 6, 2016 at 11:48 AM, Remi Forax <forax at univ-mlv.fr
> <mailto:forax at univ-mlv.fr>
> <mailto:forax at univ-mlv.fr <mailto:forax at univ-mlv.fr>>> wrote:
>
> Hi David,
> Correct me if i'm wrong,
> it seems like the proposal to be able to specify how to
> find the
> name and the version of an automatic module (i.e.
> #CustomizableAutomaticModuleNameMapping) but for the
> default module.
> The idea is that an existing module systems will be able to
> provide
> a name and a version for the default module of the layers
> it controls.
>
> so this issue should be named
> #CustomizableDefaultModuleNameMapping
> and i'm fine with it
> (obviously the devil is in the detail, i.e. how to do
> implement that ?)
>
> and the name "default module" seems to be a better name
> that the
> unamed module.
> When naming something, avoid name that refers to a property
> and use
> name that refers to the concept said an old professor of me.
>
> R?mi
>
> ----- Mail original -----
> > De: "David M. Lloyd" <david.lloyd at redhat.com
> <mailto:david.lloyd at redhat.com> <mailto:david.lloyd at redhat.com
> <mailto:david.lloyd at redhat.com>>>
> > ?:jpms-spec-experts at openjdk.java.net
> <mailto:jpms-spec-experts at openjdk.java.net>
> <mailto:jpms-spec-experts at openjdk.java.net
> <mailto:jpms-spec-experts at openjdk.java.net>>
> > Cc: "jigsaw-dev" <jigsaw-dev at openjdk.java.net
> <mailto:jigsaw-dev at openjdk.java.net>
> <mailto:jigsaw-dev at openjdk.java.net
> <mailto:jigsaw-dev at openjdk.java.net>>>
> > Envoy?: Mardi 5 Juillet 2016 16:41:52
> > Objet: Proposal: #DefaultModule
> >
> > I propose that the concept of "unnamed module" be dropped
> in favor of
> > "default module". The main difference is that the class
> loader (or
> > module finder or layer configuration or someone else)
> would be allowed
> > to (but not required to) assign a free-form name and
> version string to
> > this module. This would allow existing module systems to
> bring their
> > module concept into some form of consonance with Jigsaw
> without
> > compromising any of the restrictions that Jigsaw-style
> modules have.
> >
> > Effecting this change would suggest the addition of an
> "isDefault()"
> > method on Module, possibly replacing "isNamed()" (which
> is arguably
> > already somewhat redundant with respect to getName()).
> Also at some
> > stage, something would have to establish the default
> module name and
> > version strings, probably defaulting to the (current)
> null strings to
> > keep a stable status quo.
> >
> > --
> > - DML
> >
>
>
>
> --
> - DML
>
>
--
- DML
From mark.reinhold at oracle.com Wed Jul 6 23:13:26 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 06 Jul 2016 16:13:26 -0700
Subject: Updated EA builds with initial implementations of current proposals
Message-ID: <20160706161326.25590246eggemoggin.niobe.net>
FYI:
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-July/008467.html
- Mark
From mark.reinhold at oracle.com Wed Jul 6 23:37:11 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 06 Jul 2016 16:37:11 -0700
Subject: Draft JLS and JVMS changes updated
Message-ID: <20160706163711.319138800eggemoggin.niobe.net>
http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html
This version includes changes for the proposals for:
#CompileTimeDependences (`requires static`)
#ReflectiveAccessToNonExportedTypes (`exports dynamic`)
#ModuleAnnotations
#ClassFileAccPublic
It also defines class-file attributes for concealed packages, version
strings, main classes, and target platform information.
- Mark
From forax at univ-mlv.fr Thu Jul 7 08:09:33 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Thu, 7 Jul 2016 10:09:33 +0200 (CEST)
Subject: Draft JLS and JVMS changes updated
In-Reply-To: <20160706163711.319138800eggemoggin.niobe.net>
References: <20160706163711.319138800eggemoggin.niobe.net>
Message-ID: <123878428.1541468.1467878973727.JavaMail.zimbra@u-pem.fr>
Hi Mark,
There is a slight difference between the implementation and the spec,
the current implementation in the jake repository allows exports_flags to be a combination of ACC_MANDATED and ACC_SYNTHETIC too.
R?mi
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: jpms-spec-experts at openjdk.java.net
> Envoy?: Jeudi 7 Juillet 2016 01:37:11
> Objet: Draft JLS and JVMS changes updated
>
>
> http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html
>
> This version includes changes for the proposals for:
>
> #CompileTimeDependences (`requires static`)
> #ReflectiveAccessToNonExportedTypes (`exports dynamic`)
> #ModuleAnnotations
> #ClassFileAccPublic
>
> It also defines class-file attributes for concealed packages, version
> strings, main classes, and target platform information.
>
> - Mark
>
From pbenedict at apache.org Thu Jul 7 14:52:18 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Thu, 7 Jul 2016 09:52:18 -0500
Subject: Updated EA builds with initial implementations of current
proposals
In-Reply-To: <577d90b7.67e5420a.728c9.73ecSMTPIN_ADDED_BROKEN@mx.google.com>
References: <577d90b7.67e5420a.728c9.73ecSMTPIN_ADDED_BROKEN@mx.google.com>
Message-ID: <CABLGb9wN3LM+Mpdun8nM0GCJY5QE55mJ3cFRQL02uvrB+n97Dg@mail.gmail.com>
Hi Mark. Do these set of changes mean those alternative proposals are now
set in stone? I didn't know feedback was finished. For example, and I may
have missed this, but I can't recall one message in support of the
"requires static" syntax. As I said, I may have missed the supporters, but
I don't recall anyone championing that exact syntax. Almost everyone
objected to the use of "static" as a misleading use of the keyword.
On the other hand, if this is just a tentative change to move the feature
set along, that makes more sense. I just want to understand the
expectation. Thank you.
Cheers,
Paul
On Wed, Jul 6, 2016 at 6:13 PM, <mark.reinhold at oracle.com> wrote:
> FYI:
>
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-July/008467.html
>
> - Mark
>
From pbenedict at apache.org Thu Jul 7 15:05:50 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Thu, 7 Jul 2016 10:05:50 -0500
Subject: Updated EA builds with initial implementations of current
proposals
In-Reply-To: <577E6E15.9030903@redhat.com>
References: <577d90b7.67e5420a.728c9.73ecSMTPIN_ADDED_BROKEN@mx.google.com>
<CABLGb9wN3LM+Mpdun8nM0GCJY5QE55mJ3cFRQL02uvrB+n97Dg@mail.gmail.com>
<577E6E15.9030903@redhat.com>
Message-ID: <CABLGb9xYsHqM5hKjSOMfMbirMa1Xs1w6Sn6p5xbHNG-yakAj-w@mail.gmail.com>
Thanks Andrew. Lots of emails flying around. I picked the wrong one. I do
see he made it clear in another post. Thanks.
Cheers,
Paul
On Thu, Jul 7, 2016 at 9:58 AM, Andrew Dinn <adinn at redhat.com> wrote:
>
>
> On 07/07/16 15:52, Paul Benedict wrote:
> > Hi Mark. Do these set of changes mean those alternative proposals are now
> > set in stone? I didn't know feedback was finished. For example, and I may
> > have missed this, but I can't recall one message in support of the
> > "requires static" syntax. As I said, I may have missed the supporters,
> but
> > I don't recall anyone championing that exact syntax. Almost everyone
> > objected to the use of "static" as a misleading use of the keyword.
> >
> > On the other hand, if this is just a tentative change to move the feature
> > set along, that makes more sense. I just want to understand the
> > expectation. Thank you.
>
> I though Alan's note made this clear
>
> "The jigsaw/jake forest has been updated with an initial implementation
> of the proposals that Mark brought to the jpms-spec-experts mailing list
> last week.
>
> ...
>
> The discussion on some of these issues is ongoing so don't treat
> anything as final yet. We'll refresh the builds as needed over the
> coming weeks."
>
> I believe the idea is to allow people to try out the proposals to see
> how (and how well) they work. I'm certainly glad of the chance to
> ascertain whether #ReflectiveAccessByInstrumentationAgents is up to
> scratch or not.
>
> regards,
>
>
> Andrew Dinn
> -----------
> Senior Principal Software Engineer
> Red Hat UK Ltd
> Registered in England and Wales under Company Registration No. 03798903
> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
>
From mark.reinhold at oracle.com Thu Jul 7 15:22:43 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu, 07 Jul 2016 08:22:43 -0700
Subject: Updated EA builds with initial implementations of current
proposals
In-Reply-To: <577E6E15.9030903@redhat.com>
References: <577d90b7.67e5420a.728c9.73ecSMTPIN_ADDED_BROKEN@mx.google.com>
<CABLGb9wN3LM+Mpdun8nM0GCJY5QE55mJ3cFRQL02uvrB+n97Dg@mail.gmail.com>
<577E6E15.9030903@redhat.com>
Message-ID: <20160707082243.833051787eggemoggin.niobe.net>
2016/7/7 7:58:29 -0700, adinn at redhat.com:
> On 07/07/16 15:52, Paul Benedict wrote:
>> Hi Mark. Do these set of changes mean those alternative proposals are now
>> set in stone? I didn't know feedback was finished. ...
>
> I though Alan's note made this clear
>
> "The jigsaw/jake forest has been updated with an initial implementation
> of the proposals that Mark brought to the jpms-spec-experts mailing list
> last week.
>
> ...
>
> The discussion on some of these issues is ongoing so don't treat
> anything as final yet. We'll refresh the builds as needed over the
> coming weeks."
>
> I believe the idea is to allow people to try out the proposals to see
> how (and how well) they work. I'm certainly glad of the chance to
> ascertain whether #ReflectiveAccessByInstrumentationAgents is up to
> scratch or not.
Exactly.
If and when an open design issue is settled then it will be marked as
such in the issue list. Until then, what you see in the jake forest
and the EA builds is just a prototype for experimentation.
- Mark
From pbenedict at apache.org Thu Jul 7 19:20:18 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Thu, 7 Jul 2016 14:20:18 -0500
Subject: Proposal: #DefaultModule
In-Reply-To: <05cffdb45138a90643cee9400292e9fa@org.tizen.email>
References: <05cffdb45138a90643cee9400292e9fa@org.tizen.email>
Message-ID: <CABLGb9y3mmm449AZoH7Pcq4PCo7T6d_8iZ8LoUgZFPh_A-iW5g@mail.gmail.com>
Clearly the unnamed/default module is an aggregation of many artifacts. I
still don't find the unnamed/default module name useful, but the artifact
enmeshed in the aggregate is. Not only should the stack trace provide this
info, but there should be an API for it too. WDYT?
Cheers,
Paul
On Wed, Jul 6, 2016 at 6:05 PM, Peter Firmstone <peter.firmstone at zeus.net.au
> wrote:
> Hmm, be useful for debugging.
>
> Peter.
>
> Sent from my Samsung device.
>
> ---- Original message ----
> From: David M. Lloyd <david.lloyd at redhat.com>
> Sent: 07/07/2016 03:40:22 am
> To: Paul Benedict <pbenedict at apache.org>
> Cc: jigsaw-dev <jigsaw-dev at openjdk.java.net>; Java Platform Module System
> (JSR 376) Expert Group Observers <jpms-spec-observers at openjdk.java.net>
> Subject: Re: Proposal: #DefaultModule
>
> Unfortunately attributes cannot appear in stack traces, and there is
> value in returning something meaningful for getName() in that module as
> well.
>
> Consider that OSGi modules (among other things) can never be Jigsaw
> modules; at least it would be useful to allow them to have a clean
> appearance in stack traces and have a meaningful result for getName().
> Thanks to the possibility of multiple layers, it is already not
> guaranteed that loading a module with a name found in getName() will
> yield that module. Also consider that default modules actually do not
> have a Layer at all.
>
> Again it's *not* the lack of a name that keeps it out of resolution.
> It's being outside of resolution that caused it to have no name - but
> this is not a necessary condition for the existence of this module;
> there *must* be some default in order for all classes to have a module
> (even if they are loaded outside the module system). The class loader
> should be allowed to choose the name of this default.
>
> On 07/06/2016 12:32 PM, Paul Benedict wrote:
> > Okay. Well I still think it's strange for the default module to have a
> > name. I'm pretty sure it's meant to be analogous to the default package
> > which has no name either. It's the lack of a name that keeps it out of
> > resolution. Though to your point, maybe it's not a name you're looking
> > for, per se, as it is dynamic metadata. What is a Module had a
> > Properties or Map<String,Object>? Your need sounds custom enough that
> > you could stuff it into attributes.
> >
> > Cheers,
> > Paul
> >
> > On Wed, Jul 6, 2016 at 12:22 PM, David M. Lloyd <david.lloyd at redhat.com
> > <mailto:david.lloyd at redhat.com>> wrote:
> >
> > No, the intent is that default modules are still outside of
> > resolution altogether. Being unnamed isn't what puts the module
> > outside the system; it's just that you have to *have* one outside
> > the system in order to ensure that all classes have a Module
> > instance, so I think we ought to be able to put a name and version
> > on it (ideally free-form, not subject to the restrictions of a layer
> > which otherwise has no control over this module anyway).
> >
> > I don't want to change the design of the module system to
> > accommodate this change, which is basically just allowing two fields
> > to be filled in.
> >
> > On 07/06/2016 12:10 PM, Paul Benedict wrote:
> >
> > The only problem, I see, with renaming the "unnamed" to
> > "default" module
> > is that it also changes the semantics. The unnnamed module has
> > no name
> > so it cannot be depended upon by a named module. However, once
> > you begin
> > calling it the "default" module and allow a name to be assigned,
> > it no
> > longer makes sense for the current restrictions.
> >
> > Is the purpose of #DefaultModule to also allow normal modules to
> > explicitly depend on the "default" module? Since it could have a
> > name, I
> > don't see why it couldn't technically -- but it changes the
> > design of
> > the module system.
> >
> > Cheers,
> > Paul
> >
> > On Wed, Jul 6, 2016 at 11:48 AM, Remi Forax <forax at univ-mlv.fr
> > <mailto:forax at univ-mlv.fr>
> > <mailto:forax at univ-mlv.fr <mailto:forax at univ-mlv.fr>>> wrote:
> >
> > Hi David,
> > Correct me if i'm wrong,
> > it seems like the proposal to be able to specify how to
> > find the
> > name and the version of an automatic module (i.e.
> > #CustomizableAutomaticModuleNameMapping) but for the
> > default module.
> > The idea is that an existing module systems will be able to
> > provide
> > a name and a version for the default module of the layers
> > it controls.
> >
> > so this issue should be named
> > #CustomizableDefaultModuleNameMapping
> > and i'm fine with it
> > (obviously the devil is in the detail, i.e. how to do
> > implement that ?)
> >
> > and the name "default module" seems to be a better name
> > that the
> > unamed module.
> > When naming something, avoid name that refers to a property
> > and use
> > name that refers to the concept said an old professor of me.
>
> >
> > R?mi
> >
> > ----- Mail original -----
> > > De: "David M. Lloyd" <david.lloyd at redhat.com
> > <mailto:david.lloyd at redhat.com> <mailto:david.lloyd at redhat.com
> > <mailto:david.lloyd at redhat.com>>>
> > > ?:jpms-spec-experts at openjdk.java.net
> > <mailto:jpms-spec-experts at openjdk.java.net>
> > <mailto:jpms-spec-experts at openjdk.java.net
> > <mailto:jpms-spec-experts at openjdk.java.net>>
> > > Cc: "jigsaw-dev" <jigsaw-dev at openjdk.java.net
> > <mailto:jigsaw-dev at openjdk.java.net>
> > <mailto:jigsaw-dev at openjdk.java.net
> > <mailto:jigsaw-dev at openjdk.java.net>>>
> > > Envoy?: Mardi 5 Juillet 2016 16:41:52
> > > Objet: Proposal: #DefaultModule
> > >
> > > I propose that the concept of "unnamed module" be dropped
> > in favor of
> > > "default module". The main difference is that the class
> > loader (or
> > > module finder or layer configuration or someone else)
> > would be allowed
> > > to (but not required to) assign a free-form name and
> > version string to
> > > this module. This would allow existing module systems to
> > bring their
> > > module concept into some form of consonance with Jigsaw
> > without
> > > compromising any of the restrictions that Jigsaw-style
> > modules have.
> > >
> > > Effecting this change would suggest the addition of an
> > "isDefault()"
> > > method on Module, possibly replacing "isNamed()" (which
> > is arguably
> > > already somewhat redundant with respect to getName()).
> > Also at some
> > > stage, something would have to establish the default
> > module name and
> > > version strings, probably defaulting to the (current)
> > null strings to
> > > keep a stable status quo.
> > >
> > > --
> > > - DML
> > >
> >
> >
> >
> > --
> > - DML
> >
> >
>
> --
> - DML
>
>
From david.lloyd at redhat.com Thu Jul 7 19:44:35 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 7 Jul 2016 14:44:35 -0500
Subject: Proposal: #DefaultModule
In-Reply-To: <CABLGb9y3mmm449AZoH7Pcq4PCo7T6d_8iZ8LoUgZFPh_A-iW5g@mail.gmail.com>
References: <05cffdb45138a90643cee9400292e9fa@org.tizen.email>
<CABLGb9y3mmm449AZoH7Pcq4PCo7T6d_8iZ8LoUgZFPh_A-iW5g@mail.gmail.com>
Message-ID: <577EB123.2020406@redhat.com>
Not always. For example in the very common situation of existing module
systems where there is one module per class loader, if that module is
the default module (which BTW allows these module systems to retain
their existing resolution and delegation logic), it is certain that the
module will correspond to a single named entity if not a single artifact
as well.
On 07/07/2016 02:20 PM, Paul Benedict wrote:
> Clearly the unnamed/default module is an aggregation of many artifacts.
> I still don't find the unnamed/default module name useful, but the
> artifact enmeshed in the aggregate is. Not only should the stack trace
> provide this info, but there should be an API for it too. WDYT?
>
> Cheers,
> Paul
>
> On Wed, Jul 6, 2016 at 6:05 PM, Peter Firmstone
> <peter.firmstone at zeus.net.au <mailto:peter.firmstone at zeus.net.au>> wrote:
>
> Hmm, be useful for debugging.
>
> Peter.
>
> Sent from my Samsung device.
> ---- Original message ----
> From: David M. Lloyd <david.lloyd at redhat.com
> <mailto:david.lloyd at redhat.com>>
> Sent: 07/07/2016 03:40:22 am
> To: Paul Benedict <pbenedict at apache.org <mailto:pbenedict at apache.org>>
> Cc: jigsaw-dev <jigsaw-dev at openjdk.java.net
> <mailto:jigsaw-dev at openjdk.java.net>>; Java Platform Module System
> (JSR 376) Expert Group Observers
> <jpms-spec-observers at openjdk.java.net
> <mailto:jpms-spec-observers at openjdk.java.net>>
> Subject: Re: Proposal: #DefaultModule
>
> Unfortunately attributes cannot appear in stack traces, and there is
> value in returning something meaningful for getName() in that module as
> well.
>
> Consider that OSGi modules (among other things) can never be Jigsaw
> modules; at least it would be useful to allow them to have a clean
> appearance in stack traces and have a meaningful result for getName().
> Thanks to the possibility of multiple layers, it is already not
> guaranteed that loading a module with a name found in getName() will
> yield that module. Also consider that default modules actually do not
> have a Layer at all.
>
> Again it's *not* the lack of a name that keeps it out of resolution.
> It's being outside of resolution that caused it to have no name - but
> this is not a necessary condition for the existence of this module;
> there *must* be some default in order for all classes to have a module
> (even if they are loaded outside the module system). The class loader
> should be allowed to choose the name of this default.
>
> On 07/06/2016 12:32 PM, Paul Benedict wrote:
> > Okay. Well I still think it's strange for the default module to have a
> > name. I'm pretty sure it's meant to be analogous to the default package
> > which has no name either. It's the lack of a name that keeps it out of
> > resolution. Though to your point, maybe it's not a name you're looking
> > for, per se, as it is dynamic metadata. What is a Module had a
> > Properties or Map<String,Object>? Your need sounds custom enough that
> > you could stuff it into attributes.
> >
> > Cheers,
> > Paul
> >
> > On Wed, Jul 6, 2016 at 12:22 PM, David M. Lloyd <david.lloyd at redhat.com <mailto:david.lloyd at redhat.com>
> > <mailto:david.lloyd at redhat.com <mailto:david.lloyd at redhat.com>>> wrote:
> >
> > No, the intent is that default modules are still outside of
> > resolution altogether. Being unnamed isn't what puts the module
> > outside the system; it's just that you have to *have* one outside
> > the system in order to ensure that all classes have a Module
> > instance, so I think we ought to be able to put a name and version
> > on it (ideally free-form, not subject to the restrictions of a layer
> > which otherwise has no control over this module anyway).
> >
> > I don't want to change the design of the module system to
> > accommodate this change, which is basically just allowing two fields
> > to be filled in.
> >
> > On 07/06/2016 12:10 PM, Paul Benedict wrote:
> >
> > The only problem, I see, with renaming the "unnamed" to
> > "default" module
> > is that it also changes the semantics. The unnnamed module has
> > no name
> > so it cannot be depended upon by a named module. However, once
> > you begin
> > calling it the "default" module and allow a name to be assigned,
> > it no
> > longer makes sense for the current restrictions.
> >
> > Is the purpose of #DefaultModule to also allow normal modules to
> > explicitly depend on the "default" module? Since it could have a
> > name, I
> > don't see why it couldn't technically -- but it changes the
> > design of
> > the module system.
> >
> > Cheers,
> > Paul
> >
> > On Wed, Jul 6, 2016 at 11:48 AM, Remi Forax <forax at univ-mlv.fr <mailto:forax at univ-mlv.fr>
> > <mailto:forax at univ-mlv.fr <mailto:forax at univ-mlv.fr>>
> > <mailto:forax at univ-mlv.fr
> <mailto:forax at univ-mlv.fr> <mailto:forax at univ-mlv.fr
> <mailto:forax at univ-mlv.fr>>>> wrote:
> >
> > Hi David,
> > Correct me if i'm wrong,
> > it seems like the proposal to be able to specify how to
> > find the
> > name and the version of an automatic module (i.e.
> > #CustomizableAutomaticModuleNameMapping) but for the
> > default module.
> > The idea is that an existing module systems will be able to
> > provide
> > a name and a version for the default module of the layers
> > it controls.
> >
> > so this issue should be named
> > #CustomizableDefaultModuleNameMapping
> > and i'm fine with it
> > (obviously the devil is in the detail, i.e. how to do
> > implement that ?)
> >
> > and the name "default module" seems to be a better name
> > that the
> > unamed module.
> > When naming something, avoid name that refers to a property
> > and use
> > name that refers to the concept said an old professor of me.
> >
> > R?mi
> >
> > ----- Mail original -----
> > > De: "David M. Lloyd" <david.lloyd at redhat.com <mailto:david.lloyd at redhat.com>
> > <mailto:david.lloyd at redhat.com
> <mailto:david.lloyd at redhat.com>> <mailto:david.lloyd at redhat.com
> <mailto:david.lloyd at redhat.com>
> > <mailto:david.lloyd at redhat.com <mailto:david.lloyd at redhat.com>>>>
> > > ?:jpms-spec-experts at openjdk.java.net
> <mailto:jpms-spec-experts at openjdk.java.net>
> > <mailto:jpms-spec-experts at openjdk.java.net
> <mailto:jpms-spec-experts at openjdk.java.net>>
> > <mailto:jpms-spec-experts at openjdk.java.net
> <mailto:jpms-spec-experts at openjdk.java.net>
> > <mailto:jpms-spec-experts at openjdk.java.net
> <mailto:jpms-spec-experts at openjdk.java.net>>>
> > > Cc: "jigsaw-dev" <jigsaw-dev at openjdk.java.net <mailto:jigsaw-dev at openjdk.java.net>
> > <mailto:jigsaw-dev at openjdk.java.net <mailto:jigsaw-dev at openjdk.java.net>>
> > <mailto:jigsaw-dev at openjdk.java.net <mailto:jigsaw-dev at openjdk.java.net>
> > <mailto:jigsaw-dev at openjdk.java.net <mailto:jigsaw-dev at openjdk.java.net>>>>
> > > Envoy?: Mardi 5 Juillet 2016 16:41:52
> > > Objet: Proposal: #DefaultModule
> > >
> > > I propose that the concept of "unnamed module" be dropped
> > in favor of
> > > "default module". The main difference is that the class
> > loader (or
> > > module finder or layer configuration or someone else)
> > would be allowed
> > > to (but not required to) assign a free-form name and
> > version string to
> > > this module. This would allow existing module systems to
> > bring their
> > > module concept into some form of consonance with Jigsaw
> > without
> > > compromising any of the restrictions that Jigsaw-style
> > modules have.
> > >
> > > Effecting this change would suggest the addition of an
> > "isDefault()"
> > > method on Module, possibly replacing "isNamed()" (which
> > is arguably
> > > already somewhat redundant with respect to getName()).
> > Also at some
> > > stage, something would have to establish the default
> > module name and
> > > version strings, probably defaulting to the (current)
> > null strings to
> > > keep a stable status quo.
> > >
> > > --
> > > - DML
> > >
> >
> >
> >
> > --
> > - DML
> >
> >
>
> --
> - DML
>
>
--
- DML
From volker.berlin at goebel-clan.de Sat Jul 9 14:11:52 2016
From: volker.berlin at goebel-clan.de (Volker Berlin)
Date: Sat, 9 Jul 2016 16:11:52 +0200
Subject: Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`
In-Reply-To: <577C1402.1080704@oracle.com>
References: <20160628211815.87E88B59C4@eggemoggin.niobe.net>
<e432a1f4-603d-17f0-406e-e53c961db756@goebel-clan.de>
<577588EA.6060202@oracle.com>
<4ab19576-4e42-d77a-7669-48ee319e694a@goebel-clan.de>
<577C1402.1080704@oracle.com>
Message-ID: <4a6e4d47-9056-d9aa-4346-d97af72068df@goebel-clan.de>
Alex,
The adaption to JDK 9 was very simple to adapt the changes of the
classloader. Because we have our own module system (plugins) since
years. The only difficult is to make the JDK self available some dynamic
jar files.
Can be that there other, simpler options if we switch to Java 9 as
minimum version. Then we can define our own modules. But we need to
support for many years Java 8 and Java 9.
Volker
Am 05.07.2016 um 22:09 schrieb Alex Buckley:
> Volker,
>
> Even if reflection was enabled by default, you could be sure that JDK
> modules would choose to disable reflection on their internal APIs.
> -XaddExports would still be needed to make your code run.
>
> Also, it seems like your code is sensitive to the application class
> loader whose implementation has changed in JDK 9. Please see JEP 260
> and the "Class loaders" section of JEP 261.
>
> Alex
>
> On 7/2/2016 3:09 PM, Volker Berlin wrote:
>> Alex,
>>
>> It is to compensate bugs or handle bad design of JDK components. Mostly
>> this are classloader problems. The impossible to make the JDK classes
>> via dynamic URLClassLoader available.
>>
>> For example:
>>
>> * register dynamically a JDBC that is not in the appclassloader to
>> all our plugins.
>> * register a URL protocol with a protocol handler that is not in the
>> appclassloader
>>
>> Many hacks are needed for test environments. For example:
>>
>> * the simulation that there is no console (javaw)
>> * inverse classloder hierarchy. In the appclassloader there is a class
>> which implements/extends a class in a dynamic loaded library. The
>> current tested classes and the JUnit tests are typical in the
>> appclassloader
>>
>> This are only some points. There can be more. We have used.
>>
>> Volker Berlin
>> i-net software
>>
>>
>> Am 30.06.2016 um 23:02 schrieb Alex Buckley:
>>> Volker,
>>>
>>> Please share details of why your application needs to reflect over the
>>> internals of another module.
>>>
>>> Alex
>>>
>>> On 6/30/2016 1:09 PM, Volker Berlin wrote:
>>>> Reflection should be enabled by default. My current launcher for
>>>> Java 9
>>>> check if the needed refection is possible. If not possible then it
>>>> restart the application with a very long list of -XaddExports
>>>> switches.
>>>> I think this is not what we want.
>>>>
>>>> Volker Berlin
>>>> i-net software
>>>>
>>>>
>>>> Am 28.06.2016 um 23:18 schrieb Mark Reinhold:
>>>>> Issue summary
>>>>> -------------
>>>>>
>>>>> #ReflectiveAccessToNonExportedTypes --- Some kinds of framework
>>>>> libraries require reflective access to members of the non-exported
>>>>> types of other modules; examples include dependency injection
>>>>> (Guice),
>>>>> persistence (JPA), debugging tools, code-automation tools, and
>>>>> serialization (XStream). In some cases the particular library
>>>>> to be
>>>>> used is not known until run time (e.g., Hibernate and EclipseLink
>>>>> both
>>>>> implement JPA). This capability is also sometimes used to work
>>>>> around
>>>>> bugs in unchangeable code. Access to non-exported packages
>>>>> can, at
>>>>> present, only be done via command-line flags, which is extremely
>>>>> awkward. Provide an easier way for reflective code to access such
>>>>> non-exported types. [1]
>>>>>
>>>>> Proposal
>>>>> --------
>>>>>
>>>>> Extend the language of module declarations so that a package can be
>>>>> declared to be exported at run time but not at compile time. This is,
>>>>> roughly, the dual of the `requires static` construct proposed for
>>>>> #CompileTimeDependences, hence we propose to introduce a new
>>>>> modifier,
>>>>> `dynamic`, for use on the `exports` directive. It has the following
>>>>> meanings:
>>>>>
>>>>> - At compile time, `exports dynamic P` does not cause the package
>>>>> `P`
>>>>> to be exported, though it does require `P` to be a package
>>>>> defined
>>>>> in the module.
>>>>>
>>>>> - In phases after compile time, `exports dynamic P` behaves in
>>>>> exactly
>>>>> the same way as `exports P`. It therefore takes part fully in
>>>>> resolution and configuration, and is subject to the same
>>>>> consistency
>>>>> constraints as normally-exported packages (e.g., no split
>>>>> packages).
>>>
>>
>
From pbenedict at apache.org Mon Jul 11 20:10:57 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Mon, 11 Jul 2016 15:10:57 -0500
Subject: Proposal: #ClassFileAccPublic
In-Reply-To: <823657001.1414112.1467822232509.JavaMail.zimbra@u-pem.fr>
References: <20160706153236.A62DBB6B74@eggemoggin.niobe.net>
<823657001.1414112.1467822232509.JavaMail.zimbra@u-pem.fr>
Message-ID: <CABLGb9x4Zj9ARNN-BwFy2dSaR1R1uaqDEh+hAArDto6chMfLuQ@mail.gmail.com>
Remi, what do you think of preserving ACC_MODULE so that classes which are
exported are flagged with it? It would be an easy way for clients to
inspect a type and see if a type was exported.
Cheers,
Paul
On Wed, Jul 6, 2016 at 11:23 AM, Remi Forax <forax at univ-mlv.fr> wrote:
> ACC_TRANSITIVE looks fine to me,
> even if it really means transitive at one hop only.
>
> R?mi
>
> ----- Mail original -----
> > De: "Mark Reinhold" <mark.reinhold at oracle.com>
> > ?: jpms-spec-experts at openjdk.java.net
> > Envoy?: Mercredi 6 Juillet 2016 17:32:36
> > Objet: Proposal: #ClassFileAccPublic
> >
> > Issue summary
> > -------------
> >
> > #ClassFileAccPublic --- The `ACC_PUBLIC` constant in a `requires_flags`
> > should be encoded as `0x0001`, as it is elsewhere in the JVMS, rather
> > than as `0x0020`, which has different meanings in other contexts.
> >
> > Proposal
> > --------
> >
> > Unlike the other `ACC_PUBLIC` constants, this constant does not express
> > an access mode but, rather, the addition of an implied transitive
> > readability edge from modules that depend upon this module to some other
> > module required by this module. To avoid confusion at the class-file
> > level, rename this constant to `ACC_TRANSITIVE`.
> >
> > (Whether the syntax of module declarations should use a keyword other
> > than `public` for this purpose would be a separate issue.)
> >
> >
> > [1]
> http://openjdk.java.net/projects/jigsaw/spec/issues/#ClassFileAccPublic
> >
>
From pbenedict at apache.org Mon Jul 11 20:28:55 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Mon, 11 Jul 2016 15:28:55 -0500
Subject: Proposal: #ClassFileAccPublic
In-Reply-To: <CABLGb9x4Zj9ARNN-BwFy2dSaR1R1uaqDEh+hAArDto6chMfLuQ@mail.gmail.com>
References: <20160706153236.A62DBB6B74@eggemoggin.niobe.net>
<823657001.1414112.1467822232509.JavaMail.zimbra@u-pem.fr>
<CABLGb9x4Zj9ARNN-BwFy2dSaR1R1uaqDEh+hAArDto6chMfLuQ@mail.gmail.com>
Message-ID: <CABLGb9xq_PMkKr3i=gOB002QUzqJxSbSDniRjJUC3uWKS_9XnQ@mail.gmail.com>
Remi, perhaps my question should have been reversed. :-) What about
retaining ACC_MODULE for types that are private to the module (i.e., they
live in packages not exported)? The original use of it can still be
retained.
Cheers,
Paul
On Mon, Jul 11, 2016 at 3:10 PM, Paul Benedict <pbenedict at apache.org> wrote:
> Remi, what do you think of preserving ACC_MODULE so that classes which are
> exported are flagged with it? It would be an easy way for clients to
> inspect a type and see if a type was exported.
>
> Cheers,
> Paul
>
> On Wed, Jul 6, 2016 at 11:23 AM, Remi Forax <forax at univ-mlv.fr> wrote:
>
>> ACC_TRANSITIVE looks fine to me,
>> even if it really means transitive at one hop only.
>>
>> R?mi
>>
>> ----- Mail original -----
>> > De: "Mark Reinhold" <mark.reinhold at oracle.com>
>> > ?: jpms-spec-experts at openjdk.java.net
>> > Envoy?: Mercredi 6 Juillet 2016 17:32:36
>> > Objet: Proposal: #ClassFileAccPublic
>> >
>> > Issue summary
>> > -------------
>> >
>> > #ClassFileAccPublic --- The `ACC_PUBLIC` constant in a
>> `requires_flags`
>> > should be encoded as `0x0001`, as it is elsewhere in the JVMS, rather
>> > than as `0x0020`, which has different meanings in other contexts.
>> >
>> > Proposal
>> > --------
>> >
>> > Unlike the other `ACC_PUBLIC` constants, this constant does not express
>> > an access mode but, rather, the addition of an implied transitive
>> > readability edge from modules that depend upon this module to some other
>> > module required by this module. To avoid confusion at the class-file
>> > level, rename this constant to `ACC_TRANSITIVE`.
>> >
>> > (Whether the syntax of module declarations should use a keyword other
>> > than `public` for this purpose would be a separate issue.)
>> >
>> >
>> > [1]
>> http://openjdk.java.net/projects/jigsaw/spec/issues/#ClassFileAccPublic
>> >
>>
>
>
From forax at univ-mlv.fr Mon Jul 11 22:07:45 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Tue, 12 Jul 2016 00:07:45 +0200 (CEST)
Subject: Proposal: #ClassFileAccPublic
In-Reply-To: <CABLGb9xq_PMkKr3i=gOB002QUzqJxSbSDniRjJUC3uWKS_9XnQ@mail.gmail.com>
References: <20160706153236.A62DBB6B74@eggemoggin.niobe.net>
<823657001.1414112.1467822232509.JavaMail.zimbra@u-pem.fr>
<CABLGb9x4Zj9ARNN-BwFy2dSaR1R1uaqDEh+hAArDto6chMfLuQ@mail.gmail.com>
<CABLGb9xq_PMkKr3i=gOB002QUzqJxSbSDniRjJUC3uWKS_9XnQ@mail.gmail.com>
Message-ID: <1431393728.410276.1468274865333.JavaMail.zimbra@u-pem.fr>
Hi Paul,
I don't think it's a good idea to mark exported or not exported types with a flag.
The level of a type is the wrong level, the package level has the advantage to be already used in the wild (what's why you have doc comments saying that you should not use com.sun.foo by example) and if each class can be marked exported or not, verifying that all types of a package are exported (or not) will be burdensome.
In a well defined component model, duplicating an information already available at an upper level at a lower level is a bad idea, because while it's true that the information may be more easily available, it makes the separate compilation a nightmare. With your proposal, each time i decide to export a package in the module-info, all the classes of that package must be recompiled.
regards,
R?mi
----- Mail original -----
> De: "Paul Benedict" <pbenedict at apache.org>
> ?: "Java Platform Module System (JSR 376) Expert Group Observers"
> <jpms-spec-observers at openjdk.java.net>, forax at univ-mlv.fr
> Cc: "Mark Reinhold" <mark.reinhold at oracle.com>
> Envoy?: Lundi 11 Juillet 2016 22:28:55
> Objet: Re: Proposal: #ClassFileAccPublic
> Remi, perhaps my question should have been reversed. :-) What about retaining
> ACC_MODULE for types that are private to the module (i.e., they live in
> packages not exported)? The original use of it can still be retained.
> Cheers,
> Paul
> On Mon, Jul 11, 2016 at 3:10 PM, Paul Benedict < pbenedict at apache.org >
> wrote:
> > Remi, what do you think of preserving ACC_MODULE so that classes which are
> > exported are flagged with it? It would be an easy way for clients to
> > inspect
> > a type and see if a type was exported.
>
> > Cheers,
>
> > Paul
>
> > On Wed, Jul 6, 2016 at 11:23 AM, Remi Forax < forax at univ-mlv.fr > wrote:
>
> > > ACC_TRANSITIVE looks fine to me,
> >
>
> > > even if it really means transitive at one hop only.
> >
>
> > > R?mi
> >
>
> > > ----- Mail original -----
> >
>
> > > > De: "Mark Reinhold" < mark.reinhold at oracle.com >
> >
>
> > > > ?: jpms-spec-experts at openjdk.java.net
> >
>
> > > > Envoy?: Mercredi 6 Juillet 2016 17:32:36
> >
>
> > > > Objet: Proposal: #ClassFileAccPublic
> >
>
> > > >
> >
>
> > > > Issue summary
> >
>
> > > > -------------
> >
>
> > > >
> >
>
> > > > #ClassFileAccPublic --- The `ACC_PUBLIC` constant in a `requires_flags`
> >
>
> > > > should be encoded as `0x0001`, as it is elsewhere in the JVMS, rather
> >
>
> > > > than as `0x0020`, which has different meanings in other contexts.
> >
>
> > > >
> >
>
> > > > Proposal
> >
>
> > > > --------
> >
>
> > > >
> >
>
> > > > Unlike the other `ACC_PUBLIC` constants, this constant does not express
> >
>
> > > > an access mode but, rather, the addition of an implied transitive
> >
>
> > > > readability edge from modules that depend upon this module to some
> > > > other
> >
>
> > > > module required by this module. To avoid confusion at the class-file
> >
>
> > > > level, rename this constant to `ACC_TRANSITIVE`.
> >
>
> > > >
> >
>
> > > > (Whether the syntax of module declarations should use a keyword other
> >
>
> > > > than `public` for this purpose would be a separate issue.)
> >
>
> > > >
> >
>
> > > >
> >
>
> > > > [1]
> > > > http://openjdk.java.net/projects/jigsaw/spec/issues/#ClassFileAccPublic
> >
>
> > > >
> >
>
From mark.reinhold at oracle.com Wed Jul 13 14:28:13 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 13 Jul 2016 07:28:13 -0700 (PDT)
Subject: Discussion: #NonHierarchicalLayers
In-Reply-To: <56D7802A.9070305@redhat.com>
References: <56D7802A.9070305@redhat.com>
Message-ID: <20160713142813.C0075B78C2@eggemoggin.niobe.net>
Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#NonHierarchicalLayers
2016/3/2 16:07:06 -0800, david.lloyd at redhat.com:
> If there is any expectation of moving existing systems over to Jigsaw, I
> think that the strict hierarchical layer system should be abandoned in
> favor of a graph-oriented system. This allows containers to maintain
> separate module graphs with separate namespaces which nevertheless
> interconnect.
I suspect that generalizing the design to allow non-hierarchical layers
is possible, but tricky. Both configurations and layers would be able to
have multiple parents, so the API would become more complex, but perhaps
that's okay since this is more for expert than casual use. To avoid
having to deal with conflicts between parents (i.e., the diamond-import
problem), it'd likely be best to make the collections of parents be lists
rather than sets, so that they're searched in a determined order.
> I believe that this is a necessary requirement if (in particular) there
> is an expectation that Java EE 9 (or another Java EE specification)
> would be reframed in terms of Jigsaw-style modules, since Java EE module
> names have many fewer restrictions, and it is possible for (for example)
> more than one application to contain modules of the same name.
I don't understand how these two observations motivate non-hierarchical
layers, but maybe I'm missing something.
Module names are an entirely distinct issue [1]. The constraints on
module names could be relaxed regardless of whether or not layers are
hierarchical.
It's already possible to have multiple modules of the same name with
simple hierarchical layers. Two hosted applications can contain a module
of the same name as long as they're loaded into distinct, unrelated
layers. (Even two layers that are related can contain modules of the
same name, in which case the resolver chooses the module in the nearest
layer.)
Are there other reasons why a hypothetical future Java EE specification,
which allows applications to be built from collections of JPMS modules,
might necessitate non-hierarchical layers?
> If on the other hand there is an expectation that Java EE is not going
> to be retrofitted in a way which maps EE applications and modules to
> Jigsaw modules, then this should be clearly declared.
As I wrote earlier, we definitely do have that expectation -- that's why
the requirements include an entire section on dynamic configuration.
- Mark
[1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameCharacters
From mark.reinhold at oracle.com Wed Jul 13 14:29:13 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 13 Jul 2016 07:29:13 -0700 (PDT)
Subject: Discussion: #ResourceExistenceAndSize
In-Reply-To: <56D85B77.4040607@redhat.com>
References: <56D85B77.4040607@redhat.com>
Message-ID: <20160713142913.C1C1FB78C7@eggemoggin.niobe.net>
Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#ResourceExistenceAndSize
2016/3/3 7:42:47 -0800, david.lloyd at redhat.com:
> Module.getResourceAsStream() is the only way a resource can be loaded
> from a module. This makes it impossible to do either of the following:
>
> - Determine the size of a resource in advance of loading it
> - Determine the existence of a resource without opening it
>
> Loading resources as java.net.URLs allowed these things to be done
> (among other things), however it is also disadvantageous due to the
> heavyweight nature of that class.
What are some use cases that require determining the existence or size of
a resource without loading it?
>
> I had earlier come up with a simple patch [1],
(I'm not going to read your patch on GitHub since the comment on your
repo clearly states that it is not a contribution to this JSR nor to
OpenJDK. If you wish the EG to consider it then please either post
it here or contribute it explicitly to OpenJDK via some other means.)
> initially as a way to
> address JDK-6865375, which introduced the idea of a Resource class
> that is associated with a ClassLoader, with a name, URL, size, and
> stream factory method, which would also be a clean solution here (though
> with the addition of a Module property as well since in Jigsaw they're
> separate).
The issues laid out in JDK-6865375 [1] are entirely different from those
of resource existence and size, though I do see how a common solution
could address all these problems.
> The idea is to put resources on a similar footing to classes, whereby
> you can (if suitably permitted) acquire their class loaders just like
> Class.getClassLoader() allows you to do for classes. This also allows
> things like ServiceLoader to load classes directly from the module which
> contained the corresponding service descriptor.
I'm reluctant to introduce a brand-new general "Resource" abstraction,
along with all the complexity that would entail. There is, among other
things, a tricky compatibility issue here, as one of the comments to
JDK-6865375 points out: Any custom class loader that today overrides the
various getResource* methods would have to be retrofitted to override all
new "Resource"-returning methods.
At any rate, I don't see anything here that's inherently part of the
design of the module system, so I intend to close this issue as
unrelated.
- Mark
[1] https://bugs.openjdk.java.net/browse/JDK-6865375
From mark.reinhold at oracle.com Wed Jul 13 14:30:13 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 13 Jul 2016 07:30:13 -0700 (PDT)
Subject: Discussion: #MutableConfigurations
In-Reply-To: <56D79D56.7040801@redhat.com>
References: <56D79D56.7040801@redhat.com>
Message-ID: <20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#MutableConfigurations
2016/3/2 18:11:34 -0800, david.lloyd at redhat.com:
> It appears from what I can see in the Jigsaw code, that once a
> Configuration is calculated, it cannot be changed in any way,
That's true.
The overall model in the present design is that, given a set of modules,
you first compute a configuration, which captures the resolution of all
the modules' dependences in a consistent fashion. You then instantiate
that as a Layer (or even more than one Layer, if you want).
This model is motivated by one of our primary goals, namely reliable
configuration. Resolving a complete configuration, rather than doing
so incrementally, allows the early detection of missing, duplicate,
and conflicting dependencies.
> which
> makes them unsuitable for use in containers which add and remove modules
> at run time. It is not clear how such containers are expected to
> function.
They're expected to create Layers, which can be related hierarchically as
needed (or perhaps more generally, cf. #NonHierarchicalLayers [1]).
> If there is a deliberate intention that the Jigsaw system
> will not support dynamic modification of layers/configurations
> (including the dynamic addition and removal of modules at run time),
> then that should be explicitly stated.
I don't think it makes sense for Configurations to be mutable, for the
reasons stated above. It may be that the Layer concept needs to be made
more dynamic in certain ways, but that's a different matter.
I intend to close this issue unless there are strong objections from
other EG members.
- Mark
[1] http://openjdk.java.net/projects/jigsaw/spec/issues/#NonHierarchicalLayers
From mark.reinhold at oracle.com Wed Jul 13 14:31:13 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 13 Jul 2016 07:31:13 -0700 (PDT)
Subject: Discussion: #LazyConfigurationAndInstantiation
In-Reply-To: <56E33EEC.20706@redhat.com>
References: <56D7AE09.1030003@redhat.com>
<20160311174751.5399C9E891@eggemoggin.niobe.net>
<56E33EEC.20706@redhat.com>
Message-ID: <20160713143113.C5B45B78D2@eggemoggin.niobe.net>
Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#LazyConfigurationAndInstantiation
2016/3/11 13:55:56 -0800, david.lloyd at redhat.com:
> On 03/11/2016 11:47 AM, mark.reinhold at oracle.com wrote:
>> 2016/3/2 19:22 -0800, david.lloyd at redhat.com:
>>> It looks as though the instantiation of a Layer causes a complete load
>>> of all the modules in the layer.
>>
>> What do you mean by "complete load"? Loading all the classes?
>
> No I mean locating and loading all of the descriptors and building all
> the wiring information for every module.
Yes, that's what happens. Resolving a Configuration requires reading all
of the relevant module descriptors. Instantiating a Configuration into a
Layer builds a complete run-time module graph. Both of these operations
are linear in the number of modules in the graph.
>>> Apart from the consequent lack of
>>> dynamicism, I think this is very likely to cause problems for large
>>> applications, particularly if such an application might have many
>>> operation modes, some of which do not require *all* of the bundled
>>> modules to be loaded every single time.
>>>
>>> Can there not instead be an incremental resolution algorithm, akin to
>>> how classes are lazily loaded?
>>
>> Configuring a set of modules and instantiating that configuration as a
>> layer requires no more than reading the modules' descriptors. Nothing
>> else from any module definition will be read until it's actually needed.
>
> But it does require that all the module descriptors from a given layer
> be available, and that the load time for the first load of a module in a
> layer will always be bounded by the size of the layer, rather than just
> by the dependency subgraph of the module being loaded.
Correct.
> Based on
> application server deployments that I know about, I think the far upper
> bound for a realistic number of modules in a layer will probably lie in
> the thousands to ten thousands range (though there is always the outlier
> case where someone has to push it to see how far it goes...), which
> might translate into a substantial startup overhead.
In our experience with the prototype so far, the time required to resolve
a Configuration and instantiate it as a Layer is dominated by the time
required to locate and read module descriptors, typically from inside
individual artifacts in the filesystem.
One fairly straightforward way to speed that up is to link your modules
into a custom run-time image, so that the descriptors are all in one
optimized artifact [1]. If that's not feasible then you could achieve
much the same effect in the build, installation, or startup process of a
large application: Construct an optimized cache of the descriptors of all
your modules, and use a custom ModuleFinder to load the descriptors from
the cache but other module content from more-traditional artifacts.
> Another potential
> issue is requiring that all modules be present, though this is more
> closely related to #MutableConfigurations now I think; I suspect this
> issue could be mitigated with an acceptable solution to that issue.
As I wrote in my reply re. #MutableConfigurations [2], I think this
approach is at odds with our goal to provide reliable configuration.
>> If you really want to avoid configuring all modules in certain operation
>> modes, do layers not provide sufficient flexibility? Load the core of
>> your application into the boot layer, figure out which modules you need
>> for the requested operation mode, and then create an additional layer to
>> load those modules.
>
> That can work in some cases, but only if things are loaded one
> additional time (rather than on-demand as configuration changes, for
> example) or in such a way that you could just keep adding layers. But
> in the event of one large intermeshed layer, this won't work without
> #MutableConfigurations and/or #NonHierarchicalLayers and something to
> coordinate the load.
Suppose that we allow #NonHierarchicalLayers [3]. Are there realistic
use cases in which layers would not then provide sufficient flexibility?
- Mark
[1] http://openjdk.java.net/jeps/282
[2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-July/000359.html
[3] http://openjdk.java.net/projects/jigsaw/spec/issues/#NonHierarchicalLayers
From mark.reinhold at oracle.com Wed Jul 13 14:32:13 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 13 Jul 2016 07:32:13 -0700 (PDT)
Subject: Proposal: #DefaultModule
In-Reply-To: <577BC730.1040706@redhat.com>
References: <577BC730.1040706@redhat.com>
Message-ID: <20160713143213.CF094B78D7@eggemoggin.niobe.net>
2016/7/5 7:41:52 -0700, david.lloyd at redhat.com:
> I propose that the concept of "unnamed module" be dropped in favor of
> "default module". The main difference is that the class loader (or
> module finder or layer configuration or someone else) would be allowed
> to (but not required to) assign a free-form name and version string to
> this module. This would allow existing module systems to bring their
> module concept into some form of consonance with Jigsaw without
> compromising any of the restrictions that Jigsaw-style modules have.
>
> Effecting this change would suggest the addition of an "isDefault()"
> method on Module, possibly replacing "isNamed()" (which is arguably
> already somewhat redundant with respect to getName()). Also at some
> stage, something would have to establish the default module name and
> version strings, probably defaulting to the (current) null strings to
> keep a stable status quo.
The concept of unnamed modules has worked out pretty well so far. The
analogy with unnamed packages, while imprecise, has made it fairly easy
to explain to people. The fact that an unnamed module has no name is
not, as you correctly point out elsewhere, what keeps it from being a
candidate for resolution. If these modules could have names, however,
then people would naturally ask why they can't participate in resolution,
and we'd have yet more explaining to do.
I'm therefore reluctant to change unnamed modules into "default" modules,
so I suggest that we instead focus on your goal rather than a specific
potential solution.
Your goal seems to be to enable module systems other than JPMS to place
their own diagnostic information in stack traces and certain exception
messages. Such systems typically (always?) create a unique class loader
for each loaded module, and so in JPMS terms each such "external" module
would be represented by a class loader that loads all of its classes into
that loader's unnamed JPMS module.
If this is correct then there's a simpler way to reach your goal: Enhance
class loaders to have optional names. When the run-time system generates
a stack trace or an exception message that mentions a module name and
version, if present, then it could also insert the name of that module's
class loader, if present. This would allow an external module system to
provide better diagnostics without having to change the JPMS design in a
fundamental way.
Would that meet your needs?
- Mark
From david.lloyd at redhat.com Wed Jul 13 15:23:38 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 13 Jul 2016 10:23:38 -0500
Subject: Discussion: #NonHierarchicalLayers
In-Reply-To: <20160713142813.C0075B78C2@eggemoggin.niobe.net>
References: <56D7802A.9070305@redhat.com>
<20160713142813.C0075B78C2@eggemoggin.niobe.net>
Message-ID: <57865CFA.1010208@redhat.com>
On 07/13/2016 09:28 AM, mark.reinhold at oracle.com wrote:
> Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#NonHierarchicalLayers
>
> 2016/3/2 16:07:06 -0800, david.lloyd at redhat.com:
>> If there is any expectation of moving existing systems over to Jigsaw, I
>> think that the strict hierarchical layer system should be abandoned in
>> favor of a graph-oriented system. This allows containers to maintain
>> separate module graphs with separate namespaces which nevertheless
>> interconnect.
>
> I suspect that generalizing the design to allow non-hierarchical layers
> is possible, but tricky. Both configurations and layers would be able to
> have multiple parents, so the API would become more complex, but perhaps
> that's okay since this is more for expert than casual use. To avoid
> having to deal with conflicts between parents (i.e., the diamond-import
> problem), it'd likely be best to make the collections of parents be lists
> rather than sets, so that they're searched in a determined order.
That's good news.
>> I believe that this is a necessary requirement if (in particular) there
>> is an expectation that Java EE 9 (or another Java EE specification)
>> would be reframed in terms of Jigsaw-style modules, since Java EE module
>> names have many fewer restrictions, and it is possible for (for example)
>> more than one application to contain modules of the same name.
>
> I don't understand how these two observations motivate non-hierarchical
> layers, but maybe I'm missing something.
>
> Module names are an entirely distinct issue [1]. The constraints on
> module names could be relaxed regardless of whether or not layers are
> hierarchical.
Right, I shouldn't have conflated these issues. Names are relevant to
this issue only insofar as sibling layers allow for module name
duplication between them.
> It's already possible to have multiple modules of the same name with
> simple hierarchical layers. Two hosted applications can contain a module
> of the same name as long as they're loaded into distinct, unrelated
> layers. (Even two layers that are related can contain modules of the
> same name, in which case the resolver chooses the module in the nearest
> layer.)
>
> Are there other reasons why a hypothetical future Java EE specification,
> which allows applications to be built from collections of JPMS modules,
> might necessitate non-hierarchical layers?
It's not the specification that would require them, but the
implementation, which may allow OSGi-style dynamic addition and changing
of installed dependencies, especially from different (possibly
overlapping) module namespaces. Such dynamic changes can currently only
be modeled using incrementally-added layers, but this fails if a
previously loaded module must be updated to refer to a later one, or if
there is more than one module namespace.
Another case is where there are two separate deployment systems with
isolated name spaces (for example, OSGi and Java EE) which may be
cross-linked.
In our current solution, we don't actually layer the namespaces - in the
internal APIs we actually identify what module space the proposed
dependency exists in, along with the name of the module to find. This
allows us to (for example) have more than one declaration syntax for
dependencies.
In pre-Java 9 terms, a good metaphor is Extension-list: versus
Class-Path: MANIFEST headers - each comes from a separate namespace, but
each declares where the dependency information comes from. We also add
a Dependencies: header, which allows (the equivalent of boot layer)
module names to be specified as dependencies for a deployment,
separately from the old extension list and class path headers (which we
use to resolve modules from different namespaces). I believe that it is
presently our intention to continue to support extension lists even
after that is dropped from the spec; the namespace for extensions is
isolated from the namespace of static modules, but deployment modules
may depend on either or both (and/or additional module spaces like OSGi,
which may also have conflicting namespaces) at their option using the
appropriate MANIFEST header or other deployment descriptor.
So I would hypothetically imagine a layer for our static modules (though
this depends on the resolution or lack thereof of a couple other
issues), then another for extensions, another for EE deployments,
another sibling for filesystem-based "automatic" modules that we
generate for absolute class-path refs, and another sibling for each
auxiliary add-on system (like OSGi), all of which have the ability to
arbitrarily cross-reference to the other sibling layers. Some possible
configurations can be interpreted as a tree, but some configurations
must be a DAG or even a fully cyclic graph in order to work. Note also
that this might all only be meaningful hand-in-hand with a resolution to
#MutableConfigurations and #LazyConfigurationAndInstantiation.
Note that we could actually get by *without* non-hierarchical layers,
*if* I can explicitly establish module dependences which link between
sibling custom layers: I actually view this as a better, more general
approach, and one which we use internally. So that's another option.
>> If on the other hand there is an expectation that Java EE is not going
>> to be retrofitted in a way which maps EE applications and modules to
>> Jigsaw modules, then this should be clearly declared.
>
> As I wrote earlier, we definitely do have that expectation -- that's why
> the requirements include an entire section on dynamic configuration.
>
> - Mark
>
>
> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#ModuleNameCharacters
>
--
- DML
From david.lloyd at redhat.com Wed Jul 13 16:22:43 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 13 Jul 2016 11:22:43 -0500
Subject: Discussion: #ResourceExistenceAndSize
In-Reply-To: <20160713142913.C1C1FB78C7@eggemoggin.niobe.net>
References: <56D85B77.4040607@redhat.com>
<20160713142913.C1C1FB78C7@eggemoggin.niobe.net>
Message-ID: <57866AD3.4020807@redhat.com>
On 07/13/2016 09:29 AM, mark.reinhold at oracle.com wrote:
> Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#ResourceExistenceAndSize
>
> 2016/3/3 7:42:47 -0800, david.lloyd at redhat.com:
>> Module.getResourceAsStream() is the only way a resource can be loaded
>> from a module. This makes it impossible to do either of the following:
>>
>> - Determine the size of a resource in advance of loading it
>> - Determine the existence of a resource without opening it
>>
>> Loading resources as java.net.URLs allowed these things to be done
>> (among other things), however it is also disadvantageous due to the
>> heavyweight nature of that class.
>
> What are some use cases that require determining the existence or size of
> a resource without loading it?
You may need to pre-size data structures like arrays, or files on the
filesystem, or you may need the information to produce a
"Content-Length" style header before reading any data, or to know if you
are able to fulfill a request if the request has a minimum or maximum
size requirement.
Testing for the existence of a resource is useful for making decisions
about (for example) how to handle a deployment, as some specs require,
though opening, testing, and (if not null) closing the resource does
suffice to solve this problem, as long as doing so doesn't incur any
kind of undue overhead (and in the modular system, it *probably* won't).
>> I had earlier come up with a simple patch [1],
>
> (I'm not going to read your patch on GitHub since the comment on your
> repo clearly states that it is not a contribution to this JSR nor to
> OpenJDK. If you wish the EG to consider it then please either post
> it here or contribute it explicitly to OpenJDK via some other means.)
>> initially as a way to
>> address JDK-6865375, which introduced the idea of a Resource class
>> that is associated with a ClassLoader, with a name, URL, size, and
>> stream factory method, which would also be a clean solution here (though
>> with the addition of a Module property as well since in Jigsaw they're
>> separate).
>
> The issues laid out in JDK-6865375 [1] are entirely different from those
> of resource existence and size, though I do see how a common solution
> could address all these problems.
Yeah, I think that publishing the origin of the resource is an important
part of resource management as well, and I probably should have made it
all one; I suppose I left it off because the of the prior existence of
the JDK issue.
>> The idea is to put resources on a similar footing to classes, whereby
>> you can (if suitably permitted) acquire their class loaders just like
>> Class.getClassLoader() allows you to do for classes. This also allows
>> things like ServiceLoader to load classes directly from the module which
>> contained the corresponding service descriptor.
>
> I'm reluctant to introduce a brand-new general "Resource" abstraction,
> along with all the complexity that would entail. There is, among other
> things, a tricky compatibility issue here, as one of the comments to
> JDK-6865375 points out: Any custom class loader that today overrides the
> various getResource* methods would have to be retrofitted to override all
> new "Resource"-returning methods.
Yeah the workaround for this in my patch is to provide default
implementations which use URL to back the Resource object, which allows
the current status quo to be maintained (though the usual issues with
adding methods to a base class do apply).
> At any rate, I don't see anything here that's inherently part of the
> design of the module system, so I intend to close this issue as
> unrelated.
The relation comes in because Class and ClassLoader have a way to open a
resource with a URL, but Module does not, providing only a
getResourceAsStream() method. Adding a getResource() method would be
one solution, but the underlying requirement is really that the resource
existence and size (and origin) could/should be queriable as well, so
that was the issue I registered, especially as URL is really not a great
solution to this problem.
> [1] https://bugs.openjdk.java.net/browse/JDK-6865375
--
- DML
From david.lloyd at redhat.com Wed Jul 13 16:25:54 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 13 Jul 2016 11:25:54 -0500
Subject: Proposal: #DefaultModule
In-Reply-To: <20160713143213.CF094B78D7@eggemoggin.niobe.net>
References: <577BC730.1040706@redhat.com>
<20160713143213.CF094B78D7@eggemoggin.niobe.net>
Message-ID: <57866B92.200@redhat.com>
On 07/13/2016 09:32 AM, mark.reinhold at oracle.com wrote:
> 2016/7/5 7:41:52 -0700, david.lloyd at redhat.com:
>> I propose that the concept of "unnamed module" be dropped in favor of
>> "default module". The main difference is that the class loader (or
>> module finder or layer configuration or someone else) would be allowed
>> to (but not required to) assign a free-form name and version string to
>> this module. This would allow existing module systems to bring their
>> module concept into some form of consonance with Jigsaw without
>> compromising any of the restrictions that Jigsaw-style modules have.
>>
>> Effecting this change would suggest the addition of an "isDefault()"
>> method on Module, possibly replacing "isNamed()" (which is arguably
>> already somewhat redundant with respect to getName()). Also at some
>> stage, something would have to establish the default module name and
>> version strings, probably defaulting to the (current) null strings to
>> keep a stable status quo.
>
> The concept of unnamed modules has worked out pretty well so far. The
> analogy with unnamed packages, while imprecise, has made it fairly easy
> to explain to people. The fact that an unnamed module has no name is
> not, as you correctly point out elsewhere, what keeps it from being a
> candidate for resolution. If these modules could have names, however,
> then people would naturally ask why they can't participate in resolution,
> and we'd have yet more explaining to do.
>
> I'm therefore reluctant to change unnamed modules into "default" modules,
> so I suggest that we instead focus on your goal rather than a specific
> potential solution.
>
> Your goal seems to be to enable module systems other than JPMS to place
> their own diagnostic information in stack traces and certain exception
> messages. Such systems typically (always?) create a unique class loader
> for each loaded module, and so in JPMS terms each such "external" module
> would be represented by a class loader that loads all of its classes into
> that loader's unnamed JPMS module.
>
> If this is correct then there's a simpler way to reach your goal: Enhance
> class loaders to have optional names. When the run-time system generates
> a stack trace or an exception message that mentions a module name and
> version, if present, then it could also insert the name of that module's
> class loader, if present. This would allow an external module system to
> provide better diagnostics without having to change the JPMS design in a
> fundamental way.
>
> Would that meet your needs?
I think that would probably work very well, and fits in nicely with the
existing architecture of stack trace assembly as I understand it (since
the class loader can be directly referenced from the Class, which is
already present). Thanks!
--
- DML
From david.lloyd at redhat.com Wed Jul 13 16:42:56 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 13 Jul 2016 11:42:56 -0500
Subject: Discussion: #MutableConfigurations
In-Reply-To: <20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
Message-ID: <57866F90.5020703@redhat.com>
On 07/13/2016 09:30 AM, mark.reinhold at oracle.com wrote:
> Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#MutableConfigurations
>
> 2016/3/2 18:11:34 -0800, david.lloyd at redhat.com:
>> It appears from what I can see in the Jigsaw code, that once a
>> Configuration is calculated, it cannot be changed in any way,
>
> That's true.
>
> The overall model in the present design is that, given a set of modules,
> you first compute a configuration, which captures the resolution of all
> the modules' dependences in a consistent fashion. You then instantiate
> that as a Layer (or even more than one Layer, if you want).
>
> This model is motivated by one of our primary goals, namely reliable
> configuration. Resolving a complete configuration, rather than doing
> so incrementally, allows the early detection of missing, duplicate,
> and conflicting dependencies.
>
>> which
>> makes them unsuitable for use in containers which add and remove modules
>> at run time. It is not clear how such containers are expected to
>> function.
>
> They're expected to create Layers, which can be related hierarchically as
> needed (or perhaps more generally, cf. #NonHierarchicalLayers [1]).
>
>> If there is a deliberate intention that the Jigsaw system
>> will not support dynamic modification of layers/configurations
>> (including the dynamic addition and removal of modules at run time),
>> then that should be explicitly stated.
>
> I don't think it makes sense for Configurations to be mutable, for the
> reasons stated above. It may be that the Layer concept needs to be made
> more dynamic in certain ways, but that's a different matter.
>
> I intend to close this issue unless there are strong objections from
> other EG members.
In that case, other EG members take note: this means that hot deployment
as the world knows it is over as of Java EE 9, since a module can never
delete old or establish new dependence relationships. This also means
that OSGi loses this capability as well, so either OSGi has to be
redefined with the new restriction, or else OSGi bundles cannot be modules.
--
- DML
From pbenedict at apache.org Wed Jul 13 17:03:13 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Wed, 13 Jul 2016 12:03:13 -0500
Subject: Proposal: #DefaultModule
In-Reply-To: <57866B92.200@redhat.com>
References: <577BC730.1040706@redhat.com>
<20160713143213.CF094B78D7@eggemoggin.niobe.net>
<57866B92.200@redhat.com>
Message-ID: <CABLGb9xqCdouRjXxt10VkeBRiK2pXzVwu=kyAD3Ooo8gg=rbZw@mail.gmail.com>
Mark, I think adding a name is very elegant. Glad to see this. +1 :-)
PS: I think it's easier to explain to people "default" vs "unnamed" because
people are already familiar with default packages. Every Java programmer
knows default packages have no names. It's a no-brainier to teach that a
"default" module has no name too. Keeping the same terms should be desired,
I think. I don't know if David still is desiring the rename, but I am still
for it. Given that the new module description (in this email chain) takes
away the confusing issue of a named unnamed module, I can't see another
impediment in calling it the default module now. I would like the EG to
still consider it, but only after hearing other feedback first (especially
David, the author of the proposal).
Cheers,
Paul
On Wed, Jul 13, 2016 at 11:25 AM, David M. Lloyd <david.lloyd at redhat.com>
wrote:
> On 07/13/2016 09:32 AM, mark.reinhold at oracle.com wrote:
>
>> 2016/7/5 7:41:52 -0700, david.lloyd at redhat.com:
>>
>>> I propose that the concept of "unnamed module" be dropped in favor of
>>> "default module". The main difference is that the class loader (or
>>> module finder or layer configuration or someone else) would be allowed
>>> to (but not required to) assign a free-form name and version string to
>>> this module. This would allow existing module systems to bring their
>>> module concept into some form of consonance with Jigsaw without
>>> compromising any of the restrictions that Jigsaw-style modules have.
>>>
>>> Effecting this change would suggest the addition of an "isDefault()"
>>> method on Module, possibly replacing "isNamed()" (which is arguably
>>> already somewhat redundant with respect to getName()). Also at some
>>> stage, something would have to establish the default module name and
>>> version strings, probably defaulting to the (current) null strings to
>>> keep a stable status quo.
>>>
>>
>> The concept of unnamed modules has worked out pretty well so far. The
>> analogy with unnamed packages, while imprecise, has made it fairly easy
>> to explain to people. The fact that an unnamed module has no name is
>> not, as you correctly point out elsewhere, what keeps it from being a
>> candidate for resolution. If these modules could have names, however,
>> then people would naturally ask why they can't participate in resolution,
>> and we'd have yet more explaining to do.
>>
>> I'm therefore reluctant to change unnamed modules into "default" modules,
>> so I suggest that we instead focus on your goal rather than a specific
>> potential solution.
>>
>> Your goal seems to be to enable module systems other than JPMS to place
>> their own diagnostic information in stack traces and certain exception
>> messages. Such systems typically (always?) create a unique class loader
>> for each loaded module, and so in JPMS terms each such "external" module
>> would be represented by a class loader that loads all of its classes into
>> that loader's unnamed JPMS module.
>>
>> If this is correct then there's a simpler way to reach your goal: Enhance
>> class loaders to have optional names. When the run-time system generates
>> a stack trace or an exception message that mentions a module name and
>> version, if present, then it could also insert the name of that module's
>> class loader, if present. This would allow an external module system to
>> provide better diagnostics without having to change the JPMS design in a
>> fundamental way.
>>
>> Would that meet your needs?
>>
>
> I think that would probably work very well, and fits in nicely with the
> existing architecture of stack trace assembly as I understand it (since the
> class loader can be directly referenced from the Class, which is already
> present). Thanks!
>
> --
> - DML
>
From pbenedict at apache.org Wed Jul 13 17:07:09 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Wed, 13 Jul 2016 12:07:09 -0500
Subject: Proposal: #DefaultModule
In-Reply-To: <CABLGb9xqCdouRjXxt10VkeBRiK2pXzVwu=kyAD3Ooo8gg=rbZw@mail.gmail.com>
References: <577BC730.1040706@redhat.com>
<20160713143213.CF094B78D7@eggemoggin.niobe.net>
<57866B92.200@redhat.com>
<CABLGb9xqCdouRjXxt10VkeBRiK2pXzVwu=kyAD3Ooo8gg=rbZw@mail.gmail.com>
Message-ID: <CABLGb9xmKwfAB++zv5usAbpCUWZAj+edqKM-tgJgB=H4JNx9Xw@mail.gmail.com>
Err.. I meant "new class loader name". Ignore "new module description" --
didn't mean that. That's a typo.
Cheers,
Paul
On Wed, Jul 13, 2016 at 12:03 PM, Paul Benedict <pbenedict at apache.org>
wrote:
> Mark, I think adding a name is very elegant. Glad to see this. +1 :-)
>
> PS: I think it's easier to explain to people "default" vs "unnamed"
> because people are already familiar with default packages. Every Java
> programmer knows default packages have no names. It's a no-brainier to
> teach that a "default" module has no name too. Keeping the same terms
> should be desired, I think. I don't know if David still is desiring the
> rename, but I am still for it. Given that the new module description (in
> this email chain) takes away the confusing issue of a named unnamed module,
> I can't see another impediment in calling it the default module now. I
> would like the EG to still consider it, but only after hearing other
> feedback first (especially David, the author of the proposal).
>
> Cheers,
> Paul
>
> On Wed, Jul 13, 2016 at 11:25 AM, David M. Lloyd <david.lloyd at redhat.com>
> wrote:
>
>> On 07/13/2016 09:32 AM, mark.reinhold at oracle.com wrote:
>>
>>> 2016/7/5 7:41:52 -0700, david.lloyd at redhat.com:
>>>
>>>> I propose that the concept of "unnamed module" be dropped in favor of
>>>> "default module". The main difference is that the class loader (or
>>>> module finder or layer configuration or someone else) would be allowed
>>>> to (but not required to) assign a free-form name and version string to
>>>> this module. This would allow existing module systems to bring their
>>>> module concept into some form of consonance with Jigsaw without
>>>> compromising any of the restrictions that Jigsaw-style modules have.
>>>>
>>>> Effecting this change would suggest the addition of an "isDefault()"
>>>> method on Module, possibly replacing "isNamed()" (which is arguably
>>>> already somewhat redundant with respect to getName()). Also at some
>>>> stage, something would have to establish the default module name and
>>>> version strings, probably defaulting to the (current) null strings to
>>>> keep a stable status quo.
>>>>
>>>
>>> The concept of unnamed modules has worked out pretty well so far. The
>>> analogy with unnamed packages, while imprecise, has made it fairly easy
>>> to explain to people. The fact that an unnamed module has no name is
>>> not, as you correctly point out elsewhere, what keeps it from being a
>>> candidate for resolution. If these modules could have names, however,
>>> then people would naturally ask why they can't participate in resolution,
>>> and we'd have yet more explaining to do.
>>>
>>> I'm therefore reluctant to change unnamed modules into "default" modules,
>>> so I suggest that we instead focus on your goal rather than a specific
>>> potential solution.
>>>
>>> Your goal seems to be to enable module systems other than JPMS to place
>>> their own diagnostic information in stack traces and certain exception
>>> messages. Such systems typically (always?) create a unique class loader
>>> for each loaded module, and so in JPMS terms each such "external" module
>>> would be represented by a class loader that loads all of its classes into
>>> that loader's unnamed JPMS module.
>>>
>>> If this is correct then there's a simpler way to reach your goal: Enhance
>>> class loaders to have optional names. When the run-time system generates
>>> a stack trace or an exception message that mentions a module name and
>>> version, if present, then it could also insert the name of that module's
>>> class loader, if present. This would allow an external module system to
>>> provide better diagnostics without having to change the JPMS design in a
>>> fundamental way.
>>>
>>> Would that meet your needs?
>>>
>>
>> I think that would probably work very well, and fits in nicely with the
>> existing architecture of stack trace assembly as I understand it (since the
>> class loader can be directly referenced from the Class, which is already
>> present). Thanks!
>>
>> --
>> - DML
>>
>
>
From david.lloyd at redhat.com Wed Jul 13 17:39:42 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 13 Jul 2016 12:39:42 -0500
Subject: Discussion: #LazyConfigurationAndInstantiation
In-Reply-To: <20160713143113.C5B45B78D2@eggemoggin.niobe.net>
References: <56D7AE09.1030003@redhat.com>
<20160311174751.5399C9E891@eggemoggin.niobe.net>
<56E33EEC.20706@redhat.com>
<20160713143113.C5B45B78D2@eggemoggin.niobe.net>
Message-ID: <57867CDE.8000206@redhat.com>
On 07/13/2016 09:31 AM, mark.reinhold at oracle.com wrote:
> Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#LazyConfigurationAndInstantiation
>
> 2016/3/11 13:55:56 -0800, david.lloyd at redhat.com:
>> On 03/11/2016 11:47 AM, mark.reinhold at oracle.com wrote:
>>> 2016/3/2 19:22 -0800, david.lloyd at redhat.com:
>>>> It looks as though the instantiation of a Layer causes a complete load
>>>> of all the modules in the layer.
>>>
>>> What do you mean by "complete load"? Loading all the classes?
>>
>> No I mean locating and loading all of the descriptors and building all
>> the wiring information for every module.
>
> Yes, that's what happens. Resolving a Configuration requires reading all
> of the relevant module descriptors. Instantiating a Configuration into a
> Layer builds a complete run-time module graph. Both of these operations
> are linear in the number of modules in the graph.
>
>>>> Apart from the consequent lack of
>>>> dynamicism, I think this is very likely to cause problems for large
>>>> applications, particularly if such an application might have many
>>>> operation modes, some of which do not require *all* of the bundled
>>>> modules to be loaded every single time.
>>>>
>>>> Can there not instead be an incremental resolution algorithm, akin to
>>>> how classes are lazily loaded?
>>>
>>> Configuring a set of modules and instantiating that configuration as a
>>> layer requires no more than reading the modules' descriptors. Nothing
>>> else from any module definition will be read until it's actually needed.
>>
>> But it does require that all the module descriptors from a given layer
>> be available, and that the load time for the first load of a module in a
>> layer will always be bounded by the size of the layer, rather than just
>> by the dependency subgraph of the module being loaded.
>
> Correct.
>
>> Based on
>> application server deployments that I know about, I think the far upper
>> bound for a realistic number of modules in a layer will probably lie in
>> the thousands to ten thousands range (though there is always the outlier
>> case where someone has to push it to see how far it goes...), which
>> might translate into a substantial startup overhead.
>
> In our experience with the prototype so far, the time required to resolve
> a Configuration and instantiate it as a Layer is dominated by the time
> required to locate and read module descriptors, typically from inside
> individual artifacts in the filesystem.
>
> One fairly straightforward way to speed that up is to link your modules
> into a custom run-time image, so that the descriptors are all in one
> optimized artifact [1]. If that's not feasible then you could achieve
> much the same effect in the build, installation, or startup process of a
> large application: Construct an optimized cache of the descriptors of all
> your modules, and use a custom ModuleFinder to load the descriptors from
> the cache but other module content from more-traditional artifacts.
>> Another potenti
>> issue is requiring that all modules be present, though this is more
>> closely related to #MutableConfigurations now I think; I suspect this
>> issue could be mitigated with an acceptable solution to that issue.
>
> As I wrote in my reply re. #MutableConfigurations [2], I think this
> approach is at odds with our goal to provide reliable configuration.
I disagree. "Reliable" is only defined in the agreed-upon requirements
by stipulating that modules may have dependence relationships which are
declared. I interpreted that goal simply as an explicit rejection of
various poorly-defined customized class loader behaviors, and a move
towards clearer and more predictable behavior. By my interpretation and
experience, I consider, for example, the ability to change the explicit
dependence relationships to still be "reliable" as long as the effects
of doing so are well-defined, just as I consider the behavior of lazy
class loading to be "reliable" in that it is well-defined and
predictable, and has rules which make sense (referring to the way that
classes are loaded, resolved, and initialized in separate phases, which
allows for lazy on-demand-only progression through those phases but also
allows for almost completely arbitrary interconnection of classes, while
remaining basically predictable).
The first time that "reliable" is specified in terms of concrete
behavior is in the SOTMS document, and these issues are being raised
exactly against the state of the module system, so I think that this
issue as well as #MutableConfigurations serve to directly challenge the
validity of the definition of "reliable" which is used by the proposed
implementation, rather than being invalid due to the presumed validity
of that definition (which was not agreed upon by the EG).
Even the JSR definition (which I acknowledge is quite out of date at
this point) states that OSGi does address the problem of "reliable"
configuration, despite the fact that the OSGi solution allows for
dynamic loading and relinking. This is contrary to the definition in
the SOTMS, and also despite the fact that Jigsaw's more strict
interpretation of this actually invalidates those behaviors of OSGi,
were OSGi to try to merge the bundle concept with the Jigsaw module
concept somehow.
>>> If you really want to avoid configuring all modules in certain operation
>>> modes, do layers not provide sufficient flexibility? Load the core of
>>> your application into the boot layer, figure out which modules you need
>>> for the requested operation mode, and then create an additional layer to
>>> load those modules.
>>
>> That can work in some cases, but only if things are loaded one
>> additional time (rather than on-demand as configuration changes, for
>> example) or in such a way that you could just keep adding layers. But
>> in the event of one large intermeshed layer, this won't work without
>> #MutableConfigurations and/or #NonHierarchicalLayers and something to
>> coordinate the load.
>
> Suppose that we allow #NonHierarchicalLayers [3]. Are there realistic
> use cases in which layers would not then provide sufficient flexibility?
I'm not sure that it will (in all cases). In our current system, a
module isn't loaded until it is referenced, and it is not linked until
it is used, like the aforementioned class linking rules. Even if we
have separate (non-hierarchical) layers for every single module, because
of the way modules are defined in Jigsaw, I think, given module graph G,
we'd still have to aggressively traverse all of G in order to load any
module that is a part of G, even if nodes in that graph are never
actually referenced at run time, which puts us back at square one.
--
- DML
From mark.reinhold at oracle.com Wed Jul 13 18:06:13 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 13 Jul 2016 11:06:13 -0700 (PDT)
Subject: Issue #ServiceLoaderEnhancement
In-Reply-To: <CACzrW9Cp0tpZnU-wF3a+GYg2RBU=LeHixqAiZSFZyJjA10kawg@mail.gmail.com>
References: <CACzrW9Cp0tpZnU-wF3a+GYg2RBU=LeHixqAiZSFZyJjA10kawg@mail.gmail.com>
Message-ID: <20160713180613.27999B7953@eggemoggin.niobe.net>
2016/7/1 2:27:39 -0700, Stephen Colebourne <scolebourne at joda.org>:
> The ServiceLoader API is to be pushed as a key class by the module
> system. This issue is proposing enhancements to the API. Others may
> want to add their own proposals.
>
> Currently, ServiceLoader requires service implementations to have a
> public no-args constructor on a public class. This is very
> restrictive, especially in a Java 8 world of immutable classes and
> factory methods (with constructors actively avoided).
>
> JSR-310 has a service for loading Chronology implementations. In
> ThreeTen-Extra, I have a number of additional implementations [1].
> Unfortunately, all the implementations are forced to have a public
> no-args constructor, as ServiceLoader cannot make use of the singleton
> INSTANCE constant. As can be seen, I have to mark the public no-args
> constructor as deprecated [2]. This state of affairs is frankly
> ridiculous, and is one of the reasons putting me off using
> ServiceLoader non-JDK code.
>
> I propose that the spec of ServiceLoader is extended to make it more useful.
>
> Firstly, I propose that all reflection by ServiceLoader should use
> setAccessible(true) to allow providers to be non-public.
We could do this for service providers defined in modules, but not for
those specified via the META-INF/services convention. The problem with
the latter is that if ServiceLoader could instantiate an arbitrary
non-public class then an adversary could spoof it into doing that simply
by placing an appropriate META-INF/services configuration file on the
class path. That's not possible for providers in modules, since the
declaration of the provider and the provider itself are always tightly
associated with each other, in the same artifact, so we can relax the
restriction in that case.
> Secondly, I propose that if no public no-args constructor is found on
> the provider class, ServiceLoader should locate a public static
> constant named "INSTANCE" and use that instead.
That's reasonable, though we don't have much precedence for name-based
conventions in the SE platform itself.
> This proposal could be extended further. Instead of searching for
> "INSTANCE", it could look for all public static constants of the
> provider type, allowing one provider class to result in many provider
> instances.
I suspect that might be one heuristic too much. I could see searching
for INSTANCE, and maybe also INSTANCES where the latter refers to an
array or a collection of appropriate type, but just taking anything based
solely upon type matches could lead to difficult-to-diagnose failures.
An alternative is to take an explicitly-declarative approach, defining
something like a @Provides annotation to identify the constructors,
factories, or fields that construct or contain actual provider objects.
I'll give this some thought.
> Thirdly, it should be possible to access the set of provider classes
> without instantiating them. This would allow applications to use the
> module-info design for services without using ServiceLoader, eg to
> reflect on the provider classes, perhaps to search for annotations.
As an historical note, the original idea way back in Java 1.3 was for
complex providers to be represented by small, lightweight proxy objects
rather than full providers, or types. To quote the specification (of
the sun.misc.Service class, before it became java.util.ServiceLoader):
* ... This <i>provider class</i> will typically not be the entire
* provider itself but rather a proxy that contains enough information to
* decide whether the provider is able to satisfy a particular request together
* with code that can create the actual provider on demand. The details of
* provider classes tend to be highly service-specific; no single class or
* interface could possibly unify them, so no such class has been defined.
Most of the services defined and used in the JDK itself follow this
advice, but I accept that many others out in the world do not, so at
this point people reasonably want to be able to filter providers based
on types and annotations thereon. This should be straightforward; I'll
work up a proposal.
> I hope this proposal is given serious consideration, as at present the
> design of ServiceLoader is simply too restrictive to be appropriate to
> be hard coded into the module specification.
These are all good suggestions; thanks for raising them here. I've added
them to the issue list:
http://openjdk.java.net/projects/jigsaw/spec/issues/#ServiceLoaderEnhancements
- Mark
From pbenedict at apache.org Wed Jul 13 19:03:10 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Wed, 13 Jul 2016 14:03:10 -0500
Subject: Proposal: #DefaultModule
In-Reply-To: <CABLGb9xmKwfAB++zv5usAbpCUWZAj+edqKM-tgJgB=H4JNx9Xw@mail.gmail.com>
References: <577BC730.1040706@redhat.com>
<20160713143213.CF094B78D7@eggemoggin.niobe.net>
<57866B92.200@redhat.com>
<CABLGb9xqCdouRjXxt10VkeBRiK2pXzVwu=kyAD3Ooo8gg=rbZw@mail.gmail.com>
<CABLGb9xmKwfAB++zv5usAbpCUWZAj+edqKM-tgJgB=H4JNx9Xw@mail.gmail.com>
Message-ID: <CABLGb9xtM5NE9uw-SotFZPDKz7vca2S00nmsQ4mJTjJcc9UkTg@mail.gmail.com>
All, please nix my suggestion. Alex kindly pointed out to me the JLS has
always called it "unnamed package", but I have always used the term
"default package" -- a likely colloquialism. I didn't invent the term but
many people call it that in conversation, message boards, etc. I don't know
the genesis of "default package" but I am glad he pointed out the correct
term to me. With my interest piqued, I loaded up Eclipse Neon for fun and
went to the Wizard to create a class. When I blanked out the package name,
the warning is, "The use of the default package is discouraged." I probably
have been using Eclipse for over 10 years and got accustomed to that
phrasing.
Cheers,
Paul
On Wed, Jul 13, 2016 at 12:07 PM, Paul Benedict <pbenedict at apache.org>
wrote:
> Err.. I meant "new class loader name". Ignore "new module description" --
> didn't mean that. That's a typo.
>
> Cheers,
> Paul
>
> On Wed, Jul 13, 2016 at 12:03 PM, Paul Benedict <pbenedict at apache.org>
> wrote:
>
>> Mark, I think adding a name is very elegant. Glad to see this. +1 :-)
>>
>> PS: I think it's easier to explain to people "default" vs "unnamed"
>> because people are already familiar with default packages. Every Java
>> programmer knows default packages have no names. It's a no-brainier to
>> teach that a "default" module has no name too. Keeping the same terms
>> should be desired, I think. I don't know if David still is desiring the
>> rename, but I am still for it. Given that the new module description (in
>> this email chain) takes away the confusing issue of a named unnamed module,
>> I can't see another impediment in calling it the default module now. I
>> would like the EG to still consider it, but only after hearing other
>> feedback first (especially David, the author of the proposal).
>>
>> Cheers,
>> Paul
>>
>> On Wed, Jul 13, 2016 at 11:25 AM, David M. Lloyd <david.lloyd at redhat.com>
>> wrote:
>>
>>> On 07/13/2016 09:32 AM, mark.reinhold at oracle.com wrote:
>>>
>>>> 2016/7/5 7:41:52 -0700, david.lloyd at redhat.com:
>>>>
>>>>> I propose that the concept of "unnamed module" be dropped in favor of
>>>>> "default module". The main difference is that the class loader (or
>>>>> module finder or layer configuration or someone else) would be allowed
>>>>> to (but not required to) assign a free-form name and version string to
>>>>> this module. This would allow existing module systems to bring their
>>>>> module concept into some form of consonance with Jigsaw without
>>>>> compromising any of the restrictions that Jigsaw-style modules have.
>>>>>
>>>>> Effecting this change would suggest the addition of an "isDefault()"
>>>>> method on Module, possibly replacing "isNamed()" (which is arguably
>>>>> already somewhat redundant with respect to getName()). Also at some
>>>>> stage, something would have to establish the default module name and
>>>>> version strings, probably defaulting to the (current) null strings to
>>>>> keep a stable status quo.
>>>>>
>>>>
>>>> The concept of unnamed modules has worked out pretty well so far. The
>>>> analogy with unnamed packages, while imprecise, has made it fairly easy
>>>> to explain to people. The fact that an unnamed module has no name is
>>>> not, as you correctly point out elsewhere, what keeps it from being a
>>>> candidate for resolution. If these modules could have names, however,
>>>> then people would naturally ask why they can't participate in
>>>> resolution,
>>>> and we'd have yet more explaining to do.
>>>>
>>>> I'm therefore reluctant to change unnamed modules into "default"
>>>> modules,
>>>> so I suggest that we instead focus on your goal rather than a specific
>>>> potential solution.
>>>>
>>>> Your goal seems to be to enable module systems other than JPMS to place
>>>> their own diagnostic information in stack traces and certain exception
>>>> messages. Such systems typically (always?) create a unique class loader
>>>> for each loaded module, and so in JPMS terms each such "external" module
>>>> would be represented by a class loader that loads all of its classes
>>>> into
>>>> that loader's unnamed JPMS module.
>>>>
>>>> If this is correct then there's a simpler way to reach your goal:
>>>> Enhance
>>>> class loaders to have optional names. When the run-time system
>>>> generates
>>>> a stack trace or an exception message that mentions a module name and
>>>> version, if present, then it could also insert the name of that module's
>>>> class loader, if present. This would allow an external module system to
>>>> provide better diagnostics without having to change the JPMS design in a
>>>> fundamental way.
>>>>
>>>> Would that meet your needs?
>>>>
>>>
>>> I think that would probably work very well, and fits in nicely with the
>>> existing architecture of stack trace assembly as I understand it (since the
>>> class loader can be directly referenced from the Class, which is already
>>> present). Thanks!
>>>
>>> --
>>> - DML
>>>
>>
>>
>
From forax at univ-mlv.fr Wed Jul 13 20:09:54 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Wed, 13 Jul 2016 22:09:54 +0200
Subject: Discussion: #MutableConfigurations
In-Reply-To: <57866F90.5020703@redhat.com>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
<57866F90.5020703@redhat.com>
Message-ID: <C9342E73-D613-424A-8F92-510FCBF993E0@univ-mlv.fr>
I may say something stupid but if you have one Layer by ClassLoader, why it's not enough ?
Remi
On July 13, 2016 6:42:56 PM GMT+02:00, "David M. Lloyd" <david.lloyd at redhat.com> wrote:
>On 07/13/2016 09:30 AM, mark.reinhold at oracle.com wrote:
>> Reference:
>http://openjdk.java.net/projects/jigsaw/spec/issues/#MutableConfigurations
>>
>> 2016/3/2 18:11:34 -0800, david.lloyd at redhat.com:
>>> It appears from what I can see in the Jigsaw code, that once a
>>> Configuration is calculated, it cannot be changed in any way,
>>
>> That's true.
>>
>> The overall model in the present design is that, given a set of
>modules,
>> you first compute a configuration, which captures the resolution of
>all
>> the modules' dependences in a consistent fashion. You then
>instantiate
>> that as a Layer (or even more than one Layer, if you want).
>>
>> This model is motivated by one of our primary goals, namely reliable
>> configuration. Resolving a complete configuration, rather than doing
>> so incrementally, allows the early detection of missing, duplicate,
>> and conflicting dependencies.
>>
>>> which
>>> makes them unsuitable for use in containers which add and remove
>modules
>>> at run time. It is not clear how such containers are expected to
>>> function.
>>
>> They're expected to create Layers, which can be related
>hierarchically as
>> needed (or perhaps more generally, cf. #NonHierarchicalLayers [1]).
>>
>>> If there is a deliberate intention that the Jigsaw
>system
>>> will not support dynamic modification of layers/configurations
>>> (including the dynamic addition and removal of modules at run time),
>>> then that should be explicitly stated.
>>
>> I don't think it makes sense for Configurations to be mutable, for
>the
>> reasons stated above. It may be that the Layer concept needs to be
>made
>> more dynamic in certain ways, but that's a different matter.
>>
>> I intend to close this issue unless there are strong objections from
>> other EG members.
>
>In that case, other EG members take note: this means that hot
>deployment
>as the world knows it is over as of Java EE 9, since a module can never
>
>delete old or establish new dependence relationships. This also means
>that OSGi loses this capability as well, so either OSGi has to be
>redefined with the new restriction, or else OSGi bundles cannot be
>modules.
>
>--
>- DML
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
From david.lloyd at redhat.com Wed Jul 13 20:27:21 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 13 Jul 2016 15:27:21 -0500
Subject: Discussion: #MutableConfigurations
In-Reply-To: <C9342E73-D613-424A-8F92-510FCBF993E0@univ-mlv.fr>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
<57866F90.5020703@redhat.com>
<C9342E73-D613-424A-8F92-510FCBF993E0@univ-mlv.fr>
Message-ID: <5786A429.1040601@redhat.com>
On 07/13/2016 03:09 PM, Remi Forax wrote:
> I may say something stupid but if you have one Layer by ClassLoader, why
> it's not enough ?
Even if you have one layer per classloader per module, you can add
modules but you cannot update them when they are removed.
>
> Remi
>
>
> On July 13, 2016 6:42:56 PM GMT+02:00, "David M. Lloyd"
> <david.lloyd at redhat.com> wrote:
>
> On 07/13/2016 09:30 AM, mark.reinhold at oracle.com wrote:
>
> Reference:
> http://openjdk.java.net/projects/jigsaw/spec/issues/#MutableConfigurations
>
> 2016/3/2 18:11:34 -0800, david.lloyd at redhat.com:
>
> It appears from what I can see in the Jigsaw code, that once a
> Configuration is calculated, it cannot be changed in any way,
>
>
> That's true.
>
> The overall model in the present design is that, given a set of
> modules,
> you first compute a configuration, which captures the resolution
> of all
> the modules' dependences in a consistent fashion. You then
> instantiate
> that as a Lay! er (or even more than one Layer, if you want).
>
> This model is motivated by one of our primary goals, namely reliable
> configuration. Resolving a complete configuration, rather than doing
> so incrementally, allows the early detection of missing, duplicate,
> and conflicting dependencies.
>
> which
> makes them unsuitable for use in containers which add and
> remove modules
> at run time. It is not clear how such containers are expected to
> function.
>
>
> They're expected to create Layers, which can be related
> hierarchically as
> needed (or perhaps more generally, cf. #NonHierarchicalLayers [1]).
>
> I! f there is a deliberate intention that the Jigsaw system
> will not support dynamic modification of layers/configurations
> (including the dynamic addition and removal of modules at
> run time),
> then that should be explicitly stated.
>
>
> I don't think it makes sense for Configurations to be mutable,
> for the
> reasons stated above. It may be that the Layer concept needs to
> be made
> more dynamic in certain ways, but that's a different matter.
>
> I intend to close this issue unless there are strong objections from
> other EG members.
>
>
> In that case, other EG members take note: this means that hot deployment
> as the world knows it is over as of Java EE 9, since a module can never
> delete old or establish new dependence relationships. This also means
> that OSGi loses this capability as well, so either OSGi has to be
> redefined with the new restriction, or else OSGi bundles cannot be modules!
> .
>
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
--
- DML
From pbenedict at apache.org Wed Jul 13 20:32:15 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Wed, 13 Jul 2016 15:32:15 -0500
Subject: Discussion: #MutableConfigurations
In-Reply-To: <20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
Message-ID: <CABLGb9wu-Z7vZEraOgm9oLkgVmKfGMmmwBHrdvS6_fzbpwvf0A@mail.gmail.com>
Question: If Configurations aren't mutable, would that necessitate
rebuilding the Layer if you want a modified Configuration? And by
rebuilding, I mean obtaining a new Layer instance.
Cheers,
Paul
On Wed, Jul 13, 2016 at 9:30 AM, <mark.reinhold at oracle.com> wrote:
> Reference:
> http://openjdk.java.net/projects/jigsaw/spec/issues/#MutableConfigurations
>
> 2016/3/2 18:11:34 -0800, david.lloyd at redhat.com:
> > It appears from what I can see in the Jigsaw code, that once a
> > Configuration is calculated, it cannot be changed in any way,
>
> That's true.
>
> The overall model in the present design is that, given a set of modules,
> you first compute a configuration, which captures the resolution of all
> the modules' dependences in a consistent fashion. You then instantiate
> that as a Layer (or even more than one Layer, if you want).
>
> This model is motivated by one of our primary goals, namely reliable
> configuration. Resolving a complete configuration, rather than doing
> so incrementally, allows the early detection of missing, duplicate,
> and conflicting dependencies.
>
> > which
> > makes them unsuitable for use in containers which add and remove modules
> > at run time. It is not clear how such containers are expected to
> > function.
>
> They're expected to create Layers, which can be related hierarchically as
> needed (or perhaps more generally, cf. #NonHierarchicalLayers [1]).
>
> > If there is a deliberate intention that the Jigsaw system
> > will not support dynamic modification of layers/configurations
> > (including the dynamic addition and removal of modules at run time),
> > then that should be explicitly stated.
>
> I don't think it makes sense for Configurations to be mutable, for the
> reasons stated above. It may be that the Layer concept needs to be made
> more dynamic in certain ways, but that's a different matter.
>
> I intend to close this issue unless there are strong objections from
> other EG members.
>
> - Mark
>
>
> [1]
> http://openjdk.java.net/projects/jigsaw/spec/issues/#NonHierarchicalLayers
>
From david.lloyd at redhat.com Wed Jul 13 20:36:31 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 13 Jul 2016 15:36:31 -0500
Subject: Discussion: #MutableConfigurations
In-Reply-To: <CABLGb9wu-Z7vZEraOgm9oLkgVmKfGMmmwBHrdvS6_fzbpwvf0A@mail.gmail.com>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
<CABLGb9wu-Z7vZEraOgm9oLkgVmKfGMmmwBHrdvS6_fzbpwvf0A@mail.gmail.com>
Message-ID: <5786A64F.9000107@redhat.com>
Yes, and that means new modules, which means new classes.
On 07/13/2016 03:32 PM, Paul Benedict wrote:
> Question: If Configurations aren't mutable, would that necessitate
> rebuilding the Layer if you want a modified Configuration? And by
> rebuilding, I mean obtaining a new Layer instance.
>
> Cheers,
> Paul
>
> On Wed, Jul 13, 2016 at 9:30 AM, <mark.reinhold at oracle.com
> <mailto:mark.reinhold at oracle.com>> wrote:
>
> Reference:
> http://openjdk.java.net/projects/jigsaw/spec/issues/#MutableConfigurations
>
> 2016/3/2 18:11:34 -0800, david.lloyd at redhat.com
> <mailto:david.lloyd at redhat.com>:
> > It appears from what I can see in the Jigsaw code, that once a
> > Configuration is calculated, it cannot be changed in any way,
>
> That's true.
>
> The overall model in the present design is that, given a set of modules,
> you first compute a configuration, which captures the resolution of all
> the modules' dependences in a consistent fashion. You then instantiate
> that as a Layer (or even more than one Layer, if you want).
>
> This model is motivated by one of our primary goals, namely reliable
> configuration. Resolving a complete configuration, rather than doing
> so incrementally, allows the early detection of missing, duplicate,
> and conflicting dependencies.
>
> > which
> > makes them unsuitable for use in containers which add and remove modules
> > at run time. It is not clear how such containers are expected to
> > function.
>
> They're expected to create Layers, which can be related
> hierarchically as
> needed (or perhaps more generally, cf. #NonHierarchicalLayers [1]).
>
> > If there is a deliberate intention that the Jigsaw system
> > will not support dynamic modification of layers/configurations
> > (including the dynamic addition and removal of modules at run time),
> > then that should be explicitly stated.
>
> I don't think it makes sense for Configurations to be mutable, for the
> reasons stated above. It may be that the Layer concept needs to be made
> more dynamic in certain ways, but that's a different matter.
>
> I intend to close this issue unless there are strong objections from
> other EG members.
>
> - Mark
>
>
> [1]
> http://openjdk.java.net/projects/jigsaw/spec/issues/#NonHierarchicalLayers
>
>
--
- DML
From pbenedict at apache.org Wed Jul 13 20:58:17 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Wed, 13 Jul 2016 15:58:17 -0500
Subject: Discussion: #MutableConfigurations
In-Reply-To: <5786A64F.9000107@redhat.com>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
<CABLGb9wu-Z7vZEraOgm9oLkgVmKfGMmmwBHrdvS6_fzbpwvf0A@mail.gmail.com>
<5786A64F.9000107@redhat.com>
Message-ID: <CABLGb9yJDkb3=pMU4dyXivdw5NrXQGPYWdeTpOXGdmxzD=EDWw@mail.gmail.com>
Analyzing and resolving memory leaks is not my expertise, but I have done
it occasionally. Correct me if wrong, but if a Layer has to be rebuilt at
runtime when one module updates, doesn't that create a large graph of
objects in need of garbage collection? And doesn't that also put EE servers
in the danger of having ill-referenced old Layers that gobble up memory?
Given such a theoretical JEE 9 server which users Layers and Modules, I
would suppose rebuilding the Layer (instantiating new) means rebuilding
each Module and in turn each Class? I remember Mark Thomas (of Apache
Tomcat) once writing to Jigsaw-Dev@ on the topic of memory leaks [1]. I'm
not claiming my question/concern here is identical to his, but it might be
related.
Regarding hot deploy, I am familiar with classes being recompiled and
replaced in memory. But I don't think that's what David is driving at. So
what does it mean for module-info.java? If I change any "export" or
"require" statements in a live debugging session, is the answer do full
rebuild and redeploy? I think this similar is what David is driving at.
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-April/007376.html
Cheers,
Paul
On Wed, Jul 13, 2016 at 3:36 PM, David M. Lloyd <david.lloyd at redhat.com>
wrote:
> Yes, and that means new modules, which means new classes.
>
> On 07/13/2016 03:32 PM, Paul Benedict wrote:
>
>> Question: If Configurations aren't mutable, would that necessitate
>> rebuilding the Layer if you want a modified Configuration? And by
>> rebuilding, I mean obtaining a new Layer instance.
>>
>> Cheers,
>> Paul
>>
>> On Wed, Jul 13, 2016 at 9:30 AM, <mark.reinhold at oracle.com
>> <mailto:mark.reinhold at oracle.com>> wrote:
>>
>> Reference:
>>
>> http://openjdk.java.net/projects/jigsaw/spec/issues/#MutableConfigurations
>>
>> 2016/3/2 18:11:34 -0800, david.lloyd at redhat.com
>> <mailto:david.lloyd at redhat.com>:
>>
>> > It appears from what I can see in the Jigsaw code, that once a
>> > Configuration is calculated, it cannot be changed in any way,
>>
>> That's true.
>>
>> The overall model in the present design is that, given a set of
>> modules,
>> you first compute a configuration, which captures the resolution of
>> all
>> the modules' dependences in a consistent fashion. You then
>> instantiate
>> that as a Layer (or even more than one Layer, if you want).
>>
>> This model is motivated by one of our primary goals, namely reliable
>> configuration. Resolving a complete configuration, rather than doing
>> so incrementally, allows the early detection of missing, duplicate,
>> and conflicting dependencies.
>>
>> > which
>> > makes them unsuitable for use in containers which add and remove
>> modules
>> > at run time. It is not clear how such containers are expected to
>> > function.
>>
>> They're expected to create Layers, which can be related
>> hierarchically as
>> needed (or perhaps more generally, cf. #NonHierarchicalLayers [1]).
>>
>> > If there is a deliberate intention that the Jigsaw system
>> > will not support dynamic modification of layers/configurations
>> > (including the dynamic addition and removal of modules at run time),
>> > then that should be explicitly stated.
>>
>> I don't think it makes sense for Configurations to be mutable, for the
>> reasons stated above. It may be that the Layer concept needs to be
>> made
>> more dynamic in certain ways, but that's a different matter.
>>
>> I intend to close this issue unless there are strong objections from
>> other EG members.
>>
>> - Mark
>>
>>
>> [1]
>>
>> http://openjdk.java.net/projects/jigsaw/spec/issues/#NonHierarchicalLayers
>>
>>
>>
> --
> - DML
>
From neil.bartlett at paremus.com Wed Jul 13 22:25:48 2016
From: neil.bartlett at paremus.com (Neil Bartlett (Paremus))
Date: Wed, 13 Jul 2016 23:25:48 +0100
Subject: Discussion: #MutableConfigurations
In-Reply-To: <5786A429.1040601@redhat.com>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
<57866F90.5020703@redhat.com>
<C9342E73-D613-424A-8F92-510FCBF993E0@univ-mlv.fr>
<5786A429.1040601@redhat.com>
Message-ID: <2E469F09-1399-43BB-94FE-57099AD452EE@paremus.com>
> On 13 Jul 2016, at 21:27, David M. Lloyd <david.lloyd at redhat.com> wrote:
>
> On 07/13/2016 03:09 PM, Remi Forax wrote:
>> I may say something stupid but if you have one Layer by ClassLoader, why
>> it's not enough ?
>
> Even if you have one layer per classloader per module, you can add modules but you cannot update them when they are removed.
Speaking from an OSGi perspective (though NOT as an official representative of OSGI)? if we map the current OSGi bundle concept to a JPMS Layer then this has a few important consequences:
1. Bundle dependencies are not hierarchical but form a directed graph. I see there is a separate proposal for non-hierarchical layers, but I haven?t determined yet whether it is sufficient to achieve this mapping.
2. The dependency graph may contain cycles. Cycles are just as undesirable in OSGi as they are in JPMS, but the fact is they are supported so to forbid them now would be an incompatible change.
3. Bundles that have no dependers (i.e. they are leaf nodes) can easily be updated or uninstalled. But non-leaf bundles can cause effects on all the bundles that depend on them. This is handled by a resolving process that requires all bundles involved in it to be stopped, re-resolved (i.e. have their imports matched with corresponding exports) and then restarted. ?Stopping? and ?starting? mean invoking callback methods within the bundle so that they can clean up any instances that might refer to the obsolete types, e.g. threads. Thus a modifiable graph is intrinsically tied to the full lifecycle provided by OSGi. I find it difficult to imagine how JPMS could safely provide limited lifecycle facilities (add/update/remove) without replicating the full OSGI lifecycle or something like it.
4. If bundles are mapped to Layers then it seems likely that each Layer will consist of just a single Module, since we have no need for intra-bundle isolation other than that already provided by packages and types.
5. If Layers are arranged in an inter-dependent graph, and each contains a solitary Module, then a Layer looks very much like the thing we should be calling a ?module?, and the Modules as currently defined by JPMS are redundant!
6. OSGi?s dynamic abilities are enabled by a dynamic Service Registry. If a bundle providing a service is removed then all consumers of that service are notified so that they can release references to it. There is no corresponding facility in the ServiceLoader API. This is not a problem for OSGi users since we will continue using our own Service Registry, but how can JPMS safely support removal of a Layer/Module if it provides services that have been retrieved by another Layer/Module?
I tend to believe these issues are insurmountable, or would require more change to the JPMS spec than the EG is prepared to make. I expect therefore that OSGi will not be able to map bundles to JPMS modules. Instead our bundles would exist within the context of a single JPMS module (perhaps unnamed). This is a shame because OSGi will not be able to take advantage of the additional access-based isolation offered by JPMS, and must continue relying on its visibility-based isolation? on the other hand we at least avoid JPMS?s problems with #ReflectiveAccessToNonExportedTypes. My current work is on enabling some degree of interoperability so that bundles can declare dependencies on JPMS modules in the platform, which would be understood and honoured by the OSGi Framework.
>
>>
>> Remi
>>
>>
>> On July 13, 2016 6:42:56 PM GMT+02:00, "David M. Lloyd"
>> <david.lloyd at redhat.com> wrote:
>>
>> On 07/13/2016 09:30 AM, mark.reinhold at oracle.com wrote:
>>
>> Reference:
>> http://openjdk.java.net/projects/jigsaw/spec/issues/#MutableConfigurations
>>
>> 2016/3/2 18:11:34 -0800, david.lloyd at redhat.com:
>>
>> It appears from what I can see in the Jigsaw code, that once a
>> Configuration is calculated, it cannot be changed in any way,
>>
>>
>> That's true.
>>
>> The overall model in the present design is that, given a set of
>> modules,
>> you first compute a configuration, which captures the resolution
>> of all
>> the modules' dependences in a consistent fashion. You then
>> instantiate
>> that as a Lay! er (or even more than one Layer, if you want).
>>
Switching to addressing an earlier comment by Mark:
>> This model is motivated by one of our primary goals, namely reliable
>> configuration. Resolving a complete configuration, rather than doing
>> so incrementally, allows the early detection of missing, duplicate,
>> and conflicting dependencies.
The ability to modify a configuration after it has been created does not imply that it must be created incrementally, or that it is ?unreliable".
When launching an OSGi application we start with a number of bundles. These bundles are resolved together, allowing "early detection of missing, duplicate and conflicting dependencies? (though OSGi is more forgiving about duplicates and conflicts). At a later time, a subset of bundles can be updated or uninstalled, or new bundles added. These operations are batched and we can determine ahead of time whether a batch would succeed or would leave some bundles in an unresolved state. I?d like to know in what way this process can be characterised as unreliable.
>>
>> which
>> makes them unsuitable for use in containers which add and
>> remove modules
>> at run time. It is not clear how such containers are expected to
>> function.
>>
>>
>> They're expected to create Layers, which can be related
>> hierarchically as
>> needed (or perhaps more generally, cf. #NonHierarchicalLayers [1]).
>>
>> I! f there is a deliberate intention that the Jigsaw system
>> will not support dynamic modification of layers/configurations
>> (including the dynamic addition and removal of modules at
>> run time),
>> then that should be explicitly stated.
>>
>>
>> I don't think it makes sense for Configurations to be mutable,
>> for the
>> reasons stated above. It may be that the Layer concept needs to
>> be made
>> more dynamic in certain ways, but that's a different matter.
>>
>> I intend to close this issue unless there are strong objections from
>> other EG members.
>>
>>
>> In that case, other EG members take note: this means that hot deployment
>> as the world knows it is over as of Java EE 9, since a module can never
>> delete old or establish new dependence relationships. This also means
>> that OSGi loses this capability as well, so either OSGi has to be
>> redefined with the new restriction, or else OSGi bundles cannot be modules!
>> .
>>
>>
>> --
>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
> --
> - DML
Neil Bartlett
From ali.ebrahimi1781 at gmail.com Thu Jul 14 04:31:21 2016
From: ali.ebrahimi1781 at gmail.com (Ali Ebrahimi)
Date: Thu, 14 Jul 2016 09:01:21 +0430
Subject: Issue #ServiceLoaderEnhancement
In-Reply-To: <20160713180613.27999B7953@eggemoggin.niobe.net>
References: <CACzrW9Cp0tpZnU-wF3a+GYg2RBU=LeHixqAiZSFZyJjA10kawg@mail.gmail.com>
<20160713180613.27999B7953@eggemoggin.niobe.net>
Message-ID: <CAA0cW5DhKQpMVY0EZ5GkVPjz7dGAO_Th3-_1nzuV+jA4ENcmRw@mail.gmail.com>
Hi,
I suggests add ranking support for prioritizing service providers as OSGI
do.
provides com.services.api.MyService with
com.services.impl.MyServiceIMPL rank 10;
On Wed, Jul 13, 2016 at 10:36 PM, <mark.reinhold at oracle.com> wrote:
> 2016/7/1 2:27:39 -0700, Stephen Colebourne <scolebourne at joda.org>:
> > The ServiceLoader API is to be pushed as a key class by the module
> > system. This issue is proposing enhancements to the API. Others may
> > want to add their own proposals.
> >
> > Currently, ServiceLoader requires service implementations to have a
> > public no-args constructor on a public class. This is very
> > restrictive, especially in a Java 8 world of immutable classes and
> > factory methods (with constructors actively avoided).
> >
> > JSR-310 has a service for loading Chronology implementations. In
> > ThreeTen-Extra, I have a number of additional implementations [1].
> > Unfortunately, all the implementations are forced to have a public
> > no-args constructor, as ServiceLoader cannot make use of the singleton
> > INSTANCE constant. As can be seen, I have to mark the public no-args
> > constructor as deprecated [2]. This state of affairs is frankly
> > ridiculous, and is one of the reasons putting me off using
> > ServiceLoader non-JDK code.
> >
> > I propose that the spec of ServiceLoader is extended to make it more
> useful.
> >
> > Firstly, I propose that all reflection by ServiceLoader should use
> > setAccessible(true) to allow providers to be non-public.
>
> We could do this for service providers defined in modules, but not for
> those specified via the META-INF/services convention. The problem with
> the latter is that if ServiceLoader could instantiate an arbitrary
> non-public class then an adversary could spoof it into doing that simply
> by placing an appropriate META-INF/services configuration file on the
> class path. That's not possible for providers in modules, since the
> declaration of the provider and the provider itself are always tightly
> associated with each other, in the same artifact, so we can relax the
> restriction in that case.
>
> > Secondly, I propose that if no public no-args constructor is found on
> > the provider class, ServiceLoader should locate a public static
> > constant named "INSTANCE" and use that instead.
>
> That's reasonable, though we don't have much precedence for name-based
> conventions in the SE platform itself.
>
> > This proposal could be extended further. Instead of searching for
> > "INSTANCE", it could look for all public static constants of the
> > provider type, allowing one provider class to result in many provider
> > instances.
>
> I suspect that might be one heuristic too much. I could see searching
> for INSTANCE, and maybe also INSTANCES where the latter refers to an
> array or a collection of appropriate type, but just taking anything based
> solely upon type matches could lead to difficult-to-diagnose failures.
>
> An alternative is to take an explicitly-declarative approach, defining
> something like a @Provides annotation to identify the constructors,
> factories, or fields that construct or contain actual provider objects.
> I'll give this some thought.
>
> > Thirdly, it should be possible to access the set of provider classes
> > without instantiating them. This would allow applications to use the
> > module-info design for services without using ServiceLoader, eg to
> > reflect on the provider classes, perhaps to search for annotations.
>
> As an historical note, the original idea way back in Java 1.3 was for
> complex providers to be represented by small, lightweight proxy objects
> rather than full providers, or types. To quote the specification (of
> the sun.misc.Service class, before it became java.util.ServiceLoader):
>
> * ... This <i>provider class</i> will typically not be the
> entire
> * provider itself but rather a proxy that contains enough information to
> * decide whether the provider is able to satisfy a particular request
> together
> * with code that can create the actual provider on demand. The details of
> * provider classes tend to be highly service-specific; no single class or
> * interface could possibly unify them, so no such class has been defined.
>
> Most of the services defined and used in the JDK itself follow this
> advice, but I accept that many others out in the world do not, so at
> this point people reasonably want to be able to filter providers based
> on types and annotations thereon. This should be straightforward; I'll
> work up a proposal.
>
> > I hope this proposal is given serious consideration, as at present the
> > design of ServiceLoader is simply too restrictive to be appropriate to
> > be hard coded into the module specification.
>
> These are all good suggestions; thanks for raising them here. I've added
> them to the issue list:
>
>
> http://openjdk.java.net/projects/jigsaw/spec/issues/#ServiceLoaderEnhancements
>
> - Mark
>
--
Best Regards,
Ali Ebrahimi
From Alan.Bateman at oracle.com Thu Jul 14 11:05:52 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 14 Jul 2016 12:05:52 +0100
Subject: Discussion: #MutableConfigurations
In-Reply-To: <CABLGb9yJDkb3=pMU4dyXivdw5NrXQGPYWdeTpOXGdmxzD=EDWw@mail.gmail.com>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
<CABLGb9wu-Z7vZEraOgm9oLkgVmKfGMmmwBHrdvS6_fzbpwvf0A@mail.gmail.com>
<5786A64F.9000107@redhat.com>
<CABLGb9yJDkb3=pMU4dyXivdw5NrXQGPYWdeTpOXGdmxzD=EDWw@mail.gmail.com>
Message-ID: <6b19e63c-e362-2d10-2765-267e50f065e8@oracle.com>
I don't wish to interfere in this discussion but just a few points:
On 13/07/2016 21:58, Paul Benedict wrote:
> Analyzing and resolving memory leaks is not my expertise, but I have done
> it occasionally. Correct me if wrong, but if a Layer has to be rebuilt at
> runtime when one module updates, doesn't that create a large graph of
> objects in need of garbage collection? And doesn't that also put EE servers
> in the danger of having ill-referenced old Layers that gobble up memory?
> Given such a theoretical JEE 9 server which users Layers and Modules, I
> would suppose rebuilding the Layer (instantiating new) means rebuilding
> each Module and in turn each Class? I remember Mark Thomas (of Apache
> Tomcat) once writing to Jigsaw-Dev@ on the topic of memory leaks [1]. I'm
> not claiming my question/concern here is identical to his, but it might be
> related.
There are stress tests in the JDK 9 forest to exercise this. Clearly
someone could keep a reference to a Module or its ClassLoader that keeps
an entire graph of objects alive but that is no different to other cases.
The proposed module system was never meant to be a dynamic module system
where modules in a graph of modules (module layer) could be GC'ed and
re-instantiated individually. That's a different design.
>
> Regarding hot deploy, I am familiar with classes being recompiled and
> replaced in memory. But I don't think that's what David is driving at. So
> what does it mean for module-info.java? If I change any "export" or
> "require" statements in a live debugging session, is the answer do full
> rebuild and redeploy? I think this similar is what David is driving at.
>
If you are familiar with redefineClasses then you will know its
limitations too. There isn't any support in the platform for arbitrary
schema changes. There are frameworks and EE servers that work around
that of course but that's a big other topic.
If you are changing a module declaration then you will typically create
a new configuration (resolution, post-resolution checks) and then
instantiate it as a new layer. There is support for tool agents to
change modules in the runtime, this is limited to expanding the set of
exports, expanding the set of modules that it reads, or expanding the
set of services that it uses or provider. It may be possible for
debuggers to leverage this for some limited cases but it not arbitrary
changing a module.
-Alan
From forax at univ-mlv.fr Thu Jul 14 15:06:36 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Thu, 14 Jul 2016 17:06:36 +0200 (CEST)
Subject: Me trying to explain the problem behind
#ReflectiveAccessToNonExportedTypes
In-Reply-To: <1627574708.1456733.1468507103919.JavaMail.zimbra@u-pem.fr>
Message-ID: <1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
Let me try to explain again the problem and the possible solution.
Let say i have two modules, module1 and module2.
In module1, I have two packages module1.exported and module1.nonexported,
so module1 has this module-info.java
module module1 {
exports module1.exported;
}
in module1.nonexported, I have a class A that's implement an interface I defined like this:
package module1.nonexported;
import module1.exported.I;
public class A implements I {
@Override
public void foo() {
System.out.println("foo");
}
}
I is defined in module1.exported like this:
package module1.exported;
import module1.nonexported.A;
public interface I {
public void foo();
static I instance() {
return new A();
}
}
Now, in module2, i want to use I so module2 requires module1, like this:
module module2 {
requires module1;
}
and I have a main defined in the package modul2.main, like that:
package module2.main;
import java.lang.reflect.Method;
import module1.exported.I;
public class Main {
public static void main(String[] args) throws ReflectiveOperationException {
System.out.println(Class.forName("module1.nonexported.A"));
I i = I.instance();
Method m = i.getClass().getMethod("foo");
m.invoke(i);
}
}
If i run this code in with the two modules in the classpath,
everything work and the code prints
class module1.nonexported.A
foo
Now, if if the two modules are in the modulepath,
Class.forName works but m.invoke() throws an IllegalAccessException because A is in a non exported package
java.lang.IllegalAccessException: class module2.main.Main (in module module2) cannot access class module1.nonexported.A (in module module1) because module module1 does not export module1.nonexported to module module2
If i try with m.setAccessible(true), before m.invoke(), setAccessible throws an exception InaccessibleObjectException
java.lang.reflect.InaccessibleObjectException: Unable to make member of class module1.nonexported.A accessible: module module1 does not export module1.nonexported to module module2
A way to solve this issue, is to use the interface I to call foo instead of using the class A,
Method m = I.class.getMethod("foo");
m.invoke(i);
in that case, everything is Ok, because i don't try to use the method foo of A anymore.
What Mark as proposed is to introduce a new "export dynamic" semantics which doesn't allow the package to be visible at compile time but to be visible at runtime.
This solve the issue by supposing that everyone will write export dynamic on every non exported package to be backward compatible with the Java 8 behavior.
While i agree that "export dynamic" is a semantics that jigsaw should provide, i disagree with the syntax "export dynamic" because for me, the a non exported package should have this semantics in order to be backward compatible with existing code.
So we should have 3 way to export or not a package,
1. don't export at compile time, don't export at runtime
2. don't export at compile time, export at runtime
3. export at compile time, export at runtime.
2 should be the default, 1 should be use by the package of the JDK (or any other libs) that want strong security, 3 is for the package that defines API (that will be maintained forever).
R?mi
From david.lloyd at redhat.com Thu Jul 14 15:31:38 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 14 Jul 2016 10:31:38 -0500
Subject: Me trying to explain the problem behind
#ReflectiveAccessToNonExportedTypes
In-Reply-To: <1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
References: <1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
Message-ID: <5787B05A.7090506@redhat.com>
On 07/14/2016 10:06 AM, Remi Forax wrote:
> Let me try to explain again the problem and the possible solution.
>
> Let say i have two modules, module1 and module2.
> In module1, I have two packages module1.exported and module1.nonexported,
> so module1 has this module-info.java
>
> module module1 {
> exports module1.exported;
> }
>
> in module1.nonexported, I have a class A that's implement an interface I defined like this:
> package module1.nonexported;
>
> import module1.exported.I;
>
> public class A implements I {
> @Override
> public void foo() {
> System.out.println("foo");
> }
> }
>
> I is defined in module1.exported like this:
> package module1.exported;
>
> import module1.nonexported.A;
>
> public interface I {
> public void foo();
>
> static I instance() {
> return new A();
> }
> }
>
> Now, in module2, i want to use I so module2 requires module1, like this:
> module module2 {
> requires module1;
> }
>
> and I have a main defined in the package modul2.main, like that:
> package module2.main;
>
> import java.lang.reflect.Method;
>
> import module1.exported.I;
>
> public class Main {
> public static void main(String[] args) throws ReflectiveOperationException {
> System.out.println(Class.forName("module1.nonexported.A"));
>
> I i = I.instance();
> Method m = i.getClass().getMethod("foo");
> m.invoke(i);
> }
> }
>
> If i run this code in with the two modules in the classpath,
> everything work and the code prints
> class module1.nonexported.A
> foo
>
> Now, if if the two modules are in the modulepath,
> Class.forName works but m.invoke() throws an IllegalAccessException because A is in a non exported package
> java.lang.IllegalAccessException: class module2.main.Main (in module module2) cannot access class module1.nonexported.A (in module module1) because module module1 does not export module1.nonexported to module module2
>
> If i try with m.setAccessible(true), before m.invoke(), setAccessible throws an exception InaccessibleObjectException
> java.lang.reflect.InaccessibleObjectException: Unable to make member of class module1.nonexported.A accessible: module module1 does not export module1.nonexported to module module2
>
> A way to solve this issue, is to use the interface I to call foo instead of using the class A,
> Method m = I.class.getMethod("foo");
> m.invoke(i);
>
> in that case, everything is Ok, because i don't try to use the method foo of A anymore.
>
> What Mark as proposed is to introduce a new "export dynamic" semantics which doesn't allow the package to be visible at compile time but to be visible at runtime.
> This solve the issue by supposing that everyone will write export dynamic on every non exported package to be backward compatible with the Java 8 behavior.
>
> While i agree that "export dynamic" is a semantics that jigsaw should provide, i disagree with the syntax "export dynamic" because for me, the a non exported package should have this semantics in order to be backward compatible with existing code.
> So we should have 3 way to export or not a package,
> 1. don't export at compile time, don't export at runtime
> 2. don't export at compile time, export at runtime
> 3. export at compile time, export at runtime.
>
> 2 should be the default, 1 should be use by the package of the JDK (or any other libs) that want strong security, 3 is for the package that defines API (that will be maintained forever).
This is a good summary and I think matches my understanding. But the
basic problem is that if everyone is doing this "export dynamic" then
the safety proposed by restricting "public" is a false promise: we are
providing the feature only to take it away again, resulting in the exact
same situation we have today, but with no alternatives. If we have to
do this then I think the feature itself needs to be re-examined.
What I was suggesting was that we re-separate the notion of exports from
accessibility altogether, so you again have three different situations
rather than two:
A. Compile time
B. Class link at run time
C. Reflection at run time
This changes the basic "export" back to mean "make available for
linkage", with no bearing on accessibility. Then you have (at minimum)
the following ways to export (or not) a package:
1. No export at compile time, no export but reflection-only at run time
(but only for public types & members)
2. Yes export at compile time, yes export at run time (but only for
public types & members)
3. Yes export at compile time and run time, including specific grants to
access package-private members from targeted modules and/or packages
Note: We could add a 1? which is an export at run time but not compile
time if the EG agrees; I don't think that is particularly important
though as build tools can accomplish this easily enough. I don't want
to derail the core point of the discussion on this topic, so I'm leaving
it aside for now.
This way the Java 8 behavior is retained with no "export dynamic"
anywhere in sight. Rather you'd have something like "export private",
which you'd only grant for those special cases where a package must be
shared to a specific friend. Things not intended to be shared can be
safely migrated to the new friend package feature. No new access limits
are introduced - instead old ones are enhanced in what I think is a
simple and intuitive way. All the compatibility concerns go away, but
we can still use the mechanism to seal up the problems in the JDK with
shared code and things which should not be public.
We should retain setAccessible() for now for compatibility. We could
also introduce a "--safe-mode" flag which disables setAccessible()
*across the board* (that is, not just for modular code) without
controversy, because all those public members will remain accessible
without "help", plus a way is provided to allow access to be volunteered
from one package to another (along much the same philosophical lines as
MethodHandles.Lookup IMO).
The platform EG can then later decide if this safer mode is good for the
platform, and if so, could move to making this the default and possibly
adding an "--unsafe-mode" for a limited time.
--
- DML
From mark.reinhold at oracle.com Thu Jul 14 16:33:04 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu, 14 Jul 2016 09:33:04 -0700
Subject: Me trying to explain the problem behind
#ReflectiveAccessToNonExportedTypes
In-Reply-To: <1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
References: <1627574708.1456733.1468507103919.JavaMail.zimbra@u-pem.fr>
<1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
Message-ID: <20160714093304.841168761eggemoggin.niobe.net>
2016/7/14 8:06:36 -0700, forax at univ-mlv.fr:
> Let me try to explain again the problem and the possible solution.
>
> ...
>
> What Mark as proposed is to introduce a new "export dynamic" semantics
> which doesn't allow the package to be visible at compile time but to
> be visible at runtime.
> This solve the issue by supposing that everyone will write export
> dynamic on every non exported package to be backward compatible with
> the Java 8 behavior.
That wasn't my assumption.
The intent here is that the author of a module should be able to declare
which of the module's packages are exported, and when, and which are
not. The author should also be able to be reasonably sure that these
declarations will not be overridden lightly, by any random code that
uses reflection. (Overriding encapsulation via second-class mechanisms
such as command-line options or container-based module-descriptor
rewriting, however, is fine.) This is what strong encapsulation is
all about.
> While i agree that "export dynamic" is a semantics that jigsaw should
> provide, i disagree with the syntax "export dynamic" because for me,
> the a non exported package should have this semantics in order to be
> backward compatible with existing code.
> So we should have 3 way to export or not a package,
> 1. don't export at compile time, don't export at runtime
> 2. don't export at compile time, export at runtime
> 3. export at compile time, export at runtime.
>
> 2 should be the default, 1 should be use by the package of the JDK (or
> any other libs) that want strong security, 3 is for the package that
> defines API (that will be maintained forever).
A problem with changing the default is that many developers will never
think to do anything else, and leave all their packages unnecessarily
exported. We have examples of this sort of thing elsewhere in Java,
e.g., default public constructors, which have been problematic for
years. Do we really want to repeat that kind of mistake?
What do you think of allowing wildcards, i.e., `exports dynamic *`?
That way a developer could explicitly choose to export everything at
run time, but could do so in a very compact way.
- Mark
From pbenedict at apache.org Thu Jul 14 16:47:20 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Thu, 14 Jul 2016 11:47:20 -0500
Subject: Me trying to explain the problem behind
#ReflectiveAccessToNonExportedTypes
In-Reply-To: <5787bed7.19c8620a.64afb.ffffa251SMTPIN_ADDED_BROKEN@mx.google.com>
References: <1627574708.1456733.1468507103919.JavaMail.zimbra@u-pem.fr>
<1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
<5787bed7.19c8620a.64afb.ffffa251SMTPIN_ADDED_BROKEN@mx.google.com>
Message-ID: <CABLGb9x8XZkM+FGi_dioiMiENQyGJEubbeJmJ2BtEkg8joKUAw@mail.gmail.com>
On Thu, Jul 14, 2016 at 11:33 AM, <mark.reinhold at oracle.com> wrote:
> 2016/7/14 8:06:36 -0700, forax at univ-mlv.fr:
> > Let me try to explain again the problem and the possible solution.
> >
> > ...
> >
> > What Mark as proposed is to introduce a new "export dynamic" semantics
> > which doesn't allow the package to be visible at compile time but to
> > be visible at runtime.
> > This solve the issue by supposing that everyone will write export
> > dynamic on every non exported package to be backward compatible with
> > the Java 8 behavior.
>
> That wasn't my assumption.
>
> The intent here is that the author of a module should be able to declare
> which of the module's packages are exported, and when, and which are
> not. The author should also be able to be reasonably sure that these
> declarations will not be overridden lightly, by any random code that
> uses reflection. (Overriding encapsulation via second-class mechanisms
> such as command-line options or container-based module-descriptor
> rewriting, however, is fine.) This is what strong encapsulation is
> all about.
>
I don't thinks can be reasonable assured. With the ability to rewrite
Module Descriptors at packaging, assembly, or run time, I think this effort
is just trying to delay the inevitable. Just as there are ton of reflection
libraries for users, I expect module descriptor rewriting libraries to be
popular too. The only absolute way of stopping this and securing things is
to use the Security Manager. That's the only mechanism in the JDK that has
the authority to stop code from reflecting what is forbidden.
>
> > While i agree that "export dynamic" is a semantics that jigsaw should
> > provide, i disagree with the syntax "export dynamic" because for me,
> > the a non exported package should have this semantics in order to be
> > backward compatible with existing code.
> > So we should have 3 way to export or not a package,
> > 1. don't export at compile time, don't export at runtime
> > 2. don't export at compile time, export at runtime
> > 3. export at compile time, export at runtime.
> >
> > 2 should be the default, 1 should be use by the package of the JDK (or
> > any other libs) that want strong security, 3 is for the package that
> > defines API (that will be maintained forever).
>
> A problem with changing the default is that many developers will never
> think to do anything else, and leave all their packages unnecessarily
> exported. We have examples of this sort of thing elsewhere in Java,
> e.g., default public constructors, which have been problematic for
> years. Do we really want to repeat that kind of mistake?
>
> What do you think of allowing wildcards, i.e., `exports dynamic *`?
> That way a developer could explicitly choose to export everything at
> run time, but could do so in a very compact way.
>
I think "exports dynamic *" is tremendously better than listing out per
package. Good suggestion.
From mark.reinhold at oracle.com Thu Jul 14 17:44:34 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Thu, 14 Jul 2016 10:44:34 -0700
Subject: Discussion: #MutableConfigurations
In-Reply-To: <6b19e63c-e362-2d10-2765-267e50f065e8@oracle.com>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
<CABLGb9wu-Z7vZEraOgm9oLkgVmKfGMmmwBHrdvS6_fzbpwvf0A@mail.gmail.com>
<5786A64F.9000107@redhat.com>
<CABLGb9yJDkb3=pMU4dyXivdw5NrXQGPYWdeTpOXGdmxzD=EDWw@mail.gmail.com>
<6b19e63c-e362-2d10-2765-267e50f065e8@oracle.com>
Message-ID: <20160714104434.955149596eggemoggin.niobe.net>
2016/7/14 4:05:52 -0700, alan.bateman at oracle.com:
> The proposed module system was never meant to be a dynamic module system
> where modules in a graph of modules (module layer) could be GC'ed and
> re-instantiated individually. That's a different design.
This is probably what you meant, but just to be crystal clear: The
present design does not support the release and re-instantiation of an
arbitrary layer of modules, but it most definitely supports the release
and re-instantiation of any leaf layer, i.e., layers upon which no other
layers depend. That's how we support all the dynamic-configuration
requirements [1].
- Mark
[1] http://openjdk.java.net/projects/jigsaw/spec/reqs/#dynamic-configuration
From forax at univ-mlv.fr Thu Jul 14 21:46:21 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Thu, 14 Jul 2016 23:46:21 +0200 (CEST)
Subject: Me trying to explain the problem behind
#ReflectiveAccessToNonExportedTypes
In-Reply-To: <20160714093304.841168761eggemoggin.niobe.net>
References: <1627574708.1456733.1468507103919.JavaMail.zimbra@u-pem.fr>
<1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
<20160714093304.841168761eggemoggin.niobe.net>
Message-ID: <1028346570.1564026.1468532781557.JavaMail.zimbra@u-pem.fr>
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Jeudi 14 Juillet 2016 18:33:04
> Objet: Re: Me trying to explain the problem behind #ReflectiveAccessToNonExportedTypes
>
> 2016/7/14 8:06:36 -0700, forax at univ-mlv.fr:
> > Let me try to explain again the problem and the possible solution.
> >
> > ...
> >
> > What Mark as proposed is to introduce a new "export dynamic" semantics
> > which doesn't allow the package to be visible at compile time but to
> > be visible at runtime.
> > This solve the issue by supposing that everyone will write export
> > dynamic on every non exported package to be backward compatible with
> > the Java 8 behavior.
>
> That wasn't my assumption.
>
> The intent here is that the author of a module should be able to declare
> which of the module's packages are exported, and when, and which are
> not. The author should also be able to be reasonably sure that these
> declarations will not be overridden lightly, by any random code that
> uses reflection. (Overriding encapsulation via second-class mechanisms
> such as command-line options or container-based module-descriptor
> rewriting, however, is fine.) This is what strong encapsulation is
> all about.
The choice for the author seems to be strong encapsulation vs users being able to use their favorite reflection library.
>
> > While i agree that "export dynamic" is a semantics that jigsaw should
> > provide, i disagree with the syntax "export dynamic" because for me,
> > the a non exported package should have this semantics in order to be
> > backward compatible with existing code.
> > So we should have 3 way to export or not a package,
> > 1. don't export at compile time, don't export at runtime
> > 2. don't export at compile time, export at runtime
> > 3. export at compile time, export at runtime.
> >
> > 2 should be the default, 1 should be use by the package of the JDK (or
> > any other libs) that want strong security, 3 is for the package that
> > defines API (that will be maintained forever).
>
> A problem with changing the default is that many developers will never
> think to do anything else, and leave all their packages unnecessarily
> exported. We have examples of this sort of thing elsewhere in Java,
> e.g., default public constructors, which have been problematic for
> years. Do we really want to repeat that kind of mistake?
>
> What do you think of allowing wildcards, i.e., `exports dynamic *`?
> That way a developer could explicitly choose to export everything at
> run time, but could do so in a very compact way.
Wildcards seems a great idea.
But if we want authors to make a decision, instead of choosing a default for them,
we should let them to make a decision.
So wildcards should exist in both way 'export dynamic *' or 'export noexport *'
(or whatever we call it) and we should mandate that a module to have an export
wildcard declaration.
It's more verbose that the current settings but at least this make the choice explicit.
>
> - Mark
>
R?mi
From forax at univ-mlv.fr Fri Jul 15 08:18:42 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Fri, 15 Jul 2016 10:18:42 +0200 (CEST)
Subject: Me trying to explain the problem behind
#ReflectiveAccessToNonExportedTypes
In-Reply-To: <5787B05A.7090506@redhat.com>
References: <1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
<5787B05A.7090506@redhat.com>
Message-ID: <1363010886.1592154.1468570722083.JavaMail.zimbra@u-pem.fr>
----- Mail original -----
> De: "David M. Lloyd" <david.lloyd at redhat.com>
> ?: jpms-spec-experts at openjdk.java.net
> Envoy?: Jeudi 14 Juillet 2016 17:31:38
> Objet: Re: Me trying to explain the problem behind #ReflectiveAccessToNonExportedTypes
>
> On 07/14/2016 10:06 AM, Remi Forax wrote:
> > Let me try to explain again the problem and the possible solution.
> >
> > Let say i have two modules, module1 and module2.
> > In module1, I have two packages module1.exported and module1.nonexported,
> > so module1 has this module-info.java
> >
> > module module1 {
> > exports module1.exported;
> > }
> >
> > in module1.nonexported, I have a class A that's implement an interface I
> > defined like this:
> > package module1.nonexported;
> >
> > import module1.exported.I;
> >
> > public class A implements I {
> > @Override
> > public void foo() {
> > System.out.println("foo");
> > }
> > }
> >
> > I is defined in module1.exported like this:
> > package module1.exported;
> >
> > import module1.nonexported.A;
> >
> > public interface I {
> > public void foo();
> >
> > static I instance() {
> > return new A();
> > }
> > }
> >
> > Now, in module2, i want to use I so module2 requires module1, like this:
> > module module2 {
> > requires module1;
> > }
> >
> > and I have a main defined in the package modul2.main, like that:
> > package module2.main;
> >
> > import java.lang.reflect.Method;
> >
> > import module1.exported.I;
> >
> > public class Main {
> > public static void main(String[] args) throws
> > ReflectiveOperationException {
> > System.out.println(Class.forName("module1.nonexported.A"));
> >
> > I i = I.instance();
> > Method m = i.getClass().getMethod("foo");
> > m.invoke(i);
> > }
> > }
> >
> > If i run this code in with the two modules in the classpath,
> > everything work and the code prints
> > class module1.nonexported.A
> > foo
> >
> > Now, if if the two modules are in the modulepath,
> > Class.forName works but m.invoke() throws an IllegalAccessException because
> > A is in a non exported package
> > java.lang.IllegalAccessException: class module2.main.Main (in module
> > module2) cannot access class module1.nonexported.A (in module module1)
> > because module module1 does not export module1.nonexported to module
> > module2
> >
> > If i try with m.setAccessible(true), before m.invoke(), setAccessible
> > throws an exception InaccessibleObjectException
> > java.lang.reflect.InaccessibleObjectException: Unable to make member of
> > class module1.nonexported.A accessible: module module1 does not export
> > module1.nonexported to module module2
> >
> > A way to solve this issue, is to use the interface I to call foo instead of
> > using the class A,
> > Method m = I.class.getMethod("foo");
> > m.invoke(i);
> >
> > in that case, everything is Ok, because i don't try to use the method foo
> > of A anymore.
> >
> > What Mark as proposed is to introduce a new "export dynamic" semantics
> > which doesn't allow the package to be visible at compile time but to be
> > visible at runtime.
> > This solve the issue by supposing that everyone will write export dynamic
> > on every non exported package to be backward compatible with the Java 8
> > behavior.
> >
> > While i agree that "export dynamic" is a semantics that jigsaw should
> > provide, i disagree with the syntax "export dynamic" because for me, the a
> > non exported package should have this semantics in order to be backward
> > compatible with existing code.
> > So we should have 3 way to export or not a package,
> > 1. don't export at compile time, don't export at runtime
> > 2. don't export at compile time, export at runtime
> > 3. export at compile time, export at runtime.
> >
> > 2 should be the default, 1 should be use by the package of the JDK (or any
> > other libs) that want strong security, 3 is for the package that defines
> > API (that will be maintained forever).
>
> This is a good summary and I think matches my understanding. But the
> basic problem is that if everyone is doing this "export dynamic" then
> the safety proposed by restricting "public" is a false promise: we are
> providing the feature only to take it away again, resulting in the exact
> same situation we have today, but with no alternatives. If we have to
> do this then I think the feature itself needs to be re-examined.
>
> What I was suggesting was that we re-separate the notion of exports from
> accessibility altogether, so you again have three different situations
> rather than two:
>
> A. Compile time
> B. Class link at run time
> C. Reflection at run time
B and C may be indistinguishable. For the OpenJdk + Hotspot, you can distinguish between access control done by the VM in C++ and access control done by the reflection API done at runtime. For OpenJDK + IBM J9, there is only one entry point for the the access checks. When specifying the JSR292 (java.lang.invoke + invokedynamic), we spend a lot of time to be sure that reflection behavior and bytecode behavior were aligned. I don't think that trying to separate B and C is possible that late in the game.
>
> This changes the basic "export" back to mean "make available for
> linkage", with no bearing on accessibility. Then you have (at minimum)
> the following ways to export (or not) a package:
>
> 1. No export at compile time, no export but reflection-only at run time
> (but only for public types & members)
> 2. Yes export at compile time, yes export at run time (but only for
> public types & members)
> 3. Yes export at compile time and run time, including specific grants to
> access package-private members from targeted modules and/or packages
>
> Note: We could add a 1? which is an export at run time but not compile
> time if the EG agrees; I don't think that is particularly important
> though as build tools can accomplish this easily enough. I don't want
> to derail the core point of the discussion on this topic, so I'm leaving
> it aside for now.
friend accesssibility doesn't solve the issue here, when you want 'strong encapsulation' i.e. no way to use reflection to access members of a not-exported class, you should not have to provide the list of all possible existing module. I think we want 'export private' (strong encapsulation) and 'export dynamic' to be available without having a friend list.
And again, changing the meaning of package private is a no go for me, by example for a package like java.lang.invoke, i want the package to be exported, i want strong encapsulation and i want some classes inside the package by example the class that implements the whole method handle hierarchy to share secret (using the package private access).
>
> This way the Java 8 behavior is retained with no "export dynamic"
> anywhere in sight. Rather you'd have something like "export private",
> which you'd only grant for those special cases where a package must be
> shared to a specific friend. Things not intended to be shared can be
> safely migrated to the new friend package feature.
I agree that we should have a way to have Java 8 behavior without having to declare every packages with export dynamic,
see my discussion with mark about specifying a default export behavior.
> No new access limits
> are introduced - instead old ones are enhanced in what I think is a
> simple and intuitive way. All the compatibility concerns go away, but
> we can still use the mechanism to seal up the problems in the JDK with
> shared code and things which should not be public.
>
> We should retain setAccessible() for now for compatibility. We could
> also introduce a "--safe-mode" flag which disables setAccessible()
> *across the board* (that is, not just for modular code) without
> controversy, because all those public members will remain accessible
> without "help", plus a way is provided to allow access to be volunteered
> from one package to another (along much the same philosophical lines as
> MethodHandles.Lookup IMO).
Deprecating setAccessible is another story for me.
>
> The platform EG can then later decide if this safer mode is good for the
> platform, and if so, could move to making this the default and possibly
> adding an "--unsafe-mode" for a limited time.
>
> --
> - DML
>
regards,
R?mi
From david.lloyd at redhat.com Fri Jul 15 12:42:26 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 15 Jul 2016 07:42:26 -0500
Subject: Me trying to explain the problem behind
#ReflectiveAccessToNonExportedTypes
In-Reply-To: <1363010886.1592154.1468570722083.JavaMail.zimbra@u-pem.fr>
References: <1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
<5787B05A.7090506@redhat.com>
<1363010886.1592154.1468570722083.JavaMail.zimbra@u-pem.fr>
Message-ID: <5788DA32.1000402@redhat.com>
On 7/15/16 3:18 AM, Remi Forax wrote:
> ----- Mail original -----
>> De: "David M. Lloyd" <david.lloyd at redhat.com>
>> ?: jpms-spec-experts at openjdk.java.net
>> Envoy?: Jeudi 14 Juillet 2016 17:31:38
>> Objet: Re: Me trying to explain the problem behind #ReflectiveAccessToNonExportedTypes
>>
>> On 07/14/2016 10:06 AM, Remi Forax wrote:
>>> Let me try to explain again the problem and the possible solution.
>>>
>>> Let say i have two modules, module1 and module2.
>>> In module1, I have two packages module1.exported and module1.nonexported,
>>> so module1 has this module-info.java
>>>
>>> module module1 {
>>> exports module1.exported;
>>> }
>>>
>>> in module1.nonexported, I have a class A that's implement an interface I
>>> defined like this:
>>> package module1.nonexported;
>>>
>>> import module1.exported.I;
>>>
>>> public class A implements I {
>>> @Override
>>> public void foo() {
>>> System.out.println("foo");
>>> }
>>> }
>>>
>>> I is defined in module1.exported like this:
>>> package module1.exported;
>>>
>>> import module1.nonexported.A;
>>>
>>> public interface I {
>>> public void foo();
>>>
>>> static I instance() {
>>> return new A();
>>> }
>>> }
>>>
>>> Now, in module2, i want to use I so module2 requires module1, like this:
>>> module module2 {
>>> requires module1;
>>> }
>>>
>>> and I have a main defined in the package modul2.main, like that:
>>> package module2.main;
>>>
>>> import java.lang.reflect.Method;
>>>
>>> import module1.exported.I;
>>>
>>> public class Main {
>>> public static void main(String[] args) throws
>>> ReflectiveOperationException {
>>> System.out.println(Class.forName("module1.nonexported.A"));
>>>
>>> I i = I.instance();
>>> Method m = i.getClass().getMethod("foo");
>>> m.invoke(i);
>>> }
>>> }
>>>
>>> If i run this code in with the two modules in the classpath,
>>> everything work and the code prints
>>> class module1.nonexported.A
>>> foo
>>>
>>> Now, if if the two modules are in the modulepath,
>>> Class.forName works but m.invoke() throws an IllegalAccessException because
>>> A is in a non exported package
>>> java.lang.IllegalAccessException: class module2.main.Main (in module
>>> module2) cannot access class module1.nonexported.A (in module module1)
>>> because module module1 does not export module1.nonexported to module
>>> module2
>>>
>>> If i try with m.setAccessible(true), before m.invoke(), setAccessible
>>> throws an exception InaccessibleObjectException
>>> java.lang.reflect.InaccessibleObjectException: Unable to make member of
>>> class module1.nonexported.A accessible: module module1 does not export
>>> module1.nonexported to module module2
>>>
>>> A way to solve this issue, is to use the interface I to call foo instead of
>>> using the class A,
>>> Method m = I.class.getMethod("foo");
>>> m.invoke(i);
>>>
>>> in that case, everything is Ok, because i don't try to use the method foo
>>> of A anymore.
>>>
>>> What Mark as proposed is to introduce a new "export dynamic" semantics
>>> which doesn't allow the package to be visible at compile time but to be
>>> visible at runtime.
>>> This solve the issue by supposing that everyone will write export dynamic
>>> on every non exported package to be backward compatible with the Java 8
>>> behavior.
>>>
>>> While i agree that "export dynamic" is a semantics that jigsaw should
>>> provide, i disagree with the syntax "export dynamic" because for me, the a
>>> non exported package should have this semantics in order to be backward
>>> compatible with existing code.
>>> So we should have 3 way to export or not a package,
>>> 1. don't export at compile time, don't export at runtime
>>> 2. don't export at compile time, export at runtime
>>> 3. export at compile time, export at runtime.
>>>
>>> 2 should be the default, 1 should be use by the package of the JDK (or any
>>> other libs) that want strong security, 3 is for the package that defines
>>> API (that will be maintained forever).
>>
>> This is a good summary and I think matches my understanding. But the
>> basic problem is that if everyone is doing this "export dynamic" then
>> the safety proposed by restricting "public" is a false promise: we are
>> providing the feature only to take it away again, resulting in the exact
>> same situation we have today, but with no alternatives. If we have to
>> do this then I think the feature itself needs to be re-examined.
>>
>> What I was suggesting was that we re-separate the notion of exports from
>> accessibility altogether, so you again have three different situations
>> rather than two:
>>
>> A. Compile time
>> B. Class link at run time
>> C. Reflection at run time
>
> B and C may be indistinguishable. For the OpenJdk + Hotspot, you can distinguish between access control done by the VM in C++ and access control done by the reflection API done at runtime. For OpenJDK + IBM J9, there is only one entry point for the the access checks. When specifying the JSR292 (java.lang.invoke + invokedynamic), we spend a lot of time to be sure that reflection behavior and bytecode behavior were aligned. I don't think that trying to separate B and C is possible that late in the game.
How could they possibly be indistinguishable? All the code for causing
public to be restricted in any way has been added for Jigsaw.
>> This changes the basic "export" back to mean "make available for
>> linkage", with no bearing on accessibility. Then you have (at minimum)
>> the following ways to export (or not) a package:
>>
>> 1. No export at compile time, no export but reflection-only at run time
>> (but only for public types & members)
>> 2. Yes export at compile time, yes export at run time (but only for
>> public types & members)
>> 3. Yes export at compile time and run time, including specific grants to
>> access package-private members from targeted modules and/or packages
>>
>> Note: We could add a 1? which is an export at run time but not compile
>> time if the EG agrees; I don't think that is particularly important
>> though as build tools can accomplish this easily enough. I don't want
>> to derail the core point of the discussion on this topic, so I'm leaving
>> it aside for now.
>
> friend accesssibility doesn't solve the issue here, when you want 'strong encapsulation' i.e. no way to use reflection to access members of a not-exported class
You literally *never* want that. By making a class public, users are
saying "I do not want to encapsulate this class, I want everyone to be
able to use it". That is literally what it means. This is impossible
under the current Jigsaw solution.
If you want to prevent reflection being used, then package-private is
the one you want.
>, you should not have to provide the list of all possible existing module.
> I think we want 'export private' (strong encapsulation) and 'export
dynamic' to be available without having a friend list.
Sure, and the same can be said for the Jigsaw solution, but in the
majority of cases, your public classes would be public and for the
others, you definitely want to control who specifically has access.
> And again, changing the meaning of package private is a no go for me, by example for a package like java.lang.invoke, i want the package to be exported, i want strong encapsulation and i want some classes inside the package by example the class that implements the whole method handle hierarchy to share secret (using the package private access).
I'm not sure I follow what you mean here. If you have an internal
package like java.lang.invoke, and you want to share non-public things
to specific JDK-internal targets, you would just list them as friends.
>> This way the Java 8 behavior is retained with no "export dynamic"
>> anywhere in sight. Rather you'd have something like "export private",
>> which you'd only grant for those special cases where a package must be
>> shared to a specific friend. Things not intended to be shared can be
>> safely migrated to the new friend package feature.
>
> I agree that we should have a way to have Java 8 behavior without having to declare every packages with export dynamic,
> see my discussion with mark about specifying a default export behavior.
Yeah, the problem with that is that if you do this, you have defeated
the security mechanism completely. With the friend package solution,
you don't have to defeat the mechanism to have Java 8 behavior, allowing
all existing reflection code to continue to work, which is the entire
point. It still lets you solve the same use cases, just in a different way.
>> No new access limits
>> are introduced - instead old ones are enhanced in what I think is a
>> simple and intuitive way. All the compatibility concerns go away, but
>> we can still use the mechanism to seal up the problems in the JDK with
>> shared code and things which should not be public.
>>
>> We should retain setAccessible() for now for compatibility. We could
>> also introduce a "--safe-mode" flag which disables setAccessible()
>> *across the board* (that is, not just for modular code) without
>> controversy, because all those public members will remain accessible
>> without "help", plus a way is provided to allow access to be volunteered
>> from one package to another (along much the same philosophical lines as
>> MethodHandles.Lookup IMO).
>
> Deprecating setAccessible is another story for me.
I was under the impression that doing so is the whole point of this
encapsulation feature.
--
- DML
From mark.reinhold at oracle.com Tue Jul 19 21:08:08 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Tue, 19 Jul 2016 14:08:08 -0700 (PDT)
Subject: Discussion: #ResourceExistenceAndSize
In-Reply-To: <57866AD3.4020807@redhat.com>
References: <56D85B77.4040607@redhat.com>
<20160713142913.C1C1FB78C7@eggemoggin.niobe.net>
<57866AD3.4020807@redhat.com>
Message-ID: <20160719210808.6AA72B9510@eggemoggin.niobe.net>
2016/7/13 9:22:43 -0700, david.lloyd at redhat.com:
> On 07/13/2016 09:29 AM, mark.reinhold at oracle.com wrote:
>> Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#ResourceExistenceAndSize
>>
>> ...
>>
>> At any rate, I don't see anything here that's inherently part of the
>> design of the module system, so I intend to close this issue as
>> unrelated.
>
> The relation comes in because Class and ClassLoader have a way to open a
> resource with a URL, but Module does not, providing only a
> getResourceAsStream() method. Adding a getResource() method would be
> one solution, but the underlying requirement is really that the resource
> existence and size (and origin) could/should be queriable as well, so
> that was the issue I registered, especially as URL is really not a great
> solution to this problem.
So would a simple Module::getResourceSize(String name) method, which
returns a long value to indicate the size of the named resource, or
else -1 if it doesn't exist, be sufficient?
We could consider adding a similar method alongside the other existing
getResource* methods in java.lang.Class and ClassLoader.
- Mark
From mark.reinhold at oracle.com Tue Jul 19 21:09:08 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Tue, 19 Jul 2016 14:09:08 -0700 (PDT)
Subject: Proposal: #DefaultModule
In-Reply-To: <57866B92.200@redhat.com>
References: <577BC730.1040706@redhat.com>
<20160713143213.CF094B78D7@eggemoggin.niobe.net>
<57866B92.200@redhat.com>
Message-ID: <20160719210908.74B4EB9512@eggemoggin.niobe.net>
2016/7/13 9:25:54 -0700, david.lloyd at redhat.com:
> On 07/13/2016 09:32 AM, mark.reinhold at oracle.com wrote:
>> ...
>>
>> If this is correct then there's a simpler way to reach your goal: Enhance
>> class loaders to have optional names. When the run-time system generates
>> a stack trace or an exception message that mentions a module name and
>> version, if present, then it could also insert the name of that module's
>> class loader, if present. This would allow an external module system to
>> provide better diagnostics without having to change the JPMS design in a
>> fundamental way.
>>
>> Would that meet your needs?
>
> I think that would probably work very well, and fits in nicely with the
> existing architecture of stack trace assembly as I understand it (since
> the class loader can be directly referenced from the Class, which is
> already present). Thanks!
New issue: http://openjdk.java.net/projects/jigsaw/spec/issues/#ClassLoaderNames
- Mark
From mark.reinhold at oracle.com Tue Jul 19 21:10:08 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Tue, 19 Jul 2016 14:10:08 -0700 (PDT)
Subject: Discussion: #MutableConfigurations
In-Reply-To: <57866F90.5020703@redhat.com>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
<57866F90.5020703@redhat.com>
Message-ID: <20160719211008.7690BB9514@eggemoggin.niobe.net>
2016/7/13 9:42:56 -0700, david.lloyd at redhat.com:
> On 07/13/2016 09:30 AM, mark.reinhold at oracle.com wrote:
>> ...
>>
>> I don't think it makes sense for Configurations to be mutable, for the
>> reasons stated above. It may be that the Layer concept needs to be made
>> more dynamic in certain ways, but that's a different matter.
>>
>> I intend to close this issue unless there are strong objections from
>> other EG members.
>
> In that case, other EG members take note: this means that hot deployment
> as the world knows it is over as of Java EE 9, since a module can never
> delete old or establish new dependence relationships. This also means
> that OSGi loses this capability as well, so either OSGi has to be
> redefined with the new restriction, or else OSGi bundles cannot be modules.
First, neither OSGi nor any other existing module system will lose any
capability at all. They can continue to operate just as they do today.
We already have a requirement [1] to support interoperation with such
systems, and so far as I can see it's satisfied by the present design.
Going further to design a "meta" module system that allows other module
systems to be retrofitted to interoperate on an intimate basis with the
platform module system is, as I've written before, a research project
that's clearly outside the scope of this JSR [2].
Second, it's certainly true that the present design does not support
fully-general "hot deployment" in which arbitrary modules can come and go
and be reconfigured at any time. That's intentional, because doing so
would entail enormous complexity and make it incredibly difficult, if not
impossible, to achieve the stated goals of this JSR. That's also why the
agreed requirements mandate not this general capability but, rather, a
more constrained dynamic-configuration capability, which the present
design addresses with the concept of layers.
- Mark
[1] http://openjdk.java.net/projects/jigsaw/spec/reqs/#interoperation
[2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-February/000019.html
From mark.reinhold at oracle.com Tue Jul 19 21:11:08 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Tue, 19 Jul 2016 14:11:08 -0700 (PDT)
Subject: Discussion: #LazyConfigurationAndInstantiation
In-Reply-To: <57867CDE.8000206@redhat.com>
References: <56D7AE09.1030003@redhat.com>
<20160311174751.5399C9E891@eggemoggin.niobe.net>
<56E33EEC.20706@redhat.com>
<20160713143113.C5B45B78D2@eggemoggin.niobe.net>
<57867CDE.8000206@redhat.com>
Message-ID: <20160719211108.78701B9516@eggemoggin.niobe.net>
2016/7/13 10:39:42 -0700, david.lloyd at redhat.com:
> On 07/13/2016 09:31 AM, mark.reinhold at oracle.com wrote:
>> Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#LazyConfigurationAndInstantiation
>>
>> 2016/3/11 13:55:56 -0800, david.lloyd at redhat.com:
>>> ...
>>> Another potenti
>>> issue is requiring that all modules be present, though this is more
>>> closely related to #MutableConfigurations now I think; I suspect this
>>> issue could be mitigated with an acceptable solution to that issue.
>>
>> As I wrote in my reply re. #MutableConfigurations [2], I think this
>> approach is at odds with our goal to provide reliable configuration.
>
> I disagree. "Reliable" is only defined in the agreed-upon requirements
> by stipulating that modules may have dependence relationships which are
> declared.
Fair enough. I'll restate my position to say that #MutableConfigurations
is at odds with the way in which the present design provides reliable
configuration.
> I interpreted that goal simply as an explicit rejection of
> various poorly-defined customized class loader behaviors, and a move
> towards clearer and more predictable behavior. By my interpretation and
> experience, I consider, for example, the ability to change the explicit
> dependence relationships to still be "reliable" as long as the effects
> of doing so are well-defined, just as I consider the behavior of lazy
> class loading to be "reliable" in that it is well-defined and
> predictable, and has rules which make sense (referring to the way that
> classes are loaded, resolved, and initialized in separate phases, which
> allows for lazy on-demand-only progression through those phases but also
> allows for almost completely arbitrary interconnection of classes, while
> remaining basically predictable).
These are all reasonable interpretations of the high-level goal of
reliable configuration.
I make no claim that the present design is the only way to achieve
reliable configuration, for some reasonable definition of "reliable".
It is, however, one way to achieve it.
It might be possible to support reliable configuration, and to satisfy
all of our other agreed requirements, with a design that configures and
instantiates layers lazily rather than eagerly -- though I'm skeptical.
At any rate, that is not the design that we have today.
> The first time that "reliable" is specified in terms of concrete
> behavior is in the SOTMS document, and these issues are being raised
> exactly against the state of the module system, so I think that this
> issue as well as #MutableConfigurations serve to directly challenge the
> validity of the definition of "reliable" which is used by the proposed
> implementation, rather than being invalid due to the presumed validity
> of that definition (which was not agreed upon by the EG).
The present design is meant to satisfy the requirements previously agreed
by this EG. Those requirements do not include any constraints along the
lines of #MutableConfigurations or #LazyConfigurationAndInstantiation.
Your desire for these properties now is no basis for a claim that the
design is unacceptable with respect to previous agreements of the EG.
> Even the JSR definition (which I acknowledge is quite out of date at
> this point) states that OSGi does address the problem of "reliable"
> configuration, despite the fact that the OSGi solution allows for
> dynamic loading and relinking. This is contrary to the definition in
> the SOTMS, and also despite the fact that Jigsaw's more strict
> interpretation of this actually invalidates those behaviors of OSGi,
> were OSGi to try to merge the bundle concept with the Jigsaw module
> concept somehow.
The present design invalidates nothing. OSGi can continue to operate
just as it does today. There is no agreed requirement to support a
hypothetical future merge of OSGi's bundle concept with the platform's
module concept, which is a research project that is clearly outside
the scope of this JSR [1].
- Mark
[1] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-February/000019.html
From mark.reinhold at oracle.com Wed Jul 20 14:33:29 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 20 Jul 2016 07:33:29 -0700
Subject: Me trying to explain the problem behind
#ReflectiveAccessToNonExportedTypes
In-Reply-To: <1028346570.1564026.1468532781557.JavaMail.zimbra@u-pem.fr>
References: <1627574708.1456733.1468507103919.JavaMail.zimbra@u-pem.fr>
<1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
<20160714093304.841168761eggemoggin.niobe.net>
<1028346570.1564026.1468532781557.JavaMail.zimbra@u-pem.fr>
Message-ID: <20160720073329.361423832eggemoggin.niobe.net>
2016/7/14 14:46:21 -0700, Remi Forax <forax at univ-mlv.fr>:
> 2016/07/14 09:33:04 -0700, mark.reinhold at oracle.com:
>> 2016/7/14 8:06:36 -0700, Remi Forax <forax at univ-mlv.fr>:
>>> ...
>>>
>>> While i agree that "export dynamic" is a semantics that jigsaw should
>>> provide, i disagree with the syntax "export dynamic" because for me,
>>> the a non exported package should have this semantics in order to be
>>> backward compatible with existing code.
>>> So we should have 3 way to export or not a package,
>>> 1. don't export at compile time, don't export at runtime
>>> 2. don't export at compile time, export at runtime
>>> 3. export at compile time, export at runtime.
>>>
>>> 2 should be the default, 1 should be use by the package of the JDK (or
>>> any other libs) that want strong security, 3 is for the package that
>>> defines API (that will be maintained forever).
>>
>> A problem with changing the default is that many developers will never
>> think to do anything else, and leave all their packages unnecessarily
>> exported. We have examples of this sort of thing elsewhere in Java,
>> e.g., default public constructors, which have been problematic for
>> years. Do we really want to repeat that kind of mistake?
>>
>> What do you think of allowing wildcards, i.e., `exports dynamic *`?
>> That way a developer could explicitly choose to export everything at
>> run time, but could do so in a very compact way.
>
> Wildcards seems a great idea.
>
> But if we want authors to make a decision, instead of choosing a
> default for them, we should let them to make a decision.
>
> So wildcards should exist in both way 'export dynamic *' or 'export
> noexport *' (or whatever we call it) and we should mandate that a
> module to have an export wildcard declaration.
>
> It's more verbose that the current settings but at least this make the
> choice explicit.
Requiring something to be declared explicitly one way or the other,
rather than offer a safe and sensible default, seems very unlike the
rest of the language.
- Mark
From stef at epardaud.fr Wed Jul 20 15:24:59 2016
From: stef at epardaud.fr (Stephane Epardaud)
Date: Wed, 20 Jul 2016 17:24:59 +0200
Subject: Me trying to explain the problem behind
#ReflectiveAccessToNonExportedTypes
In-Reply-To: <20160720073329.361423832eggemoggin.niobe.net>
References: <1627574708.1456733.1468507103919.JavaMail.zimbra@u-pem.fr>
<1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
<20160714093304.841168761eggemoggin.niobe.net>
<1028346570.1564026.1468532781557.JavaMail.zimbra@u-pem.fr>
<20160720073329.361423832eggemoggin.niobe.net>
Message-ID: <578F97CB.1050203@epardaud.fr>
On 20/07/16 16:33, mark.reinhold at oracle.com wrote:
> Requiring something to be declared explicitly one way or the other,
> rather than offer a safe and sensible default, seems very unlike the
> rest of the language.
I have to say I agree with that, even though I also understand the
problem that non-public packages will cause to adopters (which is why I
proposed a meta-annotation solution to _some_ cases).
The default visibility should indeed be (module-)private, that's what it
is for regular Java declarations, and that's what it should be for
packages now that they can have visibility.
Even if we required that package declarations (in package-info.java) now
be declared with `public` modifiers for that (rather than in the module
descriptor), it would _still_ cause existing packages to become
non-exported when you convert your jar into a module.
If you don't convert your jar to a module all your packages remain
exported. If you do, you have to do _something_ to your packages
whatever solution you pick.
You can say that they should be exported by default, so that existing
applications using reflection keep working, but that's really a bad
default. Sometimes backwards-compatibility requires compromises and
introducing bad defaults, but in this case you already have to do the
manual step of adding a module descriptor, so at this point it is not
unreasonable to have the good default of non-exported modules and teach
people that when they add a module descriptor they also have to decide
and explicitly state which packages to export.
Now, granted I'm already used to that in both Ceylon and OSGi, so
perhaps I'm biased.
From forax at univ-mlv.fr Wed Jul 20 16:40:23 2016
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Wed, 20 Jul 2016 18:40:23 +0200 (CEST)
Subject: Me trying to explain the problem behind
#ReflectiveAccessToNonExportedTypes
In-Reply-To: <20160720073329.361423832eggemoggin.niobe.net>
References: <1627574708.1456733.1468507103919.JavaMail.zimbra@u-pem.fr>
<1105959088.1485118.1468508796360.JavaMail.zimbra@u-pem.fr>
<20160714093304.841168761eggemoggin.niobe.net>
<1028346570.1564026.1468532781557.JavaMail.zimbra@u-pem.fr>
<20160720073329.361423832eggemoggin.niobe.net>
Message-ID: <2141349622.931070.1469032823905.JavaMail.zimbra@u-pem.fr>
----- Mail original -----
> De: "mark reinhold" <mark.reinhold at oracle.com>
> ?: forax at univ-mlv.fr
> Cc: jpms-spec-experts at openjdk.java.net
> Envoy?: Mercredi 20 Juillet 2016 16:33:29
> Objet: Re: Me trying to explain the problem behind #ReflectiveAccessToNonExportedTypes
>
> 2016/7/14 14:46:21 -0700, Remi Forax <forax at univ-mlv.fr>:
> > 2016/07/14 09:33:04 -0700, mark.reinhold at oracle.com:
> >> 2016/7/14 8:06:36 -0700, Remi Forax <forax at univ-mlv.fr>:
> >>> ...
> >>>
> >>> While i agree that "export dynamic" is a semantics that jigsaw should
> >>> provide, i disagree with the syntax "export dynamic" because for me,
> >>> the a non exported package should have this semantics in order to be
> >>> backward compatible with existing code.
> >>> So we should have 3 way to export or not a package,
> >>> 1. don't export at compile time, don't export at runtime
> >>> 2. don't export at compile time, export at runtime
> >>> 3. export at compile time, export at runtime.
> >>>
> >>> 2 should be the default, 1 should be use by the package of the JDK (or
> >>> any other libs) that want strong security, 3 is for the package that
> >>> defines API (that will be maintained forever).
> >>
> >> A problem with changing the default is that many developers will never
> >> think to do anything else, and leave all their packages unnecessarily
> >> exported. We have examples of this sort of thing elsewhere in Java,
> >> e.g., default public constructors, which have been problematic for
> >> years. Do we really want to repeat that kind of mistake?
> >>
> >> What do you think of allowing wildcards, i.e., `exports dynamic *`?
> >> That way a developer could explicitly choose to export everything at
> >> run time, but could do so in a very compact way.
> >
> > Wildcards seems a great idea.
> >
> > But if we want authors to make a decision, instead of choosing a
> > default for them, we should let them to make a decision.
> >
> > So wildcards should exist in both way 'export dynamic *' or 'export
> > noexport *' (or whatever we call it) and we should mandate that a
> > module to have an export wildcard declaration.
> >
> > It's more verbose that the current settings but at least this make the
> > choice explicit.
>
> Requiring something to be declared explicitly one way or the other,
> rather than offer a safe and sensible default, seems very unlike the
> rest of the language.
Choisir c'est renoncer (Andr?e Gide) - To choose is to forsake
There are parts of Java that have no defaults, method calls resolution, default methods. There are parts were the defaults was great in 1995 (or later) but that are not great anymore, fields non-private/non-final by default, inference of type variable defaulting to Object instead of reporting an error. And you have defaults that are still great, unlike C++ you don't have to opt-in to enable reflection.
A great strength of Java, is to have resisted to make backward incompatible changes even for good reason, we have renounced to great features to keep backward compatibility, fully reified generics, structural typing of lambdas. In Java, backward compatibility trump all other considerations. And we have all witnesses of the Python2/Python3 debacle, we even know the consequence of not being backward compatible.
So the only safe and sensible choice is to be to have non-exported packages classes to be available at runtime by default.
And obviously to have a way to declare for a given module that all accesses to a classe of a non exported package are denied at runtime.
>
> - Mark
>
R?mi
From david.lloyd at redhat.com Thu Jul 21 16:05:55 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 21 Jul 2016 11:05:55 -0500
Subject: Discussion: #MutableConfigurations
In-Reply-To: <20160719211008.7690BB9514@eggemoggin.niobe.net>
References: <56D79D56.7040801@redhat.com>
<20160713143013.C3C3AB78CC@eggemoggin.niobe.net>
<57866F90.5020703@redhat.com>
<20160719211008.7690BB9514@eggemoggin.niobe.net>
Message-ID: <6fcfaccf-f81a-9fb3-bb1a-cdff9c60f85f@redhat.com>
On 07/19/2016 04:10 PM, mark.reinhold at oracle.com wrote:
> 2016/7/13 9:42:56 -0700, david.lloyd at redhat.com:
>> On 07/13/2016 09:30 AM, mark.reinhold at oracle.com wrote:
>>> ...
>>>
>>> I don't think it makes sense for Configurations to be mutable, for the
>>> reasons stated above. It may be that the Layer concept needs to be made
>>> more dynamic in certain ways, but that's a different matter.
>>>
>>> I intend to close this issue unless there are strong objections from
>>> other EG members.
>>
>> In that case, other EG members take note: this means that hot deployment
>> as the world knows it is over as of Java EE 9, since a module can never
>> delete old or establish new dependence relationships. This also means
>> that OSGi loses this capability as well, so either OSGi has to be
>> redefined with the new restriction, or else OSGi bundles cannot be modules.
>
> First, neither OSGi nor any other existing module system will lose any
> capability at all. They can continue to operate just as they do today.
> We already have a requirement [1] to support interoperation with such
> systems, and so far as I can see it's satisfied by the present design.
I would like this to be true, but unfortunately, operating as we do
today is not going to be a long-term option for us given
previously-discussed future requirements relating to Java EE 9 (see below).
> Going further to design a "meta" module system that allows other module
> systems to be retrofitted to interoperate on an intimate basis with the
> platform module system is, as I've written before, a research project
> that's clearly outside the scope of this JSR [2].
I disagree that it's "clearly" out of scope; what you've delineated as
"research project" versus design parameters seems arbitrary to me. It's
certainly been made to be out of scope for the current implementation
but it could have easily been a part of the design to begin with.
> Second, it's certainly true that the present design does not support
> fully-general "hot deployment" in which arbitrary modules can come and go
> and be reconfigured at any time.
We cannot support this restriction. Our Java EE container is built on
the ability to support hot deployment; this pronouncement, in
combination with the earlier pronouncement that Java EE 9 will almost
certainly require deployment units to be modules, is an unacceptable
result for us. There must be some way for us to achieve this, either by
adding the necessary flexibility to configurations or by allowing
another way to define and redefine modules outside of the constraints of
the current implementation.
> That's intentional, because doing so
> would entail enormous complexity and make it incredibly difficult, if not
> impossible, to achieve the stated goals of this JSR.
I completely disagree. Class files, as defined since the very start of
Java, meet the exact same goals for classes as are stated in the JPMS
requirements for modules, and they do so by using an incremental
algorithm that allows for classes to be loaded and unloaded at any time.
Our (JBoss) module system does the exact same thing, in a very similar
way, for the same reasons, and still manages to provide the same
requirements (sans accessibility restrictions, which can only be part of
the VM, but which could be accomplished using techniques I've previously
proposed). It's clearly possible to do.
> That's also why the
> agreed requirements mandate not this general capability but, rather, a
> more constrained dynamic-configuration capability, which the present
> design addresses with the concept of layers.
The problem is that with layers being strictly hierarchical and
configurations immutable, it will not be possible for us to deliver a
Java EE 9 compliant server without essentially starting over, and giving
up important capabilities in the process. I don't see how we can
possibly agree to this. We simply must have some way to reconfigure
modules at run time, *and* some way to introduce dependence
relationships with modules from extra-hierarchical layers - in addition
to resolutions to certain other previously-raised issues - in order to
move our technology towards Java EE 9, or else we need the guarantee
that there will be no requirement for Java EE 9 or any future Java EE or
related specification which involves mapping Java EE applications,
modules, and/or their constituent pieces directly to named JDK-style
modules. I more than anyone want to find a way to make this work but I
just don't see any way around it.
> [1] http://openjdk.java.net/projects/jigsaw/spec/reqs/#interoperation
> [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-February/000019.html
--
- DML
From david.lloyd at redhat.com Thu Jul 21 16:45:56 2016
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 21 Jul 2016 11:45:56 -0500
Subject: Discussion: #LazyConfigurationAndInstantiation
In-Reply-To: <20160719211108.78701B9516@eggemoggin.niobe.net>
References: <56D7AE09.1030003@redhat.com>
<20160311174751.5399C9E891@eggemoggin.niobe.net>
<56E33EEC.20706@redhat.com>
<20160713143113.C5B45B78D2@eggemoggin.niobe.net>
<57867CDE.8000206@redhat.com>
<20160719211108.78701B9516@eggemoggin.niobe.net>
Message-ID: <1be6cb2a-4ef3-e37c-5e34-ca777cc7ecb4@redhat.com>
On 07/19/2016 04:11 PM, mark.reinhold at oracle.com wrote:
> 2016/7/13 10:39:42 -0700, david.lloyd at redhat.com:
>> On 07/13/2016 09:31 AM, mark.reinhold at oracle.com wrote:
>>> Reference: http://openjdk.java.net/projects/jigsaw/spec/issues/#LazyConfigurationAndInstantiation
>>>
>>> 2016/3/11 13:55:56 -0800, david.lloyd at redhat.com:
>>>> ...
>>>> Another potenti
>>>> issue is requiring that all modules be present, though this is more
>>>> closely related to #MutableConfigurations now I think; I suspect this
>>>> issue could be mitigated with an acceptable solution to that issue.
>>>
>>> As I wrote in my reply re. #MutableConfigurations [2], I think this
>>> approach is at odds with our goal to provide reliable configuration.
>>
>> I disagree. "Reliable" is only defined in the agreed-upon requirements
>> by stipulating that modules may have dependence relationships which are
>> declared.
>
> Fair enough. I'll restate my position to say that #MutableConfigurations
> is at odds with the way in which the present design provides reliable
> configuration.
Indeed, and this is why these issues have been raised: the current
design is not acceptable as-is.
>> I interpreted that goal simply as an explicit rejection of
>> various poorly-defined customized class loader behaviors, and a move
>> towards clearer and more predictable behavior. By my interpretation and
>> experience, I consider, for example, the ability to change the explicit
>> dependence relationships to still be "reliable" as long as the effects
>> of doing so are well-defined, just as I consider the behavior of lazy
>> class loading to be "reliable" in that it is well-defined and
>> predictable, and has rules which make sense (referring to the way that
>> classes are loaded, resolved, and initialized in separate phases, which
>> allows for lazy on-demand-only progression through those phases but also
>> allows for almost completely arbitrary interconnection of classes, while
>> remaining basically predictable).
>
> These are all reasonable interpretations of the high-level goal of
> reliable configuration.
>
> I make no claim that the present design is the only way to achieve
> reliable configuration, for some reasonable definition of "reliable".
> It is, however, one way to achieve it.
>
> It might be possible to support reliable configuration, and to satisfy
> all of our other agreed requirements, with a design that configures and
> instantiates layers lazily rather than eagerly -- though I'm skeptical.
I think it ought to be discussed. It's my estimation that many of the
parameters of the current design are in excess of the agreed-upon
requirements and can be rolled back to an extent without compromising
what was already agreed upon.
> At any rate, that is not the design that we have today.
Agreed, and that is why we have raised these issues.
>> The first time that "reliable" is specified in terms of concrete
>> behavior is in the SOTMS document, and these issues are being raised
>> exactly against the state of the module system, so I think that this
>> issue as well as #MutableConfigurations serve to directly challenge the
>> validity of the definition of "reliable" which is used by the proposed
>> implementation, rather than being invalid due to the presumed validity
>> of that definition (which was not agreed upon by the EG).
>
> The present design is meant to satisfy the requirements previously agreed
> by this EG. Those requirements do not include any constraints along the
> lines of #MutableConfigurations or #LazyConfigurationAndInstantiation.
> Your desire for these properties now is no basis for a claim that the
> design is unacceptable with respect to previous agreements of the EG.
I can't agree with this logic: it could just as easily be said that
various behaviors of the current design (which exceed the agreed-upon
requirements in various ways) also cannot be justified with respect to
previous agreements, and that no claim that such decisions are immutable
can be valid. In fact my agreement to the requirements was based on the
fact that I found nothing in the requirements that invalidates our
current module system. I do not think it was unreasonable to expect
that we would then have had a subsequent design discussion and
agreement, let alone that requirements may be revised as new information
becomes available, though none of that ever transpired; the issues
process is now the only vehicle for raising design issues or problems
with requirements that may have only become obvious after a prototype
JDK integration has occurred, or requirements that have overreached into
design specifics, and therefore that is the process I am using for all
of these cases.
>> Even the JSR definition (which I acknowledge is quite out of date at
>> this point) states that OSGi does address the problem of "reliable"
>> configuration, despite the fact that the OSGi solution allows for
>> dynamic loading and relinking. This is contrary to the definition in
>> the SOTMS, and also despite the fact that Jigsaw's more strict
>> interpretation of this actually invalidates those behaviors of OSGi,
>> were OSGi to try to merge the bundle concept with the Jigsaw module
>> concept somehow.
>
> The present design invalidates nothing. OSGi can continue to operate
> just as it does today. There is no agreed requirement to support a
> hypothetical future merge of OSGi's bundle concept with the platform's
> module concept, which is a research project that is clearly outside
> the scope of this JSR [1].
Sure, but it is pure disingenuity to imply that such a thing might
somehow be possible in the future when it obviously will not be (without
resolution to these issues). It is more honest to be upfront and say
"this module system is designed in such a way that OSGi-style bundles
can never be mapped to it," i.e. it seems quite likely to me that we are
consciously giving OSGi and other flexible classloader-based module
systems an eventual death sentence if using Jigsaw-style modules will
eventually considered (for example) a security necessity, not to mention
the previously mentioned problems that arise if such a system is the
basis for a Java EE application server.
--
- DML
From pbenedict at apache.org Thu Jul 21 23:02:36 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Thu, 21 Jul 2016 18:02:36 -0500
Subject: Mandated Export: Why?
Message-ID: <CABLGb9wxjE_K1z-+aQXFzaaE5ZeSMACHGZLggcOgqGSPN8dq5Q@mail.gmail.com>
>From the document that describes the binary module format for exports [1]:
0x1000 (ACC_SYNTHETIC)
Indicates that the dependence was not explicitly or implicitly declared in
the source of the module declaration.
0x8000 (ACC_MANDATED)
Indicates the dependence was implicitly declared in the source of the
module declaration.
I'd like to find out why ACC_MANDATED is necessary because it seems
ACC_SYNTHETIC is enough. What's keenly interesting is that the description
of ACC_MANDATED is nearly the same to what the JVMS says about
ACC_SYNTHETIC for a class: "it was generated by a compiler and does not
appear in source code" [2].
It appears something is incorrect with the description of these flags. What
is ACC_MANDATED really for and how it is useful?
[1] http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html#jigsaw-2.3
[2] https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html
Cheers,
Paul
From alex.buckley at oracle.com Thu Jul 21 23:59:19 2016
From: alex.buckley at oracle.com (Alex Buckley)
Date: Thu, 21 Jul 2016 16:59:19 -0700
Subject: Mandated Export: Why?
In-Reply-To: <CABLGb9wxjE_K1z-+aQXFzaaE5ZeSMACHGZLggcOgqGSPN8dq5Q@mail.gmail.com>
References: <CABLGb9wxjE_K1z-+aQXFzaaE5ZeSMACHGZLggcOgqGSPN8dq5Q@mail.gmail.com>
Message-ID: <579161D7.2030606@oracle.com>
This isn't a JPMS issue per se; it's the JPMS following a broader pattern.
The distinction between ACC_SYNTHETIC and ACC_MANDATED was introduced by
the MethodParameters attribute in Java SE 8. It is extremely useful to
be able to distinguish between "something in the class file that a
compiler put there as an implementation artifact, to assist other class
files generated by the same compiler" (ACC_SYNTHETIC) and "something in
the class file that a compiler was required to put there by a JCP
specification, because for whatever reason the thing could not be
denoted in source code" (ACC_MANDATED). No-one should rely on the
presence or semantics of an ACC_SYNTHETIC artifact, but everyone can
rely on the presence and semantics of an ACC_MANDATED artifact. There's
a long history of ACC_SYNTHETIC being set for artifacts as if the former
description is appropriate when in fact the latter description is
appropriate; the prime example is fields of an enum type.
In principle, every artifact in the ClassFile structure should allow its
reason-for-being to be flagged. For modules, it's clear that the
JLS-defined dependency on java.base should be ACC_MANDATED, while a
dependency added by a tool (because the tool added classes to a module)
would be ACC_SYNTHETIC. There aren't yet any exports which would be
ACC_MANDATED, but exports added by a tool as in the previous sentence
would legitimately be ACC_SYNTHETIC.
Alex
On 7/21/2016 4:02 PM, Paul Benedict wrote:
> From the document that describes the binary module format for exports [1]:
>
> 0x1000 (ACC_SYNTHETIC)
> Indicates that the dependence was not explicitly or implicitly declared in
> the source of the module declaration.
>
> 0x8000 (ACC_MANDATED)
> Indicates the dependence was implicitly declared in the source of the
> module declaration.
>
> I'd like to find out why ACC_MANDATED is necessary because it seems
> ACC_SYNTHETIC is enough. What's keenly interesting is that the description
> of ACC_MANDATED is nearly the same to what the JVMS says about
> ACC_SYNTHETIC for a class: "it was generated by a compiler and does not
> appear in source code" [2].
>
> It appears something is incorrect with the description of these flags. What
> is ACC_MANDATED really for and how it is useful?
>
> [1] http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html#jigsaw-2.3
> [2] https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html
>
> Cheers,
> Paul
>
From scolebourne at joda.org Wed Jul 27 09:57:54 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Wed, 27 Jul 2016 10:57:54 +0100
Subject: Issue #ServiceLoaderEnhancement
In-Reply-To: <20160713180613.27999B7953@eggemoggin.niobe.net>
References: <CACzrW9Cp0tpZnU-wF3a+GYg2RBU=LeHixqAiZSFZyJjA10kawg@mail.gmail.com>
<20160713180613.27999B7953@eggemoggin.niobe.net>
Message-ID: <CACzrW9BXspz3LF-AiQCHQoapXNwghWzwkx5vKiQ9xKZTKnAZRA@mail.gmail.com>
On 13 July 2016 at 19:06, <mark.reinhold at oracle.com> wrote:
> 2016/7/1 2:27:39 -0700, Stephen Colebourne <scolebourne at joda.org>:
>> Firstly, I propose that all reflection by ServiceLoader should use
>> setAccessible(true) to allow providers to be non-public.
>
> We could do this for service providers defined in modules, but not for
> those specified via the META-INF/services convention. The problem with
> the latter is that if ServiceLoader could instantiate an arbitrary
> non-public class then an adversary could spoof it into doing that simply
> by placing an appropriate META-INF/services configuration file on the
> class path. That's not possible for providers in modules, since the
> declaration of the provider and the provider itself are always tightly
> associated with each other, in the same artifact, so we can relax the
> restriction in that case.
I'm happy with this restriction if I understand it correctly, ie. that
access to non-public is allowed only if the provider is specified
using the module-info "provides" section, and not via
META-INF/services.
>> Secondly, I propose that if no public no-args constructor is found on
>> the provider class, ServiceLoader should locate a public static
>> constant named "INSTANCE" and use that instead.
>
> That's reasonable, though we don't have much precedence for name-based
> conventions in the SE platform itself.
> ...
> An alternative is to take an explicitly-declarative approach, defining
> something like a @Provides annotation to identify the constructors,
> factories, or fields that construct or contain actual provider objects.
> I'll give this some thought.
If it is an annotation, I would not expect to also declare it in
module-info, as that would be redundant. But as a solution it would be
fairly neat.
There is a more explicit alternative to the module-info defined approach:
module org.threeten.extra {
requires java.base;
exports org.threeten.extra.*;
provides java.time.chrono.Chronology with
org.threeten.extra.chrono.CopticChronology::INSTANCE
}
The format would be ::staticConstant or ::staticMethod, using a syntax
along the lines of Java 8 method references. This would precisely
specify the desired constant/method without needing to add a
name-based convention or search. It would also allow a class to have
multiple methods/constants defining the same service. This would work
with META-INF/services as well AFAICT.
>> Thirdly, it should be possible to access the set of provider classes
>> without instantiating them. This would allow applications to use the
>> module-info design for services without using ServiceLoader, eg to
>> reflect on the provider classes, perhaps to search for annotations.
>
> Most of the services defined and used in the JDK itself follow this
> advice, but I accept that many others out in the world do not, so at
> this point people reasonably want to be able to filter providers based
> on types and annotations thereon. This should be straightforward; I'll
> work up a proposal.
Great, thanks
Stephen
From matcdac at gmail.com Thu Jul 28 06:57:32 2016
From: matcdac at gmail.com (Prakhar Makhija)
Date: Thu, 28 Jul 2016 12:27:32 +0530
Subject: Issue #ServiceLoaderEnhancement
In-Reply-To: <CACzrW9BXspz3LF-AiQCHQoapXNwghWzwkx5vKiQ9xKZTKnAZRA@mail.gmail.com>
References: <CACzrW9Cp0tpZnU-wF3a+GYg2RBU=LeHixqAiZSFZyJjA10kawg@mail.gmail.com>
<20160713180613.27999B7953@eggemoggin.niobe.net>
<CACzrW9BXspz3LF-AiQCHQoapXNwghWzwkx5vKiQ9xKZTKnAZRA@mail.gmail.com>
Message-ID: <CAPu805p-oaEXDd8wEPieUbCAs98nGNs4sdMS0s+aA83DcQ+sAA@mail.gmail.com>
I am seeing "provides" in module, which was not talked about in the keynote
session of Java 9 by Mark. Could you please describe what it is.
Looking forward to hear from you.
Thanks
Best Wishes & Regards,
Prakhar Makhija
On Jul 27, 2016 3:28 PM, "Stephen Colebourne" <scolebourne at joda.org> wrote:
> On 13 July 2016 at 19:06, <mark.reinhold at oracle.com> wrote:
> > 2016/7/1 2:27:39 -0700, Stephen Colebourne <scolebourne at joda.org>:
> >> Firstly, I propose that all reflection by ServiceLoader should use
> >> setAccessible(true) to allow providers to be non-public.
> >
> > We could do this for service providers defined in modules, but not for
> > those specified via the META-INF/services convention. The problem with
> > the latter is that if ServiceLoader could instantiate an arbitrary
> > non-public class then an adversary could spoof it into doing that simply
> > by placing an appropriate META-INF/services configuration file on the
> > class path. That's not possible for providers in modules, since the
> > declaration of the provider and the provider itself are always tightly
> > associated with each other, in the same artifact, so we can relax the
> > restriction in that case.
>
> I'm happy with this restriction if I understand it correctly, ie. that
> access to non-public is allowed only if the provider is specified
> using the module-info "provides" section, and not via
> META-INF/services.
>
> >> Secondly, I propose that if no public no-args constructor is found on
> >> the provider class, ServiceLoader should locate a public static
> >> constant named "INSTANCE" and use that instead.
> >
> > That's reasonable, though we don't have much precedence for name-based
> > conventions in the SE platform itself.
> > ...
> > An alternative is to take an explicitly-declarative approach, defining
> > something like a @Provides annotation to identify the constructors,
> > factories, or fields that construct or contain actual provider objects.
> > I'll give this some thought.
>
> If it is an annotation, I would not expect to also declare it in
> module-info, as that would be redundant. But as a solution it would be
> fairly neat.
>
> There is a more explicit alternative to the module-info defined approach:
>
> module org.threeten.extra {
> requires java.base;
> exports org.threeten.extra.*;
> provides java.time.chrono.Chronology with
> org.threeten.extra.chrono.CopticChronology::INSTANCE
> }
>
> The format would be ::staticConstant or ::staticMethod, using a syntax
> along the lines of Java 8 method references. This would precisely
> specify the desired constant/method without needing to add a
> name-based convention or search. It would also allow a class to have
> multiple methods/constants defining the same service. This would work
> with META-INF/services as well AFAICT.
>
> >> Thirdly, it should be possible to access the set of provider classes
> >> without instantiating them. This would allow applications to use the
> >> module-info design for services without using ServiceLoader, eg to
> >> reflect on the provider classes, perhaps to search for annotations.
> >
> > Most of the services defined and used in the JDK itself follow this
> > advice, but I accept that many others out in the world do not, so at
> > this point people reasonably want to be able to filter providers based
> > on types and annotations thereon. This should be straightforward; I'll
> > work up a proposal.
>
> Great, thanks
>
> Stephen
>
From Alan.Bateman at oracle.com Thu Jul 28 07:01:42 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 28 Jul 2016 08:01:42 +0100
Subject: Issue #ServiceLoaderEnhancement
In-Reply-To: <CAPu805p-oaEXDd8wEPieUbCAs98nGNs4sdMS0s+aA83DcQ+sAA@mail.gmail.com>
References: <CACzrW9Cp0tpZnU-wF3a+GYg2RBU=LeHixqAiZSFZyJjA10kawg@mail.gmail.com>
<20160713180613.27999B7953@eggemoggin.niobe.net>
<CACzrW9BXspz3LF-AiQCHQoapXNwghWzwkx5vKiQ9xKZTKnAZRA@mail.gmail.com>
<CAPu805p-oaEXDd8wEPieUbCAs98nGNs4sdMS0s+aA83DcQ+sAA@mail.gmail.com>
Message-ID: <b9801e2d-c816-90aa-9833-a7a74b1e1112@oracle.com>
On 28/07/2016 07:57, Prakhar Makhija wrote:
> I am seeing "provides" in module, which was not talked about in the keynote
> session of Java 9 by Mark. Could you please describe what it is.
>
>
The Services section [1] in the SOTMS document provides a good introduction.
-Alan
[1] http://openjdk.java.net/projects/jigsaw/spec/sotms/#services